In the TERMINET project, Karolinska Institutet (KI) participates via the Health Informatics Centre (HIC) as a use case partner, together with the Swedish SME Alteruna. Alteruna develops training simulators in virtual reality (VR) for surgery and medical treatments targeting both teams and individuals. Their solution supports mobile VR-sets and hand interaction through their own semi-physical middleware solution and enables multiplayer scenarios. Thereby, it allows the integration, not only of the simulators, but also of multiplayer capabilities into the TERMINET architecture.
The use case led by KI and Alteruna will support two of the key technologies of the TERMINET project.
Federated learning for gesture recognition: Firstly, we will adapt a federated learning framework to the gesture recognition solution of the semi-physical hand interaction. The ”weights” of the individual (local) training sessions will be uploaded to a central server and processed there. The new version of the hand interaction algorithm will then be pushed out to the clients.
Edge node computing to render video streams: The other key technology is edge node computing. We will make use of the computing power of the local edge nodes to reduce network latency. It will be especially important for our use case, where we will use video streaming instead of web applications and transitional data only. To render the video stream in the cloud, instead of from the VR device, and to render it locally in the edge node, will assumedly reduce network latency. The video stream will be interactive in the sense that the web browser user will interact in the VR-session by using the web RTC protocol.