Karolinska Institutet (KI) is a medical university providing research and education in medicine and healthcare-related disciplines. In TERMINET, KI participates via the Health Informatics Centre (HIC) as a use case partner, together with the Swedish SME Alteruna. Alteruna develops training simulators in virtual reality (VR) for surgery and medical treatments targeting both teams and individuals. Their solution, the Alteruna TeamSimulater application, is running in head mounted displays (Oculus Quest2). Participants in a virtual reality session can further participate in training without the head mounted display (HMD) through a web browser using any mobile device or computer.
Problems with the current solution are however two-fold. First, instructors, teachers, supervisors, or other learners joining the virtual scene from mobile devices will, depending on their geographical location and connection speed experience severe delays of the video streams. Second, using controllers for training surgical interventions can be perceived as cumbersome and non-natural. However, hands interaction with surgical instruments requires fine-grained and accurate gesture recognition to interpret.
TERMINET is expected to contribute to a solution to the above-mentioned problems through the following activities: First, load balancing based on latency and distance detection will help to choose the most effective streaming server for the respective observer window/web application and thereby provide acceptable latency rates for each individual end user. Second, the TERMINET federated learning components are expected to improve the accuracy of gesture recognition.
KI has thus far contributed to 1) defining the requirements of a TERMINET-integrated VR training solution, 2) defining test and validation protocols for three different test scenarios, and will 3) contribute to the technical validation of the integrated platform as well as perform end user evaluations. Thereby, we will be able to capture quantitative measures and to determine the perceived quality of the service.