Abstract:

Modern operation microscopes deliver optical excellent pictures to the surgeon. Beam splitting allows for multiple viewers to the scene. However, the resulting image is limited in perspective. Additonally, preoperative data can only be shown as overlay on the surface or on additional external screens. As a solution a fully digital microscope is proposed. A mulicamerasetup is proposed to create a digital 3D clone of the operational situs. This allows for a multitude of view angles, and a complete integration of preoperative data with the observed scene. The data can be presented with an head-mounted display. Yet, a fully virtual approach is not suitable, interaction with tools and persons in the operation room has to be ensured. This work tackles this problem by proposing a mixed reality approach as an environment for the above data. Since see-through-HMDs, like the Hololens, don’t offer a solution for opaque pixels, a video see through approach was chosen. As an HMD the HTC Vive Pro was used, since it comes with 2 front cameras. The mixed reality was designed using the engine Unity and their plugins SteamVR, OpenVR and SRworks. The captured pictures of the frontcameras are displayed on a virtual plane. A switch to a fully virtual scene was implemented, in which 3D-models are created and shown in realtime. The mixed-reality was evaluated by a userstudy. A general suitability of the approach was shown, however it lacks on several important aspects. These include image quality, delay from captured pictures and spacial shift between real word and mixed reality. The problems are mostly due to the low quality frontcameras of the Vive Pro. The approach needs further improvement for usage in clinical application. A realtime creation and rendering of 3D-Data has been succesfully implemented, however reached framerates still have to be improved.