Abstract:

Graphical visualization systems are a common clinical tool for displaying digital images and three-dimensional volumetric data. These systems provide a broad spectrum of information to support physicians in their clinical routine. For example, the field of radiology enjoys unrestricted options for interaction with the data, since information is pre-recorded and available entirely in digital form. However, some fields, such as microsurgery, do not benefit from this yet. Microscopes, endoscopes, and laparoscopes show the surgical site as it is. To allow free data manipulation and information fusion, 3D digitization of surgical sites is required. We aimed to find the number of cameras needed to add this functionality to surgical microscopes. For this, we performed in silico simulations of the 3D reconstruction of representative models of microsurgical sites with different numbers of cameras in narrow-baseline setups. Our results show that eight independent camera views are preferable, while at least four are necessary for a digital surgical site. In most cases, eight cameras allow the reconstruction of over 99% of the visible part. With four cameras, still over 95% can be achieved. This answers one of the key questions for the development of a prototype microscope. In future, such a system can provide functionality which is unattainable today

Abstract:

Contemporary surgical microscope systems have excellent optical properties but some desirable features re- main unavailable. The number of co-observers is currently re- stricted, by spatial and optical limitations, to only two. More- over, ergonomics poses are a problem: Current microscope systems impede free movement and sometimes demand that surgeons take uncomfortable postures over long periods of time. To rectify these issues, some companies developed surgi- cal microscope systems based on a streaming approach. These systems remove some of the limitations. Multi-observer po- sitions, for example, are not independent from each other, for example. In order to overcome the aforementioned limitations, we are currently developing an approach for the next genera- tion of surgical microscope: Namely the fully digital surgi- cal microscope, where the current observation system is re- placed with a camera array, allowing real-time 3D reconstruc- tion of surgical scenes and, consequently, the rendering of al- most unlimited views for multiple observers. These digital mi- croscopes could make the perspective through the microscope unnecessary allowing the surgeon to move freely and work in more comfortable postures. The requirements on the camera array in such a system have to be determined. For this purpose, we propose of estimation the minimal number of cameras and their positions needed for the 3D reconstruction of microsurgi- cal scenes. The method of estimation is based on the require- ments for the 3D reconstruction. Within the MATLAB simu- lation environment, we have developed a 3D model of a mi- crosurgical scene, used for the determination of the number of required cameras. In a next step a small, compact and cost- ef cient s ystem w ith f ew o pto-mechanical c omponents could be manufactured.

Abstract:

Existing methods for outgoing quality control of manufactured parts are accurate, but slow and expensive. With the ever increasing popularity of direct digital manufacturing, a future need for fast and automated inspection of additively manufactured parts in large volumes is expected. Leveraging the rapidly improving processing power and sensing capability of consumer smartphones and tablets, their use to fill a projected gap in rapid, automatic, low-cost part inspection is envisioned. This requires investigation of the feasibility of the comparison of physical 3D objects with ground truth computer aided design (CAD) models on mobile consumer devices. To avoid computationally expensive direct comparison in 3D, an analysis-by-synthesis approach to detection of real object deviation from CAD models in 2D images is proposed, based on comparison from different views and using mobile device camera pose estimation from AR tracking. A mobile testbed for application of this approach to different use-cases was developed, using a Samsung Galaxy S20+ smartphone, the modelbased tracking library VisionLib and the game engine Unity. The testbed supports integration of custom image processing pipelines, using OpenCV, for detection and comparison of use-case specific features between real objects and ground truth CAD models. The testbed implementation was verified through tests based on the example use-case of straight edge length measurement on 3D-printed polymer parts. The tests have proven the viability and feasibility of 2D image-based 3D object comparison on mobile devices, using an analysisby- synthesis approach based on AR-tracking. A key limitation of this approach arises from the requirement for robust trackability of 3D objects of interest using mode-based tracking. In future, the testbed can be used to investigate the application of this comparison approach to a range of different quality control use-cases. Transfer of the comparison approach to seemingly unrelated use-cases, e.g. in medicine, with the same basic need for 3D object comparison is also conceivable.

Abstract:

Despite their maturity, contemporary surgical microscope systems still leave aspects to be desired. The number of co-observers is currently restricted, by spatial and optical limitations, to only two. The co-observers also suffer from decreased stereoscopic depth perception. Moreover, ergonomics pose a problem, with current microscope systems impeding mobility and sometimes demanding that surgeons take uncomfortable postures over lengthy periods of time. The next evolutionary step in surgical microscope design should hence improve surgical microscope ergonomics and remove the limitations on co-viewing. A promising approach to this is the fully digital surgical microscope, where the current observation systems are replaced with a camera array; allowing real-time 3D reconstruction of surgical scenes and, consequently, the rendering of almost unlimited views for different observers. Digital microscopes could thus also relieve the surgeon of the necessity to look through the microscope, hence allowing him to move more freely and take more comfortable postures. The requirements on the camera array in such a system have yet to be determined. To this effort, we have have contributed an estimation of the minimal number and positions of cameras needed for the 3D reconstruction of microsurgical sites, based on the simulated reconstruction of reference models. With the simulation program we developed for this purpose, the requirements for a camera array in a compact and cost effective fully digital surgical microscope could be estimated.