VR, AR and 3D Simulation
Various VR, AR and 3D web applications for the navigation of surgical procedures and teaching are being developed as part of research projects, final theses and project work in the Computer Science and Biomedical Engineering degree programmes. While AR technologies are aimed at improving the perception of surgeons for improved precision in open surgery, VR and web simulations are primarily concerned with the virtualisation of technologies that are difficult to access, such as the rooms and equipment used in radiology or radio-oncology.
Mixed reality in surgery
Projects and funding:
- AIARLiver: Internal (2021 - 2024)
- INMOTION: STMWK Bavaria (2024 - 2027)
- AIARConcepts: Internal (2024 - 2027)
Clinical partner: Agatharied Hospital, Bayreuth Clinical Centre, LAKUMED Hospital Landshut-Achdorf
Industrial partners: Cariad SE, Corscience GmbH, Cosinuss GmbH, Infineon AG, mediCAD Hectec GmbH
Professors:
Prof. Dr Stefanie Remmele (Image Processing & Synthesis)
Prof. Dr Christopher Auer (Mixed Reality)
Prof. Dr Eduard Kromer (AI Algorithms)
Prof. Dr Norbert Babel (3D Printing)
Prof. Dr Aida Anetsberger (Medical support)
Research assistants: Serouj Khajarian, Michael Schwimmbeck
This research focus area investigates technologies to support surgeons using mixed reality technologies that allow target and risk structures of an intervention to be superimposed on the patient. The focus is on AI-supported algorithms, for example for tracking and the precise visualisation of models on the patient (registration) and concepts for implementation in medical applications.
The Free State of Bavaria is funding the "INMOTION - Innovative Motion Tracking Technologies in Medical Applications" initiative as part of the 7th funding round of the programme to promote applied research and development at HAWs and THs with a funding amount of €714,977. The university is supporting the project by funding a qualification position to set up the research group (AIARConcepts).
Rapid Room Planning - Planning radiological rooms
Contact: Prof Dr Stefanie Remmele
Master project: Artur Maleta
Partner: Philips DXR Hamburg
The application was developed in co-operation with Philips DXR in Hamburg and serves to support the early phases of a spatial planning process, in which often only paper sketches or manual digital sketches are available as a basis for discussion. It is also intended to facilitate the creation of detailed plans as a project progresses. Ideas for possible room layouts should be directly sketchable virtually with real device models and on a real scale. In addition to 2D room layout planning (object arrangement) and 3D verification (visual inspection) of the structure, the room layout can be experienced in VR to make it more tangible for the user and reduce planning errors (e.g. regarding distances or ceiling height).
Cardboard excursions in medical rooms
Contact: Prof. Dr Christopher Auer, Prof. Dr Stefanie Remmele
Student Project Computer Science 2022
Partner: EBM Papst Landshut
Large medical devices such as linear accelerators, CT, MRI and X-ray machines take up a lot of space and are expensive to purchase. At the same time, it is important to understand how they work in order to be able to evaluate them for use. Two-dimensional representations, e.g. on lecture slides, even if they are well made, can only reproduce the dimensions and spatial arrangements of the components of the devices with difficulty.
In a study project, a team of computer science students developed a VR application that allows the functionality and components of an X-ray machine, LINACs, CTs and MRIs to be experienced spatially, thus further deepening the lecture topics through a new perspective. Particular emphasis was placed on user-friendliness to ensure a quick introduction to the application. In addition, the smartphone VR glasses used ("Cardboard VR") enable uncomplicated use in the lecture theatre, without any cables or complicated set-up.
EBM Papst in Landshut sponsored over 40 Cardboard headsets for the project!
Tools for the development of AR apps
Contact: Prof Dr Stefanie Remmele
BMT project work (2019): T. Feulner, M. Kaiser, R. Stolz
Augmented reality applications are playing an increasingly important role in many industrial sectors. At the same time, a growing number of development tools, assets and libraries are entering the market.
In summer 2019, a project group compared various strategies for developing augmented reality apps (including Unity, Vuforia, Wikitude, Aframe, AR.js, ...). The project team developed an interactive simulation of an X-ray tube for the comparison and compared the development effort, robustness of the application and user satisfaction. You are welcome to try it out for yourself here (see image).
3D simulations for teaching
Contact: Prof Dr Stefanie Remmele
Project/Hiwi/thesis: Artur Maleta, Roland Stolz, Julian Fischer
As part of a smartVHB grant and various student research projects in the medical technology laboratory, various 3D web simulations have been developed that enable experimentation at home in digitally supported teaching. Here, the influence of electrical parameters on the
- parameters of electric and magnetic fields can be investigated or
- patients can be examined in MRI, X-ray and CT scanners.
The simulations are freely accessible at Link.