Back to Projects List
- Steve Pieper (Isomics)
- Gabor Fichtinger (Queens)
- YOU
Investigate less expensive but still good tracking for IGT. Modern AR/VR devices use inside-out tracking with IMUs, cameras, lidar, and other sensors (e.g. in phones and glasses). These are small enough and becoming (maybe?) good enough to consider for IGT. Would these be options for NousNav or the SlicerTMS projects?
- Make a plan for determining accuracy and utility of options
- Plan any implementation efforts or further experiments
- Consider issues like form-factor, sterilization, re-usability, etc.
- Survey developments in the field pushed by AR/VR devices
- Look at any prototypes, e.g. Steve's WebXR experiment
- Determine next steps
- Improved demo to work with https using letsencript on Google Cloud virtual machine running Slicer
- Added touch screen events to control attributes of Slicer model (brighter yellow when touching the screen).
- Gave demos to colleagues at the Wednesday IGT breakout and discussed the tradeoffs of intrinsict tracking vs EM and extrinsic optical tracking
- Explore the use of phone-based tracking for SlicerTMS research
- Experiment with local rendering and touch interactions on phone mixed with remote rendering and computation on CPU/GPU with Slicer
- Consider developing native phone app to avoid https performance overhead vs upgrading Slicer's web server to support web sockets for faster performance
- Brainstorm about other applications of this technology
- Monitor developments of intrinsic tracking systems in non-phone form factors for use in other tracking scenarios (e.g. in IGT)
- Phone as a 6 DOF controller demo (no audio). Demo uses moto g100 Android phone that includes Qualcom chips for tracking with WebXR API in JavaScript to communicate with 3D Slicer web server.
- https://github.com/pieper/SlicerWeb/blob/master/WebServer/docroot/WebXR-controller/index.html
- https://immersive-web.github.io/webxr-samples/
- https://www.qualcomm.com/research/cognitive-technologies/immersive-experiences/augmented-reality
- https://www.qualcomm.com/news/onq/2021/08/16/how-snapdragon-xr1-powers-lenovo-thinkreality-a3-smart-glasses-and-moto-g100