Ghost Hands VR System
An innovative Unity-based VR training system that guides precise hand movements through ghosted visual feedback for medical education
Project Overview
As part of an ongoing fellowship with the Omaha Virtual Reality Pipeline (OVRP), I'm collaborating with faculty and staff from the University of Nebraska Medical Center (UNMC), as well as industry professionals, to prototype an educational VR simulation for medical training. This project is still in development and focuses on creating a safe, repeatable environment for teaching intricate hand movements and techniques—especially for high-stakes procedures like intubation.
Traditional training often relies on observation and verbal guidance, which can fall short in conveying spatial coordination and muscle memory. This Ghost Hands system addresses that gap by offering visual, spatial guidance through a "follow-me" mechanic that lets trainees mirror expert actions in real-time. While still under development, this prototype is being explored for integration into medical education to support improved learning outcomes and reduce time-to-competence for complex procedures.
My Role & Responsibilities
- Lead Designer & Developer: Responsible for the entire design system and front-end implementation
- Technical Implementation: Built the prototype ghost hand mechanic / training system framework using Unity and C#, integrating seamlessly with XR Interaction Toolkit
- User Experience Design: Developed intuitive calibration workflows and visual feedback systems for optimal learning
- Project Management: Coordinated with stakeholders and managed project timeline
- Cross-functional Collaboration: Worked directly with medical professionals and the development team to iterate on system requirements and validate training effectiveness
Key Features
Step-by-Step Guidance
Precise ghost hand positioning system that guides users through structured training sequences one step at a time
Interactive Trigger System
Game objects with interactable triggers that detect when users correctly match ghost hand positions
Ghost Handler Scripts
Custom scripts that manage step progression and integrate with dialogue menu systems for seamless user experience
Configurable Timing
Required time settings for each step's collider detection, ensuring users maintain proper positioning before advancing
Haptic Feedback System
Customizable haptic amplitude and duration settings for controller feedback when steps are completed correctly
Technology Stack
Challenges & Solutions
- User Experience: Created intuitive navigation to guide users through the procedure outside of providing them with just the correct steps, allowing for better playability and engagement
- Scalability: Designed modular architecture to handle growing user base
- Training Validation: Collaborated with medical professionals to ensure haptic timing and positioning requirements meet real-world procedural standards