Notifications
Insight
Showcase
24
0

Title: Insight

HTC Vive Sponsor Award Recipient
Finalist for Overall Best VR Experience

About The Team

Brian Cohn, Lead Scientist
Ali Marjaninejad, Data Scientist
Dilan Shah, Programmer/Project Manager
Martin Shapiro, Health Strategist
Serhan Ulkumen, 3D Designer/Programmer

Inspiration

Parkinson’s disease is a slowly progressing neurodegenerative disorder with symptoms including tremor, limb rigidity, slowness of movements, and problems with balance. Advancement of the disease can severely impact the quality of life from physical disability to depression. We developed Insight, a VR patient-centric platform for Parkinson’s disease patients and their family. Insight empowers Parkinson’s patients to live a life of their choosing rather than that of their disease.

What it does

Insight augments Parkinson’s patient’s quality of life by integrating into their existing clinical care our VR assessment, management, and health education. Patients typically see a physician to monitor disease progression and adjust medication and rehabilitation therapy at set interval clinic visits based on symptoms. Insight stays with the patient throughout their life, continuously assessing the user between visits and aiding their providers at the clinic.

Our platform draws upon current care via third-party health information such as medical records and movement data collected in VR to create an assessment of the patient’s health status, provide personalized rehabilitation exercises, and guide the physician team in data-driven decision making. Before the patient begins rehabilitation exercises, he/she touches the wind chime in the house which then transfers symptoms over to the virtual world. For the remainder of the experience, the wind chime movement and sound will signify the symptoms of the patient while the patient’s movements will now be tremor free. The patient will then be guided through evidence-based personalized rehabilitation exercises that while improving physical function will also collect data for disease progression assessment. At the end of the assessment, the patient receives an overview of their current health status including medications, Insight’s health score derived from symptom measurements, and an option to contact a physician through telemedicine. This physician will have a report generated by Insight that includes the collected symptom information.

Insight provides a platform for voluntary data collection and we have integrated neural network integration to allow for artificial intelligence deep learning of prognostic and treatment recommendations going forward.

How we built it

The Insight Patient Data platform was built using a combination of Unity game engine and data analysis tools Matlab and Python.

Low Pass Filter for Hand Tremor
To live with a tremor is to live with constant motion, disruption, and often frustration. We designed a tunable Filter that lets a patient have visible feedback of their hand operating completely smoothly, almost as if they have no visible symptoms of Parkinson's. While they may be shaking vigorously, our filter "cures" hand movement in VR. We took a moving average of the last N controller positions and calculated the average position and Quaternion. The SmoothedHandScript that we attached to the user's hand model is responsible for collecting this data and only showing the user the smoothed output, and we expose the window size to the GUI so the VR proctor can dynamically change the filter settings, so the movement is smooth & responsive.

Environment
The inspiration for the environment was heavily influenced by one mentor, Hannah Luxenborg, who explained that rather than an art direction similar to a clinic, we should aim for a soothing ambiance. A majority of the models were modeled and created in Maya by Serhan Ulkumen. Using Unity's terrain system, we sculpted a landscape using a heightmap and trees.

Data Analysis and Reporting
Data is useless without Insight. We designed a frequency analysis of tremor that compares tremor in different dimensions (X = triceps/biceps tremor, Y = shoulder tremor, and Z = wrist abduction/adduction), so a clinician can see the specifics of how the disease has progressed or responded to different medications. A script called RawTrialDataSaver is responsible for taking the experimental trial data and saving it to a local CSV file. The appearance of that file is observed by the Python script that uses MatPlotLib to produce a PDF document for the doctor, and a PNG file with a visualization for the patient. For patients who want to use the most optimal medication, the Insight system provides incredible value in comparing treatment outcomes, tracking disease progression, and contextualizing the score of the patient against other patients. We include an in-game visualization of one of the metrics that the patient can observe after the assessment is complete, with a score that ranks them against different levels of normalized tremor behavior

Hardware Used

HTC Vive

Attribution

Asset Store
SteamVR
Frames Pack
Post-Processing Stack
Tools Used
For analysis we used: NumPy and Pandas
For visualization: MatPlotLib
Textures for Models
CGTextures
Images, Video, and Media
Generated by our team
Thank you for visiting our repo. Please reach out to us if you are interested in learning more!

Built With
c#
shaderlab
html
python
hlsl
glsl
tex
matlab
matplotlib

Try it out

GitHub Repo

Dilan Shah
VR Creator/Engineer | Oculus Start Developer - Producer
4
Comments