VRbal is a smart VR experience that uses machine learning to help users prepare for an anxiety inducing event. Using step-by-step tasks, VRbal gradually exposes the user to the situation.
Stuttering and other communication disorders affects over 1% of the population, barring those affected from situations like interviews, speeches, and meetings. Current treatments involve time-intensive and costly therapies that don't work reliably. We built a Virtual Reality solution that uses Artificial Intelligence to learn and predict user levels, providing step-by-step training that the speech-impaired can undergo daily.
In interviews with speech pathologists and sufferers, we learned that stuttering depends on the perception of the listener and the quietness of the surroundings. Speech therapists use systematic desensitization, a gradual replacement of fear with relaxation training to counter condition. Competitors like Mimerse, CleVR, and SimSensei require supervision or lack customization. Instead we use AI to track training progress and physiological state, creating a desensitization regiment customized for individual users in training speed and difficulty.
In the program, the user is first greeted by the AI, who asks her for the relaxation atmosphere (forest or beach) and activity (slow reading or deep breathing) she prefers. Using natural language comprehension, the AI loads the selected scene, then asks the user for her level of anxiety, using this info to run the simulation scene where she practices speech. It can be easy (no people, quiet room) or various degree more difficult (people, noisy).
Technologies used:
Ocuclus Rift system,
Watson SDK for Unity,
Unity3D environment,
Blender 3D modeling,
Kodak sp360 camera,
Grove GSR sensor,
We had 19 user tests including stuttering individuals for 1st, 2nd, or 3rd stage of our prototypes. Their feedback led us to lower the fidelity of the relaxation, improve AI voice, reduce blur, speed up AI, reduce unreliable GSR use, and increase fidelity of the speech simulation.
On demo day 2018 we presented a workflow and brochure for mimicking a real life training session.
Supported by Verizon Connected Futures III. Paper at ACM Artech. More web doc here.