Andrew Beall & Peter Schlueer
The clock is ticking on already funded research studies - but how to conduct them with subjects confined to their homes during the ongoing pandemic? Here is what prominent researchers said in a recent WorldViz webinar panel discussion.
WorldViz Chief Scientist Dr. Andrew Beall and Solutions Architect Sado Rabaudi interviewed Psychology and Communications researchers Dr. Gary Bente (Michigan State University), Dr. Mary Hegarty (University of California Santa Barbara), Dr. Sarah Creem-Regehr (University of Utah), and John Allen (Experimental Officer at the University of Kent) about their experiences and thoughts of conducting VR studies remotely.
See the full webinar video recording here on the WorldViz YouTube channel or click the image below.
How important is it to get immersive VR based science running remotely, and how realistic is it?
All speakers agreed this is very important, because there are research studies that are funded and the clock is ticking, while bringing subjects into the labs is no longer possible.
Being able to use VR at home would allow researchers to tap into greater numbers of subjects. But that may only be possible if the quality of the experiences really matches that of high-quality setups found in laboratories. That begs the question of how can we bring the equipment into the homes as most homes don't yet have VR equipment.
Since so many studies today utilize VR and in the lab are using VR headsets, it's important to find out how these studies can be adapted to equipment available in homes, such as desktop PCs or laptops. More and more, we'll probably grapple with the trade-offs between VR headset based (sensory immersion) and monitor-based (psychological immersion) approaches. It's likely that some studies can still be achieved without VR headsets.
What's your experience with conducting remote science for behavioral research pre-pandemic, and how successful is it so far?
One researcher already had a study that while it was being run in the lab, it was based on PsychoPy, and it turned out that there was a ready-made method for deploying PsychoPy based studies over the web. But the real challenge became designing instructions that were robust enough for subjects to use offline. Even an "on-call" person was made available but it turned out no one needed to call. In the end, the quality of the data seems to be as good as data collected in the lab.
Another approach used pre-pandemic was using VR technology to create motion-capture animations, and then these animations were converted to 2D video clips deployed using Qualtrics. This has been successful so far, but certainly limitations exist with this approach because of lack of programmability. Also, the 2D video clip approach is strictly limited in the types of paradigms you can undertake.
Other complexities when switching studies to remote are having the remote executables even unzipped and run, as well as sending data back (e.g., via email). Researchers find themselves up against a hurdle to have software polished just about to the point of consumer software. That's no to mention issues related to hardware consistency.
One researcher has been using Inquisit and Pavlovia for running reaction time studies successfully. Instead of sending out executables, researchers have been using Zoom to provide stimuli from a VR engine. In this case, the researcher runs the study and the subject gives a verbal response which the researcher keys in to guide the simulation.
What are your thoughts on controlling a remote setting for good experimental consistency?
A question was raised about possibly using the VR headset’'s sensors to measure some form of subject attentiveness or compliance. This might help ensure subjects are wearing the VR headsets (and are insulated from local distractions) and are simultaneously fully engaged (as monitored through appropriate head and eye movements).
We will need to be concerned about challenges with protocol approvals and IRB (Institutional Review Board) approval. Identifying risks and how to protect the subjects will be important topics as studies become run from homes.
What are your concerns about data retrieval such as reliability, privacy, or IRB approval?
A good point was made that when running studies remotely, the data is collected in the privacy of someone's home and at the end of the study, the subject can choose whether or not to release data to the investigator.
Beyond collecting standard motion capture data, one of the speakers has integrated the Muse consumer mobile device to collect 4 channel EEG and PPG heart rate. This shows the promise of combining various consumer based technologies to get greater results.
Products/topics mentioned
Here are inks to some of the products/topics that were discussed:
Qualtrics https://www.qualtrics.com
Pavlovia https://pavlovia.org
Prolific Academic https://www.prolific.co
Mechanical Turk https://www.mturk.com
Unity 3D https://unity.com
Vizard https://www.worldviz.com/vizard-virtual-reality-software
Inquisit https://www.millisecond.com/products/inquisit6/weboverview.aspx
Psychopy https://www.psychopy.org
XRDRN https://www.xrdrn.org/about/
Oculus Quest https://www.oculus.com/quest/
HTC Cosmos https://www.vive.com/us/product/vive-cosmos/overview/
Please contact us at sales@worldviz.com to discuss how WorldViz can help you get ready for conducting your own research study with remote subjects.
