Spatial Computing in Healthcare: Uses & What IT Leaders Need to Know

What Is Spatial Computing?

Spatial computing combines physical and digital realities into one space. Rather than reading information on a two-dimensional screen — a phone or laptop, for example — spatial computing technology allows people to process and interact with data in a 3D format. Virtual, augmented and mixed reality are all forms of spatial computing.

How Is Spatial Computing Used in Healthcare?

Developers have built hundreds of apps for the Apple Vision Pro, and providers are exploring their best uses in clinical and educational settings.

Medical Education

Virtual and augmented reality platforms are increasingly used to supplement medical training. Immersive simulations allow doctors, nurses and other providers to not only practice procedures but also experience the strong emotions that critical care situations can trigger. 

“A student can, for example, have a visceral sense of what a code blue situation feels like,” Lichtenstein explains.

“There are psychological and emotional results of spatial computing that, if leveraged correctly, can help people learn in a way they couldn’t with any other technology,” says Dr. Brennan Spiegel, director of health services research for Cedars-Sinai in Los Angeles and a 2023 HealthTech influencer. “If you think about the strongest memories from your own life, they’re usually tied to an emotion. People learn best from experiences they can imagine in their heads.” 

EXPLORE: Virtual reality helps healthcare students experience clinical scenarios.

Mental and Behavioral Health

The Centers for Disease Control and Prevention estimates that 20 percent of U.S. adults have a mental illness, and workforce data shows that more than half of Americans live in areas with a shortage of mental health providers. Researchers are exploring how spatial computing can be used with artificial intelligence to expand access to mental and behavioral health services and help providers treat more patients. 

One innovation is the therapist-trained AI platform that Spiegel co-founded: the eXtended-Reality Artificially Intelligent Ally, or Xaia, which runs on the Apple Vision Pro. Patients speak to a digital robot that also generates contextually relevant virtual environments. For instance, Xaia may create a rock garden or play soothing music for a patient who describes feeling anxious. 

“Xaia can augment the care that’s already being provided by a human,” says Spiegel, adding that Xaia alerts the therapist if it detects a risk of self-harm. His team is currently teaching the system to write clinical notes.

Enhanced Imaging

Lichtenstein, who helped launch Sharp HealthCare’s Spatial Computing Center of Excellence to explore new ways to use Apple Vision Pro in healthcare settings, describes the device as a “phenomenal tool for people who need to touch anatomy in some form.” He explains that new spatial apps are providing 3D views of anatomical images, which radiologists can use for more precise diagnoses. 

He also predicts that the technology will become more widely used as a presurgical tool. “We’re seeing a high level of interest from our surgeons and proceduralists about this,” Lichtenstein says. “The technology can help them understand the nuances of a patient’s specific anatomy before going into the operating room.”