News & Updates

Researchers unveil 3D medical imagery with mixed reality smartglasses

Hundreds of thousands of Americans enter a cardiac catheterization lab each year for a heart stent procedure, something that can be made easier for doctors with holographic, 3D medical imagery that comes to life with mixed-reality smartglasses, researchers say.

The technology was just one of the innovations unveiled at Philips Cambridge Research Center this week.

When a doctor puts the Microsoft HoloLens glasses on, holographic screens displaying a patient’s vitals and other crucial information appear.

“The ability to move your screens anywhere in the room without having to touch it, with just being able to control things with your voice was a big huge plus,” said P.J. Keenan, a development engineer with Philips Research North America, which is developing the technology.

With simple voice commands and hand gestures, doctors can zoom in on real-time images of the patient’s arteries by lifting their index finger and thumb and pinching in.

Molly Flexman, principal scientist at Philips, said, “So we basically call it our virtual cockpit so that you now have access to everything that you need from this virtual world.”

Among the other innovations, Philips also presented the VitalSigns Camera technology that can turn a standard video camera into a tool to help monitor patients who must wait in emergency rooms, sometimes for long periods of time, when there is not always someone to monitor their breathing or heart rate.

“Some of these conditions can get worse while you wait, you have to wait a long time sometimes, and you’re not being monitored, nobody’s paying attention, nobody’s taking your vital signs,” said Kees Van Zon, principal scientist at Philips.

The VitalSigns technology can use any standard video camera to measure pulse and breathing rate by looking for microfluctuations in facial color.

Van Zon said in the future, the camera technology could even be strapped to first responders in a mass causality incident to find and monitor failing patients in a large crowd.

“That can help with triage, and with detecting if somebody is really going south,” Van Zon said, “like getting worse in terms of their health status, so that they need to be prioritized.”