My visit to the medical Holodeck – cancer research at Weill Cornell using HoloLens and the VR Cave

I just spent a few days in New York setting up a workshop to help minority students to get into development (soon more on that). I was lucky to be in Microsoft’s Reactor when Alex Sigaras, a research associate in computational biomedicine at Weill Cornell Medicine gave a talk about how HoloLens transforms healthcare research for the HoloLens Developer Group in New York.

@AlexSigaras is about to start his presentation on how #Hololens transform healthcare at the #Devs meet-up NYC https://t.co/eD8MLdmzrQ pic.twitter.com/FiypETVDt0

— Andrea Sboner (@asbonerw) 11 January 2017

I took the opportunity to talk to Alex for Decoded Chats about that. We also covered other topics such as sharing of information in healthcare. And how HoloLens despite being a high-end and rare device allows for collaboration of experts in all feld and not only developers.

If you prefer to have an audio version, you can download it here (MP3, 19MB)

Here are the questions we covered:

1. You just gave a talk at a HoloLens meetup about medical research. Who are you and what do you do with HoloLens?
2. What are the benefits of using the HoloLens as a visualisation tool in computational medicine compared to VR environments?
3. Is there a collbaboration benefit in augmented reality and mixed reality rather than virtual reality? Does it scale better in bigger groups?
4. Genomics is known to have to deal with huge amounts of data. Isn’t that an issue on a device that is self-contained like the HoloLens?
5. Most of the HoloLens demos you see are single person use. Your use case is pushing the collaborative part of the device. How does that work out?
6. What is the development stack you use? Did you find it hard to port to the device and to re-use code of other, VR, solutions you already had?
7. Do you also have some success stories where using HoloLens helped find a data correlation faster than any of the other ways you used before?
8. Is there any way for the audience here to collaborate in your research and help you further breaking down silos in medical research?

You can see the HoloLens work Alex and his team are working on in this tweet.

Our latest work on HoloLens with RoshalSophia ElementoLab engIPM. Source code coming soon on github. pic.twitter.com/tTtO8vWCPI

— Alex Sigaras (@AlexSigaras) 29 September 2016

The slides of his talk are on SlideShare and have a lot more information on the topic.

In addition to visiting Alex at work, I also got a special treat to have a demo of their other VR work, including The Cave, a room with 5 walls that are rear-projected screens allowing you to get detailed 3D views of MRI scans.

Here’s a very raw an unedited video of Vanessa Borcherding (@neezbeez) showing their research in VR and the insights it can give you.

Warning: unless you are also wearing 3D glasses, this video flickers a lot:

I left the hospital and research facility and had to take a long walk in Central Park. It is not every day you see things that you always considered science fiction and a faraway dream happen right now. I’m looking forward to working more with these people, even if I felt utterly lost and the dummy in the room. It is great to see that technology that on first glance looks great for gaming and entertainment can help experts of all walks of life to do important work to make people live longer.