top of page
IRIS Logo.png

Machine Vision

Introducing IRIS

IRIS is a tool that uses Machine Vision to detect occupants as they walk into MSK buildings. Using multiple video cameras, digital signal processing, and analog-to-digital conversion, IRIS can identify individuals without QR code or badge verification.


Hospitals use a range of security measures to keep staff, patients, and visitors safe. Hospitals mainly use security staff, CCTV cameras, duress alarms, and electronic access on door systems. IRIS acts the extra set of eyes for MSK buildings, not only to assist security staff for protection needs, but IRIS is also used to enhance the patient and visitor experience.


Eric Weintraub

Sr. Software Engineer


Louis Riccardi

Project Manager


Mac Macgari

Sr. Software Engineer 

Headshot 2.jpeg

Erica Parker

Sr. Product Designer


What is Machine Vision

Machine vision (MV) is a technology that uses digital or analogue cameras to capture and process images in order to automate visual inspection, guidance, sorting, recognition, recording, and any other process that requires a visual understanding of the environment. It is based on the principle of artificial intelligence and can be used for robotic vision, navigation, automatic inspection, surveillance, multimedia applications, and other automated processes.

What Are We Seeing?

At MSK, machine vision can be used in a medical setting for recognition purposes, mistake detection, inspections, measurement, and repetitive tasks. Where human vision is best for qualitative interpretation of a complex, unstructured scene, IRIS excels at quantitative measurement of a structured scene because of its speed, accuracy, and repeatability.


Enhanced Storytelling

First we created a step by step storyboard. We looked at the experience of registering at home as a user (top row). Then we created a story flow that outlined the steps of review by staff member (bottom row).  


"There are some really good use cases in Josie (which is good on its own) – well sealed & contained / controlled environment"  - Security 

"2 nights prior we receive patient list (25-28). Busy days is 50+ procedure schedules. Most of the patients come in with a care giver". 
- Associate Director 

“We welcome everyone, from visitors, to caregivers, and even employees from other buildings".  - Security

“There will always be 24/7 camera access in the building so this use case can easily happen. We have a security watch list that you should also consider integrating”- Head of Security 

Stakeholder Interviews

We met and interviewed the Josie staff security who intake patients and guide them to their surgical floors. 

Interview Findings

  • Machine Vision could potentially be helpful for security in identifying shapes or bulges / packages / explosive devices

  • We can use MV to highlight anomalous behavior + facial recognition 

  • Familiarity can blessing & curse, some people may be scared by being automatically recognized

  • MSK employee status can change relatively quickly but the information doesn’t get disseminated fast enough so MV can help us identity employees who shouldn't be on site

  • Patient’s faces  can be registered and remembered so we don't have to badge them

  • Rely on  MV to let security staff know that a clinical team member has arrived 

Understanding the user flows was an integral part for the development of this proof of concept. There were several key groups that needed to be identified and processed in the system.  

  • Detection of Staff (Top Row)

  • Detection of Patients (Middle Row)

  • Administration Intake of Patrons (Bottom Row)

User Flow.png

User Flows

MSK Staff 

A final product was developed and designed for the security staff who process intake and registration. They currently give patients RTLS badges to track them in real time throughout the hospital and then wait for a member of the clinical team to escort them to a surgical floor for processing. With this MV tool, security can eliminate the need for RTLS badges, check in, and immediately greet patrons who enter as they walk up to the counter. 


Mobile Interface

Visitors can register at home and maintain of continuous log of their facial recognition activity. They can view the amount of scans that are taken by camera during their visit.


MSK employees can see other staff and patients who are within close range in order to better assist them and address them by name if they have on PPE and are unrecognizable at first glance. 

Scan Interface

In order to get started, users must scan their face and have their data entered manually. For this experience a detailed GUI  was prepared that identified key points on a users face and written instructions on which direction a user would need to move in order to collect all target face points. 


Patient / Visitor Registration Prototype

Patient registration prototype detailing a user who hasn't pre-made a profile and is walking into the surgery center for the first time. 

Interface: iPad 

IMobile Scan Prototype 

Patient mobile face scan for registration off site. 

Interface: Mobile iOS

Desktop Prototype 

Internal product to manage facial recognition scans, registrations, threats, and building occupant locations. 

Interface: Web

iPad Prototype 

MSK employees mentioned that a part of the patient experience is enhanced when patients are greeted directly at the door. For this product they also asked for a prototype version of an iPad for when they need to monitor the software but step away from the desk. 

Interface: iPad

bottom of page