IRIS is a tool that uses Machine Vision to detect occupants as they walk into MSK buildings. Using multiple video cameras, digital signal processing, and analog-to-digital conversion, IRIS can identify individuals without QR code or badge verification.
Hospitals use a range of security measures to keep staff, patients, and visitors safe. Hospitals mainly use security staff, CCTV cameras, duress alarms, and electronic access on door systems. IRIS acts the extra set of eyes for MSK buildings, not only to assist security staff for protection needs, but IRIS is also used to enhance the patient and visitor experience.
Sr. Software Engineer
Sr. Software Engineer
Sr. Product Designer
What is Machine Vision
Machine vision (MV) is a technology that uses digital or analogue cameras to capture and process images in order to automate visual inspection, guidance, sorting, recognition, recording, and any other process that requires a visual understanding of the environment. It is based on the principle of artificial intelligence and can be used for robotic vision, navigation, automatic inspection, surveillance, multimedia applications, and other automated processes.
What Are We Seeing?
At MSK, machine vision can be used in a medical setting for recognition purposes, mistake detection, inspections, measurement, and repetitive tasks. Where human vision is best for qualitative interpretation of a complex, unstructured scene, IRIS excels at quantitative measurement of a structured scene because of its speed, accuracy, and repeatability.
First we created a step by step storyboard. We looked at the experience of registering at home as a user (top row). Then we created a story flow that outlined the steps of review by staff member (bottom row).
"There are some really good use cases in Josie (which is good on its own) – well sealed & contained / controlled environment" - Security
"2 nights prior we receive patient list (25-28). Busy days is 50+ procedure schedules. Most of the patients come in with a care giver".
- Associate Director
“We welcome everyone, from visitors, to caregivers, and even employees from other buildings". - Security
“There will always be 24/7 camera access in the building so this use case can easily happen. We have a security watch list that you should also consider integrating”- Head of Security
We met and interviewed the Josie staff security who intake patients and guide them to their surgical floors.
Machine Vision could potentially be helpful for security in identifying shapes or bulges / packages / explosive devices
We can use MV to highlight anomalous behavior + facial recognition
Familiarity can blessing & curse, some people may be scared by being automatically recognized
MSK employee status can change relatively quickly but the information doesn’t get disseminated fast enough so MV can help us identity employees who shouldn't be on site
Patient’s faces can be registered and remembered so we don't have to badge them
Rely on MV to let security staff know that a clinical team member has arrived
Understanding the user flows was an integral part for the development of this proof of concept. There were several key groups that needed to be identified and processed in the system.
Detection of Staff (Top Row)
Detection of Patients (Middle Row)
Administration Intake of Patrons (Bottom Row)
A final product was developed and designed for the security staff who process intake and registration. They currently give patients RTLS badges to track them in real time throughout the hospital and then wait for a member of the clinical team to escort them to a surgical floor for processing. With this MV tool, security can eliminate the need for RTLS badges, check in, and immediately greet patrons who enter as they walk up to the counter.
Visitors can register at home and maintain of continuous log of their facial recognition activity. They can view the amount of scans that are taken by camera during their visit.
MSK employees can see other staff and patients who are within close range in order to better assist them and address them by name if they have on PPE and are unrecognizable at first glance.
In order to get started, users must scan their face and have their data entered manually. For this experience a detailed GUI was prepared that identified key points on a users face and written instructions on which direction a user would need to move in order to collect all target face points.
Patient / Visitor Registration Prototype
Patient registration prototype detailing a user who hasn't pre-made a profile and is walking into the surgery center for the first time.
IMobile Scan Prototype
Patient mobile face scan for registration off site.
Interface: Mobile iOS
Internal product to manage facial recognition scans, registrations, threats, and building occupant locations.
MSK employees mentioned that a part of the patient experience is enhanced when patients are greeted directly at the door. For this product they also asked for a prototype version of an iPad for when they need to monitor the software but step away from the desk.