Mass casualty incidents can rapidly overwhelm rescuers. Advances in robotic coordination may help human rescuers more effectively use the triage process to prioritize who to help first.
CMU Team Responds to $7-Million DARPA Challenge with Robot-Human Team, Guided by AI
In triage, first responders to a mass-casualty incident classify multiple patients by the level of care they need: those who can wait, those who need care now, and those who are beyond help. This helps a small number of rescuers save the most people. A team from Carnegie Mellon University used PSC’s flagship Bridges-2 supercomputer to build an artificial intelligence (AI) that can run a system of robots, flying and walking, that explores a mass-casualty scene, locating patients and helping to guide human medics to the ones they need to help first.
WHY IT’S IMPORTANT
It’s a first responder’s nightmare. You roll up on a site where there’s been a massive car pileup, an outdoor festival hit by a lightning storm, or a downed aircraft. People — dozens, or hundreds, or more — are lying all around.
For the moment, there’s only you, and maybe a partner.
To deal with such a mass casualty situation, first responders like EMTs, paramedics, firefighters, and police use a system called triage. In order to help the most of your many patients most effectively, you hold off on treating anybody immediately. Instead, you move from person to person, doing a quick evaluation. You mark them with cards. Green cards are for people who are well enough to evacuate themselves. Yellow means they need help, but it can wait. Red are the people who need assistance urgently. Black are deceased, or so badly hurt they can’t be helped.
At least, that’s the theory. In real life, responders are human beings, and in the chaos and horror of the moment it’s hard to apply the system exactly the way it’s intended.
“DARPA first said the challenge we want to tackle is, how to find casualties faster. And then, via standoff triage, how do we autonomously identify injury patterns, casualties that need to be addressed first … based on the injury pattern, and identifying the life-saving interventions needed.” — Kimberly Elenberg, CMU
Kimberly Elenberg, a principal project scientist at CMU’s Robotics Institute and a previous paramedic instructor and nurse, leads Team Chiron, a group of physicians, researchers, scientists, and engineers studying the use of robots to help tame mass casualty sites. Recently they responded to a 2023, $7-million call from the U.S. Defense Advanced Research Projects Agency for just this purpose. One important tool in their work has been PSC’s NSF-funded Bridges-2 system.
HOW PSC HELPED
Team Chiron envisioned a combined team of flying drones, four-legged robots on the ground, and humans to reach and help the wounded. To ease the incredible shortage of human rescuers in any mass-casualty incident, the robots would need to have three difficult-to-achieve capabilities. They’d need to operate autonomously, without human guidance. They’d need to communicate and coordinate with each other. And they’d need to combine input from multiple sensors in a way that meaningfully measured how much trouble a given victim was in.
The only way to get all that to work was with AI. The robots would need a kind of hive mind that managed the bigger picture in a way that fed useful data like casualty status and geo-location data to the humans moving rapidly around a casualty field.
Building this AI was going to be a massive task. Really it wasn’t just one AI algorithm, or series of rules by which the AI would need to make decisions. It was more a family of algorithms. Some would assess heart and breathing rates. Others would determine whether a given patient is bleeding. Yet others would discover patients’ locations. All, of course, without actually touching the patient. Then the AI would need to combine all that to determine whether a given victim should be classed, for example, red or yellow, and display it all on a smartphone map that directed rescuers to where they needed to go.
“Our partnership with PSC on Bridges-2 was immensely helpful, because training these models takes a lot of bandwidth … Sergiu [Sanielevici, PSC’s Senior Scientific Advisor] and others have really been helpful, participating in many of our meetings, listening in, offering thoughts or guidance. So it wasn’t just about the technical hardware side, it was the soft side … You know, the coaching that we got.”
— Kimberly Elenberg, CMU
Elenberg successfully applied to the NAIRR Pilot program for an allocation on Bridges-2, which proved indispensable for the work. Its AI-friendly, late-model graphics processing units, or GPUs, helped run the many parallel tasks necessary for the AI to test out connections in the data that pointed to what color card each patient needed. By using ground-truth data provided by DARPA, the AI could teach itself rapidly, pruning connections that didn’t lead to correct answers and strengthening those that did. Once trained, the AI was ready to be tested in the field, where the answers weren’t labeled.
Two years into the three-and-a-half-year project, Elenberg’s team has made substantial progress. Thanks to both field testing and feedback from medics involved in the tests, they identified simply locating the patients as a huge help to the human rescuers. A mix of flying and walking robots has also become effective at identifying good routes for responders to access and evacuate the patients, as well as the protective equipment the rescuers would need to wear and bring with them.
Team Chiron is also cracking some of the more difficult tasks faced by the system. Something as simple as a patient lying face down can make it difficult to get usable data on breathing and heart rates. Night responses are more difficult than when the scene is well lit. In general, algorithms that work well in clean emergency rooms or operating rooms struggle in the messiness of an outdoor response. And the system will also need to be able to report a high-level assessment of the scene to the mass-casualty incident’s command staff. That way, the humans can alert and prepare community hospitals, air-medical services, and regional trauma centers for the incoming patients. Finally, while today the humans will take the lead on medical care, a walker robot that could transfuse blood into a critically injured patient could sharply decrease fatalities. That’s also in the works.