CMU Group Uses Bridges-2 to Create Video Game for AI to Learn From, Teach Human Engineers
The nation’s bridges are literally crumbling. Authorities don’t have enough trained engineers or money to inspect bridges properly. And inspection itself is as much art as science. A team from Carnegie Mellon University used PSC’s flagship Bridges-2 supercomputer to create a kind of bridge-inspection video game, allowing an artificial intelligence (AI) to learn from experienced engineers and for new engineers to learn from the AI, pairing the strengths of humans and AI and making inspection more effective and efficient.
WHY IT’S IMPORTANT
It was a typical early weekday morning in Pittsburgh’s East End — a little after 6:30 am on a Friday in late January 2022. Not quite rush hour yet. A good thing, when the Fern Hollow Bridge gave way. The span, which had been built in 1973, dumped five vehicles, including a PAT Transit articulated bus, deep into the Fern Hollow ravine below.
Miraculously, only 10 people, including rescuers, were injured. No one was killed.
Then came the reveal. The bridge, as it turned out, had been unsafe for quite some time. The bridge’s inspection and repair regime had broken down in many places. But it highlighted the precarious state of bridges in Pennsylvania and throughout the U.S. It especially cast into harsh light the state of the art of bridge inspection.
Bridge inspectors are skilled workers with very specific training. But inspectors who are great at spotting damage in a particular type of bridge may sometimes not be as strong at inspecting other types. How they go about their work can be idiosyncratic. It’s hard to establish a single, or single set, of “proper” ways to go about checking a bridge for damage. Ultimately, it’s partly science, partly art. And you can’t really make art consistent.
“We want to enable insight about how different people checked the same structure … in the past 20 years, and to help us to identify which parts of the structure attracted more attention from more people … Can we find the evidence from the structure [and from] engineering domain knowledge to explain their behavior, and how we can generalize those explanations to allow additional engineers to understand [what] is the best strategy [for] a human to do the work? It’s a training system for humans. It’s a training system for AI.”
— Pingbo Tang, CMU
Pengkun Liu, a graduate student working with CMU’s Pingbo Tang, wanted to know if there was a way to give bridge inspectors more consistent, more dependable, and explainable ways to look for damage, by making skillful inspectors’ search tactics visible. Tang’s laboratory has had success using AI both to learn from engineers and to train new engineers in fields such as water treatment plant operation. Together, they applied these tools to pick apart what makes a bridge inspection effective, and how bridge engineers can be better trained. Through an allocation from the NSF’s ACCESS network of supercomputing centers, they obtained time on their tool of choice: PSC’s Bridges-2.
HOW PSC HELPED
The first step in analyzing the bridge-inspection problem would be to wrangle its complexity. Every bridge is a little different, and every inspector goes about the work in a unique way. Also important would be to establish a “ground truth” of what goes wrong with bridges. Data on defect records of real bridges compared with the reliability of the individual inspectors who recorded those data would be needed as well.
They approached the problem in the same way Tang’s group had attacked the water treatment problem. They would create a kind of video game that presented 96 experienced bridge inspectors with damaged girder-type bridges. By tracking the inspectors’ eye movements and where they clicked their mouse for a closer look, the AI could learn both from how they proceeded, as well as their success rates in finding different types of damage. By blending the processes, it could refine an improved process. Better, that same video game could be used to train new inspectors in that improved process to create a partnership between AI and humans that uses the strengths of each.
A “video game” paired the CMU team’ s AI with expert engineers as they inspected virtual bridges, so that humans and AI could teach each other. Image from “Quantifying the reliability of defects located by bridge inspectors through human observation behavioral analysis,” https://doi.org/10.1016/j.dibe.2023.100167
“We want to simulate the bridge with different defects [that would] affect the structure’s performance … If we want to simulate very detailed structural behaviors with different combinations of effects, we would need strong computing resources with GPU servers that will help us to generate a lot of scenarios … Then we have different scenarios too, and the experiments to test the bridge inspectors.”
— Pengkun Liu, CMU
Both training the AI and creating the video game would require serious computational power. In particular, they’d need many late-model graphics processing units (GPUs). These chips were developed to create more lifelike video game animations. But they also turned out to be particularly powerful in AI training. The NSF-funded Bridges-2, offering 360 GPUs backed up with massive RAM, would be able to handle the volume of data and demanding image and AI calculations.
The AI highlighted some important factors. Higher-accuracy inspectors tended to approach the bridges by focusing on connections between elements in the same section and across different spans, reflecting a better understanding of where stress on the bridge was more likely to cause damage. The result, which again can be used to train both new and experienced inspectors, increased the true detection rate of damage to a little over 90 percent, compared with 82 percent when combining the detections of several inspectors without such training.
These results only reflect inspections of girder-type bridges. Next, the CMU team plans to widen their work to include more types to see how they’re different and how the method can be optimized for wider use. Ultimately, while their work can help make damage inspections more reliable, such inspections are a product of engineering, budget, and political factors. A more reliable method of inspection can ultimately help authorities get more and safer results within these constraints.