Socially Savvy Artificial Intelligence
Researchers hope to create an AI agent that can gather and use information about humans’ social interactions to help teams meet their goals.
University of Arizona researchers have been awarded $7.5 million to create an artificial intelligence agent that can understand social cues and human interactions and use that information to help teams achieve their goals.
The grant comes from the Defense Advanced Research Projects Agency and is part of DARPA’s Artificial Social Intelligence for Successful Teams program.
Xinyu Zhang illustration
“The goal of the ASIST program is to develop artificial intelligence with a ‘theory of mind’ and create AI that is a good teammate to humans,” says Adarsh Pyarelal, a research scientist in the Machine Learning and Artificial Intelligence Lab in the School of Information. Pyarelal is the principal investigator for the DARPA-funded project.
“Theory of mind” refers to humans’ ability to infer the beliefs, desires and intentions of other humans. Popular existing AI agents, such as Siri, Alexa and the Google Assistant, do a good job scouring the internet for information, but they aren’t good at reading social cues, Pyarelal says. For example, Siri doesn’t know the difference between someone yelling or speaking politely.
“The thing that makes a human a good teammate is having a good sense of what other people on their team are thinking, so they can predict what their teammates are going to do with some level of certainty,” Pyarelal says. “We’re trying to build AI with some social awareness and this kind of theory of mind.”
The project, funded over four years, is called ToMCAT, which stands for Theory of Mind-Based Cognitive Architecture for Teams. It is a collaboration among the School of Information in the College of Social and Behavioral Sciences, the Department of Computer Science in the College of Science, and the Department of Family Studies and Human Development in the College of Agriculture and Life Sciences’ John and Doris Norton School of Family and Consumer Sciences.
Researchers will develop AI agents that they will test in a Minecraft video game environment. The agents will be partnered with one to four human players and will collaborate with the players to complete custom Minecraft missions. Throughout the missions, the AI agents will gather information about the individual players as well as their interactions with their other human teammates, through a variety of digital and physical sensors.
Webcams and microphones will record human players’ eye movements, facial expressions and voices, and an electrocardiogram machine will monitor the electrical activity of their hearts. Players also will wear a cap that measures brain activity using two brain imaging techniques — electroencephalography, which detects electrical activity in the brain, and functional near-infrared spectroscopy, which uses skull-penetrating infrared light waves to map blood flow in the brain.
Adarsh Pyarelal / Chris Richards photo
In the first phase of the project, AI agents will merely observe and gather information about individual players through the sensors. Later, the agents will observe and collaborate with up to four human players in a team. Eventually, the goal is for the AI agent to have enough information about the individual teammates and the team’s goals and social dynamics to suggest interventions, via plain-text chat messages, to help the team meet its objectives.
“The AI agent is going to provide helpful suggestions or check in from time to time about the situation,” Pyarelal says. “So, everything comes full circle — the agent observes, it learns, and then, if needed, it can intervene to help the team. For example, if Person A and Person B don’t get along, the AI agent may suggest that maybe you don’t want to put them on the same team.”
The ToMCAT project aligns with the University of Arizona’s continued focus on the Fourth Industrial Revolution, which is characterized by the increasing convergence of the digital, physical and biological worlds.
“Artificial intelligence and machine learning are increasingly present and important in our everyday lives, and the development of new technologies and understanding in this area is a key focus for the university,” says University of Arizona President Robert C. Robbins. “With DARPA’s support, our talented researchers have a tremendous opportunity to take the next step toward making AI even more beneficial to our daily routines and interactions.”