DARPA Project to Create AI-Enabled Cultural Interpreters to Aid Defense Operations

The U.S. Defense Advanced Research and Projects Agency (DARPA) is working on Artificial Intelligence (AI)-enabled interpreters to aid in military operations.

Communicative understanding, not simply of local languages but also of social customs and cultural backgrounds, lies at the heart of Civil Affairs and Military Information Support Operations activities. These collectively comprise a vast majority of U.S. counterinsurgency and stabilization efforts. Within these activities, cross-cultural miscommunication can derail negotiations, incite hostile discourse – even lead to war. The likelihood of communicative failure increases dramatically where significant social, cultural, or ideological differences exist.

Unlike the human cultural interpreters who enable U.S. forces today, current AI-enabled systems are incapable of accurately analyzing cross-cultural communication or providing useful assistance beyond basic machine translation. While there have been significant advances made in machine learning and multimedia analysis, a number of critical deficiencies in these systems still remain.

“To support users engaged in cross-cultural dialogue, AI-enabled systems need to go beyond providing language translation – they need to leverage deep social and cultural understanding to assist communication,” said Dr. William Corvey, a program manager in DARPA’s Information Innovation Office (I2O). “Moving AI from a tool to a partner in this capacity will require significant advances in our machines’ ability to discover and interpret sociocultural factors, recognize emotions, detect shifts in communication styles, and provide dialogue assistance when miscommunications seem imminent – all in real-time.”

To assist negotiations and aid critical interactions, DARPA developed the Computational Cultural Understanding (CCU) program. The goal of CCU is to create a cross-cultural language understanding service to improve a DoD operator’s situational awareness and ability to effectively interact with diverse international audiences. The program seeks to develop natural language processing (NLP) technologies that recognize, adapt to, and recommend how to operate within the emotional, social, and cultural norms that differ across societies, languages, and communities.

CCU is divided into two primary research areas. The first is focused on research and development efforts to address a specific set of challenges limiting the application of current human language and communication technologies. These challenges include: 1) the discovery of sociocultural norms, 2) emotion recognition as a function of sociocultural norms, and 3) detection of impactful changes in sociocultural norms and emotions.

Humans acquire inferred knowledge of diverse and varied sociocultural norms through a lifetime of learning and interaction. CCU aims to emulate this learning capability, creating technologies that are capable of automatic discovery of the sociocultural norms that influence discourse, including the social, cultural, and contextual factors that impact effective communication and rapport building.

CCU also aims to create a capability that can recognize speaker emotions across different languages and cultures. Human interpreters continuously monitor emotional feedback from conversational participants (e.g. facial expression, tone of voice, diction), using this information to gauge how the interaction is progressing and alter the exchange as needed. In order to interpret speaker emotions as influenced by sociocultural context, CCU will focus on developing multimodal human language technologies capable of generalizing their recognition of emotion across different languages and cultures.

Furthermore, CCU seeks to create capabilities to detect significant changes in communication that could indicate impending conflict or dispute. While promising change detection methods exist, current frameworks lack an understanding of which features are most crucial to detecting imminent communicative failure.

To help promote truly effective cross-cultural interaction, CCU technologies must be able to not only detect potential misunderstandings but also generate alternative socioculturally-appropriate responses. As such, the second research area in CCU will focus on the development of a dialogue assistance service. The target service will automate the detection of sociocultural context, including aspects related to identity and group affinity, and leverage the outputs of the first research area in order to follow ongoing conversations, detect misunderstandings in real-time, and provide dialogue assistance to human operators to prevent communications from going awry and/or provide remediation support.