This project is still under development. The user must work with Taema, an agent who is trying to solve the case of the missing amulet that has been causing chaos throughout the city of Te Pouri. This project has a similar style of gameplay as Survival on Jungle Island, but new features are being added.
Humans are uniquely good at coordinating their actions. For example, in a two-player video game they can synchronize actions or interleave them, cue when to move and how, or signal when to stop and think. People do this not only with words, but also with prosodic patterns, involving intonation, loudness, and timing. We are discovering these patterns.
A shared project with the Communications Department, for the Smithsonian Latino Center. An interactive documentary that simulates the cultural experience of connecting over food by sharing recipes. Embodied Conversational agents (ECAs) are used to engage users with cultural food heritage from the U.S.-Mexico border.
In dialog, people frequently look away from their dialog partner, for example while thinking or when starting a new turn. A possible way to make video chat more efficient is to only send video frames that are likely to be looked at by the remote participant. We are studying gaze behaviors in dialog, with the aim of eventually building a predictive model.
A mix of Jeopardy and Wheel of Fortune, this game has players interacting with our game host agent Alex Raybeck. Alex asks questions to the user ranging from movies and games to UTEP history. This project was developed by the UTEP Game Builders student organization by utilizing tools from our lab.
Spoken dialog systems are generally disliked by users, in part because of slow and awkward turn-taking. In human-human interaction, people are able to predict each other's future actions and respond accordingly. We are building improved predictors using novel prosodic features and new modeling techniques.
“Survival on Jungle Island” adventure, is a 40 minute (on average) interaction where users carry conversations and a series of activities led by our ECA, Adriana. We simulated a survival scenario so the user could cooperate with the ECA and create rapport-building opportunities.
Many language learners never acquire truly native-sounding prosody, but there are no good comprehensive models of the deficit. We recorded 90 minutes of six advanced non-native speakers of English in conversation with natives. Using newly-developed methods, we identified three prosodic constructions common in English-native conversations but not in the learners data, relating to the pragmatic functions of showing involvement, and of explaining from a factual perspective.
A spoken-language adventure game entitled “Escape from the Castle of the Vampire King” with a graphical interface and a full-sized ECA that served as the game’s narrator, and the player controlling the game through speech commands. We iteratively developed three versions of the game where each version had a different combination of speech generation and virtual environment.
To help improve search in audio archives, we are exploring the use of prosodic information. We have developed a vector-space model of the non-content aspects of dialog, and applied this to news broadcasts and to telephone conversations. Using both supervised and unsupervised machine-learning techniques, we have shown that this can support searching on new dimensions of information, like locally relevant or new information, and can improve on search using words alone.