I’ve been back for a few days from Semdial 2011 and a co-located workshop at the Institute for Creative Technologies in LA.
Both events were a great success and I’ve posted a personal round-up here.
I’ve published a demo of our micro-domain “pentomino” system. It’s available here using Java Webstart (you need to have Java installed). Removed for now, but check out our toolkit if you want to build the system yourself.
The system was built using our toolkit for incremental spoken dialogue systems (InproTk, current release). The ASR is based on the CMU sphinx4 engine (plus incrementality).
The system allows users to select and delete puzzle pieces on a canvas with voice instructions. These will be processed incrementally in real-time. For instance, an instruction “take the yellow piece” will display understanding of “take” before the end of the sentence and execute as soon as possible, i.e. after “yellow” as in the examples below (it currently only understands German).
System utterances, consisting of different sorts of short back-channel utterances (ok?, ok!, mmhh), reflect whether the system believes the users input was complete or not, based on pitch classification and previous input.
The exact nature of how dialogue and interaction management are achieved are described in our Sigdial and Semdial 2010 papers.
Example interaction states (click for larger version).
In order: initial, after partial user utterance “take”, after “take the yellow” and after just “the yellow”:
I’m a researcher in Computational Linguistics at Potsdam University. My research concerns incrementality in interaction, with special focus on dialogue management in spoken dialogue system. The aim is to create such systems that are more robust, natural and responsive than normal systems.
On this site you can also find my publications. I’ll occasionally post news related to my research. You’ll find more news and updates here.
