A Multimodal Conversational Interface for a Concept Vehicle

  • Roberto Pieraccini
  • Krishna Dayanidhi
  • Jonathan Bloom
  • Jean-Gui Dahan
  • Michael Phillips
  • Bryan R. Goodman
  • K. Venkatesh Prasad


This paper describes a prototype of a conversational system that was implemented on the Ford Model U Concept Vehicle and first shown at the 2003 North American International Auto Show in Detroit. The system, including a touch screen and a speech recognizer, is used for controlling several non-critical automobile operations, such as climate, entertainment, navigation, and telephone. The prototype implements a natural language spoken dialog interface integrated with an intuitive graphical user interface, as opposed to the traditional, speech only, command-and-control interfaces deployed in some of the vehicles currently on the market.


Pieraccini, R., Caskey, S., Dayanidhi, K., Carpenter, B., & Phillips, M. (2001). "ETUDE, a Recursive Dialog Manager with Embedded User Interface Patterns," Proc. of ASRU01 -IEEE Workshop, Italy.

Heisterkamp, P. (2001). Linguatronic - Product

Level Speech System for Mercedes-Benz Cars, Proc. of HLT 2001, Kaufmann, San Francisco.

Jaguar Cars. (2001). Limited Publication Part No. JJM 18 09 24/22, Coventry, UK.

Minker, W., & Haiber, U., Heisterkamp, P. (2002). Intelligent Dialog Strategy for Accessing Infotainment Applications in Mobile Environments, Proc of IDS02, Kloster Irsee, Germany.

Bernsen, N. O., & Dybkjaer, L., A. (2002). Multimodal Virtual Co-driver's Problem with the Driver, Proc of IDS02, Kloster Irsee, Germany.

Pieraccini, R., Carpenter, B., Woudenberg, E., Caskey, S.,

Springer, S., Bloom, & J., Phillips, M. (2002). Multi-modal Spoken Dialog with Wireless Devices, Proc of IDS02, Kloster Irsee, Germany.

VoiceXML eXtensible Markup Language Version 1.0, http://www.w3.org/TR/voicexml