Yorick Wilks, Florida Institute of Human and Machine Cognition

Yorick Wilks, Florida Institute of Human and Machine Cognition



COMPANIONS is a concept that aims to change the way we think about the relationships of people to computers and the Internet by developing a virtual 'Companion' to stand between individuals and the torrent of data on the Internet, including their own life information, which will soon be too large for people to handle easily without some new form of assistance. The Companion is intended as an agent or 'presence' that stays with a user for periods of time, longer than in conventional task-based dialogue systems, developing a relationship and 'knowing' and assisting its owner's experiences, preferences, plans and wishes. The Companion communicates with the user primarily through conversational speech. Our starting point is an observation of Feigenbaum that we understand cognitive phenomena at the present time largely in terms of what we can model computationally: that the complexity of human dialogue functions are to be understood not in terms of linguistic or psychological theory but in terms of models we can program and evaluate. Until now, researchers have concentrated on implementing simple task-based systems, and so one can easily get the idea normal human dialogue is like that, but it is not.

This talk describes as an example the functionality of a Senior Companion (SC), one of two initial prototypes built in the first two years of the EU Companions* project. The Senior Companion provides a multimodal interface for eliciting and retrieving personal information from a user through a conversation about their photographs. The Companion will, through conversation, elicit their life memories, prompted by discussion of these images; the aim being that the Companion should come to know a great deal about its user, their life histories, tastes, likes, dislikes, emotional reactions etc, through conversation. Although it can carry out tasks, the Companion is not intended as primarily a task-based system with a defined end state. It is a further assumption that life information will be stored on the internet (as in the Memories for Life project : http://www.memoriesforlife.org/) and the SC is linked directly to photo inventories in Facebook,  to gain initial information about people and relationships, as well as to Wikipedia to enable it to respond about places mentioned in conversations about images. The key notion here was the use of open web sources of information, accessed at time of need, rather than the standard closed knowledge base of conventional AI. The overall aim of the SC, not yet achieved, was to produce a coherent life narrative for its user from these materials, although its short term goals are also to assist, amuse, entertain and gain the trust of the user.

The SC uses Information Extraction to get content from the speech input, rather than conventional parsing, and retains utterance content, extracted internet information and ontologies all in RDF formalism over which it does Jena-style reasoning about people, relationships and places. Its research hypotheses are stated in terms of its stack and network virtual machine for conversation, intended to capture mixed initiative, plans, recovery mechanisms in dialogue and the ability to reenter unfinished conversations without repetition. Even in its current state it raises issues about what kinds of entities people want to empathize with and trust, and how one can best synthesize personality and emotional rapport. If a Companion were to become the internet repository for someone's whole life--- to be a "cognitive prosthesis" for dealing with their own life's records, as well as the wider world----- what safeguards are essential, both technical and legal, concerning access to such a repository during the owner's life and after?

Developments of forms of a Companion are currently planned in both education and in eHealth, where it could be not only a counseling interface to physiological/medical data but also a self-model of which a patient can ask "what is happening to me now?". Applications can be placed anywhere on a task-task-free continuum; depending on the application, a conversation may finish with an outcome or continue indefinitely. Plausible applications could include debriefing of individuals, cross-lingual/cross-cultural communication, and any information-based elicitation dependent on a strong model of the user.

 Yorick Wilks