MR-IS

IRIS Wiki - IS Systems - MR-IS

Availability

MRIS (Mixed Reality Interactive Storytelling) is an IS system which was developed as a research prototype. It is not available for download.

Technical Description

Interactive Storytelling aims at immersing users in fantasy worlds where they can play a part in an evolving narrative responding to their intervention. Implementing such a concept involves a wide range of computing technologies: the recreation of an artificial world is generally achieved using virtual or mixed reality, while the real-time generation of a narrative including virtual characters makes uses of Artificial Intelligence techniques and formalisms.

If the user is to play a role as a character in the interactive narrative, s/he will have to communicate with virtual characters in a way which should reflect traditional acting. This brings a novel, specific context for multi-modal communication as well as a number of technical challenges. Acting involves a number of attitudes and body gestures that are highly significant both for drama presentation and for communication with virtual actors. At the same time, spoken communication is an essential aspect of a realistic interactive narrative. Dealing with such form of multi-modal communication is faced with several difficulties in terms of real-time performance, coverage and accuracy.

See also I-Storytelling.

Result Description (end user perspective)

Videos of several examples of the interactive sessions can be viewed here.

Authoring Description

The overall narrative authoring is processed via the encoding of the main character's role using Hierarchical Task Networks (HTN). The user is expected to be playing his own role as a secondary character.

Strong Points

The very essence of this system is in its ability to provide natural means of interaction for the user, as well as a narratively-immersive experience from the point of view of her interactions' influences on the unfolding of the narrative, in real-time.

Limitations

The overall system is fairly complex to put together due to the technologies involved.

Main Publications

• Cavazza M., Martin O., Charles F., Mead S.J., Marichal X. and Nandi A. (2004). Multi-modal Acting in Mixed Reality Interactive Storytelling. IEEE Multimedia, July-September 2004, Vol. 11, Issue 3. [1]

Supporting Narrative Theories

MRIS is strongly linked to a computational model of Narrative units from Roland Barthes (structuralism).

Computational Model

MRIS relies on AI planning tools and techniques, namely HTN planning (STRIPS-like planner). See pddl.

Type of interaction

Multimodal interactions:

• speech-based NL interaction
• gesture-based interaction