You are here

Technical report | Event Sequencing for Situation Narratives

ABSTRACT

Situation Awareness is a critical factor for decision making in complex dynamic environments. Information fusion techniques enable machines to augment human situation awareness, provided the machine representation of the situation can be conveyed to the human operator reliably.

The problem, addressed in this report, is to order a set of events (a plot) in such a way as to result in a narrative which maximises audience engagement, but does not mislead with respect to the temporal relationship of significant events. A novel algorithm is proposed which uses a constrained optimisation approach to achieve an audience implementation. Measures of temporal tension, continuity and centrality are proposed and used as optimization criteria. Applying different weights to these measures produces narratives with different characteristics.

The algorithm has been implemented and the results of testing using a large historical naval battle (the World War II sinking of the battleship Bismarck) are presented. With appropriate weights the algorithm was shown to produce a plot comparable to human authored text. A number of vignettes are analysed which provides insight into the operation and limitations of the algorithm. In particular, incompleteness in the input data permitted misleading output in some cases. While unsurprising, this highlights the need for careful consideration of the effect of incompleteness and uncertainty on automatic situation assessment aids.

The need for future work has been identified including: refinement of the optimisation measures; broader and more rigorous evaluation; and integration into a complete situation assessment system.

Executive Summary

Situation Awareness is a critical factor for decision making in complex dynamic environments. Information fusion techniques enable machines to augment human situation awareness, provided the machine representation of the situation can be conveyed to the human operator reliably.

Situations generally evolve over time and situation awareness requires the ability to project this into the future so as to assess likely impacts. In complex situations involving multiple actors, telling events in strict time order tends to result in a disjoint narrative. Audience engagement may be enhanced by following one actor for a sometime before switching to follow another actor. The problem, addressed in this report, is to find an order of events (a plot) which maximizes audience engagement but does not mislead with respect to the temporal relationship of significant events.

This report proposes a Multiple Intersecting Chronologies (MIC) model, inspired by the structure of fictional narratives such as The Lord of the Rings [Tolkien, 1966], which is flexible enough to represent narratives aimed enhancing situation awareness, but sufficiently constrained to allow efficient implementation.  In this model each actor has its own strictly chronological story line, but story lines are asynchronous with respect to each other. Story lines may intersect or merge at which point they become synchronized.

A novel algorithm has been developed which takes an unordered list of events as input, constructs multiple story lines and traverses them in a way which respects the constraint that each story line be strictly chronological. Measures of temporal tension, continuity and centrality are used as optimization criteria. Temporal tension refers to the degree to which events are presented out of order, continuity is the degree to which the narrative continues to follow the events experienced by one entity and centrality was introduced to allow greater emphasis on following the story line of central characters. Applying different weights to these measures produces narratives with different characteristics. High temporal tension weight produces a narrative which tends towards a strict chronology. High continuity weight tends to produce a narrative which follows the action from cause to effect. High centrality weight tends to produce a narrative which follows the central character and defers telling the background of secondary characters until they interact with a more central character.

The algorithm has been implemented and was tested using a large historical naval battle (the World War II sinking of the battleship Bismarck). With appropriate weights the algorithm was shown to produce a plot comparable to human authored text.

A number of vignettes are analysed which provides insight into the operation and limitations of the algorithm. In particular, incompleteness in the input data permitted misleading output in some cases. While unsurprising, this highlights the need for careful consideration of the effect of incompleteness and uncertainty on automatic situation assessment aids.

The testing of the algorithm has yielded significant insight into its characteristics and limitations; however it cannot be considered a rigorous evaluation. The output was evaluated by comparing statistics with human authored text, but this leaves unanswered the question of whether these are the relevant statistics. Also, human authored text is not necessarily the ‘gold standard’. Depending on the case, it may well be possible for the machine generated event sequence to be better than that found in human authored text. It would be more valid to assess audience attention, recall and comprehension or their ability to perform a relevant task although such methods are expensive and often impractical [Reiter and Belz, 2009]. Future work should include this kind of evaluation.

Only one scenario (albeit a large one) has been used leaving open the question of whether the algorithm is generally applicable and whether the weights need to be tuned for each scenario. In particular temporal tension has been identified as being problematic because of the varying tempo within scenarios and between scenarios. There is considerable scope for future work to investigate better measures of the temporal tension or replacing it with a new concept.

Finally, event sequencing is just one part of the automated narrative generation problem. Integration of this work into a complete system is necessary to achieve the goal of enhanced situation awareness. This would also enable rigorous human evaluation as discussed above.

Key information

Author

Ian Dall and Bradley Donnelly

Publication number

DST-Group-TR-3351

Publication type

Technical report

Publish Date

June 2017

Classification

Unclassified - public release

Keywords

narrative, situation awareness