StoryFlow: Tracking the Evolution of Stories

A storyline visualization of the movie The Lord of the Rings. The yellow line indicates the path of the ring. The black layers are the armies. For example, the one at the bottom is Sauron's army bundled by the level of detail technique.

Visualization of the 2012 US presidential election. It contains 89,174,308 tweets, 900 opinion leaders, and 63 time frames. The opinion leaders are organized by a 2-level topic hierarchy and are bundled together by the LOD technique. Each color represents an opinion leader group. Three interesting attention transition patterns related to (a) the Benghazi issue of the Obama administration, (b) the Sensata scandal of Romney, and (c) hurricane Sandy have been identified.

Storyline visualizations, which are useful in many applications, aim to illustrate the dynamic relationships between entities in a story. However, the growing complexity and scalability of stories pose great challenges for existing approaches. In this paper, we propose an efficient optimization approach to generating an aesthetically appealing storyline visualization, which effectively handles the hierarchical relationships between entities over time. The approach formulates the storyline layout as a novel hybrid optimization approach that combines discrete and continuous optimization. The discrete method generates an initial layout through the ordering and alignment of entities, and the continuous method optimizes the initial layout to produce the optimal one. The efficient approach makes real-time interactions (e.g., bundling and straightening) possible, thus enabling users to better understand and track how the story evolves. Experiments and case studies are conducted to demonstrate the effectiveness and usefulness of the optimization approach.

To evaluate the usefulness of StoryFlow, we conducted a semistructured interview with three domain experts, a film professor (User F), a sociology PhD student (User S), and a professor (User P) in media and communication studies.

User S was very impressed by the visualization and commented "It is stunning and engaging. It definitely outperforms the static infographics that I have seen before." User P said, "StoryFlow would be particularly useful for data-driven journalism because it not only provides a clear visual summary of events but also shows informative context for investigative analysis". Both scholars suggested several potential applications for StoryFlow. For example, User S indicated that it would be interesting to see the dynamic relationships of the liberal and conservative opinion leaders over time, while User P suggested adding sentiment information to the StoryFlow visualization to provide richer context for further analysis.

Overall, StoryFlow was well received by User F. He commented, "I love this visualization! It is a great way to show the interactions between characters over time, which can definitely help filmmaking." He suggested several use scenarios:

• Film directors can use StoryFlow to understand the story evolution in a movie to make a proper plan for the film shooting.
• Script adapters can use Storyflow to review a script quickly to decide whether to add or remove certain characters or scenes.
• Actors can use StoryFlow to better trace their related scenes and see immediately where and who they will interact with, so that they can better prepare for their performance.

Storyline visualizations were originally invented by XKCD for illustrating interactions among characters in movies. In recent years, the visualizations have been applied to solving different practical problems where dynamic relationships are involved. The followings are some of the typical use scenarios of the technique.

• Egocentric Storylines for Visual Analysis of Large Dynamic Graphs.
• Sequence Bundles are a new visualisation method created by designers in collaboration with scientists. They offer a new way to uncover patterns in biological data suited to human intuition.
• TimeNets is a new visualization technique to trace generational relationships in genealogical data.
• Visualizing the Evolution of Community Structures in Dynamic Social Networks is a research paper published in EuroVis 2011. It demonstrates the use of storyline visualizations to facilitate the analysis of patterns of interaction between social entities.
• Software Evolution Storylines is an interesting work that employs the visual metaphor of storylines to illustrate software evolution.
• Design Considerations for Optimizing Storyline Visualizations is an excellent work published in IEEE InfoVis 2012 to create aethetically-appealing storyline layouts automatically using genetic algorithms. They used the technique to visualize Militarized Interstate Dispute (MID) data. Their results are very intuitive for illustrating the temporal conflicts between conutries in World War II.

Download the paper

@article {YWu2013a,
author = {Shixia Liu and Yingcai Wu and Enxun Wei and Mengchen Liu and Yang Liu},
title = {{StoryFlow}: Tracking the Evolution of Stories},
journal = {IEEE Transactions on Visualization and Computer Graphics (Proceedings of IEEE InfoVis 2013},
year = {2013},
volume = {19},
number = {12},
pages = {2436--2445}
}

The authors would like to thank Yuzuru Tanahashi for providing the comparison data and helping generate some of the comparison examples and Stephen Lin for proofreading the paper.