It’s been well over a month since I started this blog, and it’s long past time for me to finish my first post. The draft has been weeks in the making and while that’s not a very good sign, especially given the length of the article, the upcoming release date of the iPad is helping me along because the subject of this post is storytelling and the evolution of books as a medium, and I’d like to get it published before the iPad is available and becomes more of a distraction than it already is.
Here I’d like to focus on digital books, which following Mike Cane, I will distinguish from ebooks. In particular, I’m interested in understanding if and how digital technology and devices will redefine the notion of the book. I’ve been thinking about this for a long time. Since Apple’s announcement of the iPad on January 27th and Steve Job’s keynote presenting it, the number of people who are also thinking about it has grown substantially. The iPad will offer what e-Ink readers such as the Kindle have not been able to provide, a totable device that will be able to store a complete personal library and display not only black-and-white text, but color video and audio too.
The convergence of these functionalities in a single device has lead many to announce an impending redefinition of the book. Likewise, others have pointed out that personal computers and laptops, devices that are also capable of rendering high-resolution video and audio, have been available for years as digital readers. Despite the versatility of personal computing devices, books have remained books, and we have not yet witnessed any fundamental change in the nature of the medium. While, the form factor of the iPad, it’s portability and tactile properties will lead to a different relationship and interaction with it compared to that of a typical computer, whether desktop or laptop, it remains to be seen whether this difference will be significant enough to generate new forms of media.
In what follows, I am interested primarily in works of fiction. Some of what follows may be relevant to certain types of non-fiction such as histories, biographies, or other works that tell a story, but I am not concerned with textbooks, or other learning materials, for which video and audio can contribute in obvious pedagogical ways.
Numerous articles have discussed the enhanced digital book. I will here mention a few articles representing different viewpoints. I’ve chosen recent articles, most of which revolve around the iPad.
Craig Mod wrote a thoughtful article, Books in the Age of the iPad, discussing Definite and Formless content, and illustrating how non-textual content contributes to the reading experience.
Paid Content reported on a presentation by Penguin Book’s CEO John Makinson: First Look: How Penguin will reinvent books with iPad. Makinson said, “We will be embedding audio, video and streaming in to everything we do.” The accompanying video resembled more an interactive educational application than a revolutionary enhanced book. As pointed out by Mike Shatzkin, publishers experimented with enhanced CD-ROMs in the 90’s. Not only did enhanced CD-ROMs not catch on, they did not seem to lead to a “reinvention of the book.” Writing for TeleRead, Chris Meadows noted,
Just because there is a new capacity doesn’t mean it will be used unless people see a specific need for it. If you want to make a comparison, compare the change from parchment scrolls to paginated books. The “random access” that a book offers compared to a scroll is much the same as a DVD offers compared to a videotape: for the first time ever, you can flip to any page. How many centuries did it take after books came along for books that specifically leveraged that random access to be invented? Books that would not have been possible on a scroll but were only enabled by having random access to pages?
As far as I know, the only kind that comes to mind offhand—“choose-your-own-adventure” books—wasn’t invented until the 20th century.
In all the articles I’ve read and example videos I’ve seen, nothing comes close to illustrating what could be the revolutionary definition of the digital book. If someone has seen one, please let me know in the comments.
I’ve experimented with reading comics on the iPhone, using applications such as those produced by Ave!Comics that take advantage of zoom, transition and other animation effects to create a visually appealing reading experience. While I’ve enjoyed them, and I think using the smartphone platform has created new possibilities to engage the reader, I wouldn’t say they’ve redefined the comic book.
The closest I’ve seen to something that qualifies is Vook. Launched in 2009, Vook has attempted to create a new medium that merges the printed word with video, audio and Internet services that enhance the book’s content and allow social features. It’s an ambitious project and a worthy idea. I’ve downloaded the Sherlock Holmes Experience as an iPhone application and while I enjoyed rereading these classic stories and discovering the video and Internet enhancements, I think there is still a long way to go before such applications can qualify as redefining the medium.
So, I’ve been asking myself, if so many people are thinking about this, if so many people have tried this, and yet so far no one has found anything fundamentally new, why is this? Why is it so difficult?
In fact, to understand this observation, I think it is necessary to understand the different types of information conveyed by the different content elements. I’m sure many media scholars and content creators have already given a great deal of thought to this–much more so than I have–but I’ve yet to see any mention of or treatment of this aspect of the equation. What are the fundamental differences in the types of information conveyed by text, images, video, audio or the spoken word?
Books have included images since before the invention of the printing press. This is nothing new. Images have the ability to render enormous amounts of detail that can be processed in a fraction of a second by the visual system, conveying information that would take pages of text to fully describe. Illustrations and photos also evoke emotional responses and can be used effectively to create a mood complementing that of the written word.
Video makes these images dynamic. Actions can be visualized, a rhythm can be established, a mood can be created. Describing all the techniques of film-making and the way they influence the the viewer’s understanding of and reaction to a story are far beyond the scope of this post. Grant McCracken has written a wonderful little essay on the restless camera technique invented by Leslie Dektor that provides a glimpse into the almost indescribable ways the camera shapes our experience of a story on film.
Audio is something else entirely. Audio can convey the spoken word, sound effects or accompanying music. Each of these is an art of its own, activating different systems of inference in the brain that shape our reaction to and understanding of a story. Music especially has the ability to evoke strong emotional responses. Audio can tell a story by itself, and by this I am referring not to audio books but to experiences such as radio plays, operas and musicals, the latter of which usually also include visual elements.
However, of all the different elements that I can think of that could make up a digital book, text is the only one that allows the reader to share the experience of the characters in the first person. It’s the only one that allows the reader to participate (virtually) in the story as if he were present. Although the viewer watches a video, he can only observe. He can’t participate. He can’t know the thoughts and feelings of the characters; he can only imagine or infer them. Only the written word in the form of a book allows the author to communicate to the reader what it is like to be the protagonist in the story, allowing him to understand it vicariously.
There has been a great deal of innovation and experimentation in fictional narratives. Most of us are familiar with innovations such as those involving the sequential nature of the story–the chronology, flashbacks, and changes in perspective. One novel that stands out in my experience is Faulkner’s The Sound and the Fury, a technical masterpiece in which the story of the Compson family is told through the eyes of four different protagonists. It is difficult to imagine being able to tell a story so fully in any other way except through the eyes of different participants.
In fact, we could convey the same depth of experience if the reader could experience the story himself in a virtual way, instead of simply reading and imagining it from the perspective of different characters. In an episode from Star Trek: The Next Generation (“The Inner Light”) Captain Picard is rendered unconscious by an alien probe, and during his lapse of consciousness, he lives out the life of an inhabitant of a dying planet. In this way, the alien race has transmitted to him knowledge of their world, their history, their way of life in a way that would otherwise not be possible.
Perhaps, no certainly, someday such a method of storytelling will be technologically possible. Until then, the only other medium I can think of that provides a storytelling experience rivaling that of the written word is games. Games allow the players to experience a story through the eyes of a character, and through role-playing, different characters and scenarios can be explored to provide a complete understanding of all the different facets of the story. This then is the strong link between the book and the game. However most games are not primarily about telling stories, and this link is therefore a complex one.
Thus it seems that books, through the written word, provide a unique way to experience a story. If true, this explanation may describe what is special about text and perhaps why it is so difficult to redefine the nature of the book. Additional elements that can be added–sound or video–can only contribute to the book in significant but non-transformational ways. Such enhancements can be extremely valuable in transmitting knowledge, for example in works such as textbooks, cookbooks, guidebooks, for example. However here I’m talking about transformational elements that contribute primarily to fictional works.
Furthermore, research has shown that humans are incapable of true multi-tasking. While music contributes to video experience, and most of us can read while listening to music, watching a video and reading text are fundamentally sequential activities. How many people can fully enjoy the visual aspects of a film while reading the subtitles? Perhaps then looking toward the redefinition of the book leads us to understand that different elements provide unique ways of experiencing a story, which in turn leads us to transmedia: the telling or re-telling of different aspects of a story through multiple platforms.
A well-crafted transmedia experience will take advantage of each medium of communication to present parts or aspects of the story in the most compelling way. For this reason, transmedia can be powerful and immersive.
But does this explain why there is no book revolution? Perhaps we have been looking for it in the wrong place. Perhaps the real digital book revolution lies in the reconstruction of these different parts of the story across diverse media, not in a single converged multi-media experience.
An overview of transmedia can be found in Transmedia Storytelling 101 on Henry Jenkin’s blog Confessions of an Aca-Fan.
The well-prepared mind 