Can the image be turned around? Can we imagine the Ocean as a source primarily for readers rather than writers? Could there be a "story space" (to use Michael Joyce's resonant expression) like the Ocean, in which a reader might take a dip, encountering stories and story-segments as he or she flipped and dived? In these waters, turbulences created by the swimmer's own motion might cause an intermingling of the Streams of Story; the Ocean is a dynamic narrative region, a Heraclitean river into which one could never step twice, a lake of Heisenbergian uncertainty where the very attempt to examine a particular story-stream transforms it. What a goal to create such an Ocean! And how suitable an ideal for an interactive fiction!

Fiction, Cinema and Cybernetics

My own work is in the pull of a pair of forces that have defined the late twentieth century--the Cinema and Cybernetics, the Projector and the Computer. I have made two installations that incorporate computers and moving images that I call works of interactive cinema. In both Sonata and The Erl King the participant's (inter-)actions affect the temporal conglomerate of images and sounds. The computer itself is not a medium or a tool, but a device which can be used to control existent media, so the questions that arise are about how cinema changes when its apparatus is linked to a computer--just as one can investigate changes in the structure of cinematic communication when sound was invented.

The first two questions to arise for me in thinking about an interactive cinema can be posed in quite traditional terms: what kind of story will fit the medium, and what will be the grammar of its telling? Early on I realized that one would have to find a subject for which the sequence of events was not of central importance, since if the viewer is to wander around and through the narrative, the order in which the depicted events are accessed would have to be open to variation. And this requirement led me directly to Freud's techniques of dream interpretation.

A Branching Structure

The analysis revolves around the dream image of the staring wolves, introduced early on by the patient. Freud describes the process of gradually uncovering the components of the dream, linking each element with an event, a character, or an emotion remembered but perhaps suppressed. The dream's significance for the dreamer, manifested in the overwhelming emotional effect it had on him and the fact that it remained in his memory for decades, led Freud to seek further explanation. He finally accounts for this power in his proposal that the dream encapsulates the dreamer's greatest fears and desires, as transformed memories of the events that first produced them. For my purposes the details (and--it goes without saying--the `truth') of the dream-analysis are not important. I wish only to appropriate certain aspects of Freud's methodology in my own search for a paradigmatic story structure suitable for an interactive cinema.

Condensation is the key concept. The dream is formed by compressing and combining a set of mental entities. The dream can function in the dreamer's mental framework as the distillation of a set of emotional charges. The dream's powerful affect comes from the fact that, in an important sense, it embodies a set of memories and the specific emotions linked with them. Repeatedly Freud stresses that there is no universal symbol translation table--every element of the dream image, and every property of every element, is understood by the dreamer in his own individual way. Each element substantiates a combination of particular fears, hopes, desires or beliefs, transformed, by the laws of the unconscious, into a component of the dream image. Seeing the images through the dreamer's eyes--identifying the underlying atomic parts and understanding how they are altered by the dreamer's mental process into the dream image--is understanding the dream. In this understanding is written a page or several chapters of the biography of the dreamer.

Freud's notion of a dream is a conception of a narrative-type based on a hermeneutic method. Unravelling a dream reveals the narrative of the dreamer's interlocking emotional states. But it is not a narrative that unfolds in time--all the elements are simultaneously present. Freud goes to great trouble to convey this atemporality, but even for him it is a notion that eludes expression, since, after all, his own mode of communication--writing--is, of necessity, linear, one word following another, forming paragraphs that follow one another, etc., while his conception of the dreamwork is, by its very nature, non-linear, unsuited to the writing forms of the early 20th century.

Considered as a narrative structure, the underlying elements of the dream can be revealed in any order whatsoever, and the same story will emerge. Thus, it is truly a narrative without specificity of sequence.

The Interpretation of Dreams

The elements associated with a particular dream-image are not by themselves sufficient to define the biographical narrative underlying the dream. This would be a gross oversimplification. Of course it is also essential to incorporate how the elements are transformed and combined into the dream. Often this syntax and its application can be expressed only verbally. It is difficult, for example, to imagine an effective visual expression of the transformation of something into its opposite (from "staring" to "being stared at," or from the ornate motions of sexual intercourse to the stillness of the white wolves), or the transfer of a particular quality from one object to another (as the color white is lifted from sheep and flour and attributed to the wolves). Freud's interpretation of the dream is far more than a simple compacting of memory-images into one conglomerate: the grammar of the image-elements' metamorphoses and rearrangements is as significant as the elements themselves.

I am not suggesting that the principles of condensation and displacement could not form a foundation of a visual narrative, but only that some depiction of the types of transformation will have to be incorporated alongside the results of the transformations. The point, to reiterate, is to develop a type of narrative that can retain its identity and make sense independent of the sequence of events. Thus, in Sonata, I found that I needed to make the formative elements of the dream into components of the dream-narrative--without them it became merely a collection of scenes connected only by association.

Desire

Furthermore, the impossibility of impacting on the cinematic is one of the sources of our pleasure in it. "Don't go up(/down) the stairs!" we inwardly cry out while watching Hitchcock's Psycho, first to the private detective Arbogast, and later to the heroine Lila Crane, all the time knowing that however deeply felt, our distress will not influence their behavior. The experience of suspense would be fatally distorted by the elimination of inevitability in the characters' actions. If Lila could turn back because of our pleas, the entire effect of the horror film would dissolve. Much of cinema's power over us is our lack of power over it, and, in this sense, suspense is a paradigm of cinematic response. It could be argued that the introduction of viewer impact on the representation is a destructive step for the cinema. The removal of the possibility of suspense is the removal of desire from the cinematic, and, ultimately, the removal of the very fascination of the medium.

To find interactive forms in which desire can be sustained will require the construction of a new cinematic grammar. And, to be successful, this search, this construction-process, must foreground that aspect of cinematic communication that is fundamentally a grammar of temporality.

Time. Time. Time.

This gives us the first sense of cinematic time--it can be called Screen Time: the clock-time required for the filmstrip to run through the projection apparatus.

The second sense of time is that of the world depicted in the film--and here the limits are more or less the cinema of story-telling, and its Diegetic Time. A cinema narrative may jump forward, eliminating decades (or centuries) in a single cut; or slide back, using perhaps one of the various narrative strategies that fall under the category of flashback; or remain in the present, so that a given passage of film denotes a continuous passage of time. This latter case (the most frequently used) still allows for a broad range of variation: one continuous two minute portion of a story can occupy five minutes of screen time, while the next portion compresses seven years into as many seconds.

Meanwhile...

Experienced Time. Detective Arbogast's walk up the stairs seems painfully extended, so that his stabbing at the first landing is a dreadful shock; this prepares us for Lila's later descent into the basement, which stretches time even further as each step seems to last a full minute, for we now (rightly) expect the worst to be waiting for her. On another end of the scale, a contemporary action film can make us feel as if no time passes during its 100 minutes, while in another discourse entirely, a film like Wavelength (Michael Snow, 1968) insists on equating real and depicted time, a position transformed into an ideology in the 1970s by such filmmakers as Peter Gidal. The relationship of film grammar, plot, and the experience of time is a fertile area of study, especially since the compression of time is undoubtedly one of the major determinants of cinema's phenomenal success.

The question is: what happens to cinematic time when viewer input becomes a component of the screen amalgam? To what extent does the incorporation of viewer impact keep time real, canceling out the magnetism of cinema itself--when does it cease to be cinema and become "multimedia" in its drab information-delivery costume, the slick transmission of data in fields of "hot spots," "buttons," and point-and-click menus?

The Kuleshov Effect

The Liberation of the Filmstrip

The equivalent in my interactive cinema is formed by a sequence in which the middle term is produced by an action of the viewer. If the viewer does not act, the first shot continues. But on action by the spectator the B-shot appears, then, after an appropriate period, the A-shot reappears, perhaps transformed by the interspersed shot, perhaps unchanged. In Sonata this structure is used as a bridge to an alternative point of view (for example of another fictional character, or of the author); as a jump to an earlier (or later) time in the story; as a glance at a different depiction of the narrative situation (e.g., a classical painting of the Judith and Holofernes theme rather than its continuing narration by a story-teller); or the momentary introduction of a parallel narrative line. Does the sequence still denote a continuity of space and/or time? The interpretive mode the viewer takes toward the new material is associative. Because the new image or scene was produced, i.e., brought on screen, by the viewer, he is forced into connecting to the image it replaces--an act of association, rather than spatio-temporal suturing. In Sonata this act is reinforced by two strategies:

1. the automatic return to the previous image, so that it seems that the image produced by the viewer interaction is a temporary interruption of a continuing logic;

2. audio continuity: the sound from the first image continues through the interruption, which reinforces the impression that the viewer's actions are disturbing the natural flow, thus demanding that a sense be made of the new complex.

Experienced Time, on the other hand, becomes open and indeterminate. At one extreme the viewer can find himself in the extended time-instant of the computer hacker or videogame player compulsively acting on the screen image. It is this semi-hypnotic state that allows the computer programmer to spend twenty-four hours at a stretch in front of a CRT, "jacked in" as the novelist William Gibson puts it. In his fiction Gibson often compares the state to that produced by imagined mind-altering drugs of the future. In this mental condition, the user's impact on the screen output is paramount, while awareness of content and interpretive distance are subordinated to action. Videogames and virtual reality environments are often designed to stimulate this condition--the content is minimal and ancillary to the actions of the user, which are immediate and powerful, either floridly destructive requiring hand-eye coordination, or effortlessly navigational: and most often a combination of the two. Unlike a virtual reality environment or videogame, however, changes in interactive cinema are driven by content, and consequently compulsiveness will not be the overriding ingredient of the mental state of the viewer. Here the need for evaluation, interpretation and understanding are in the foreground, though the obsessive need to fully explore the narrative space can serve well as an incentive and accompaniment.

Freed from the predicament where the apparatus alone dictates the temporal experience, time can now expand or contract based on the extent of the viewer's involvement or attention, no longer only because of the hills, gullies, and plateaus, the changes in elevation of the plot. One can imagine the user of an interactive cinema surfing between compulsive input, loss of self in the flow of the narrative, and a sense of distance and control of his own experience of time, as the tides of the story ebb and flow based on his own actions on and in it.

The notion of suspense, for example, can be retained but transformed. If the viewer identifies with a character, seeing him as transfixed with horror at one moment, overcome with relief at the next, there may be some hesitation about accessing the cause of his distress. Now a new emotional affect, begins to emerge. "Don't look behind the door!" we inwardly cry--but now whether the character opens the cellar door is determined by us, and the vacillation, the hesitation, related to a particular experience of suspense, will put the viewer, unexpectedly, in a different grip of the screen. The new pull is a hook of agency--whether we have to face the horror that we fear and are fascinated by is now our decision, and in an effective work we will be equally compelled in both directions.

Then What Can the Interactive Cinema Depict?

We ard multi-tasking units. We can whistle and daydream while working, fantasize while having sex, speak the English translation while listening to the German, and so on. And we can switch from one mental activity--one state, one condition--to another, instantly and without effort. It is easy and natural for most people to keep many thoughts and perceptions simultaneously active in their minds, transferring from one to the next at will, so that their current inner experience is a conglomeration of perceptions of the present, memories of the past, hopes for and guesses about the future, along with beliefs and fears independent of time markers, dreams, imaginings, pains, etc. Each mental element forms an undercurrent in what is happily called the "stream of consciousness," and navigating these waters is part of what it is to be human. Rushdie's Ocean can be heard in these shells.

How do we move from one mental entity to the next? One thing is certain: it is nothing like making a selection from a list. The "menu" model incorporated in contemporary computer software is aptly named--using it is like negotiating a path through a meal at a fast food restaurant. Switching between streams of mental activity involves responding to hardly perceptible internal and external cues, much as one rides a bicycle around obstacles, keeping balance by slight shifts in position, changing direction by combining such shifts with handlebar adjustments and greater weight adjustments. Except in the least significant cases, we affect things in our lives not by making choices, but by actively responding to situations--with speech acts or in behavior, and equally by silence or inaction. Only in restaurants or department stores are we faced with a closed list of alternatives. The interface of an interactive cinema cannot restrict itself to a model of choice, though this does not mean that choice is entirely banned. Response is the operative concept.

Open Issues: What Isn't the Interactive Cinema?

But it will not be a linear story where viewer input determines what-happens-next. Such a structure does not contribute to the notion of interactive form since everything that appears will remain within the limitations of the linear--the fact that he has selected which line the story takes is irrelevant. This particular structure becomes interesting only when the viewer is exposed to different hypothetical situations, so that she can see what would happen if the characters took this turn, that path. Only in this case might the overall experience of the piece retain the quality of a story space of multiple narratives simultaneously present for exploration.

To put this point more generally, registering response alone will not satisfy the requirement that the interactive cinema incorporate the interest or desire of the spectator.

The basis of the interactive cinema is that the viewer has some control over what is on-screen. He or She knows that what is there will change if she or he acts, that it would have been different if he or she had acted differently earlier. Thus, the viewer is aware of a fundamental indeterminacy. I have called this epistemological state a subjunctive relationship to the screen--the viewer is constantly aware that things could have been otherwise. This state is grounded in the viewer's continual knowledge that what is on screen is a result of her interactions--inaction, naturally, counting as decisively as action.

The subjunctive mental state is in direct opposition to the epistemology I identified as essential to the linear cinema, a conception of the screen complex as unalterable, the events in the diegesis as inevitable. In an advanced interactive cinema, everything will be in flux, open to the possibility of change--like conversation or competitive sports--and the more sophisticated the system, the more fluid and wide-ranging the possibilities. Awareness of this liquidity has radical consequences for a viewer's relationship to the cinematic material. In terms of the Lyrical, the exploration of a single image-moment and its underlying expression-set, the success of a work of interactive cinema depends on its viewer's recognition that behind each element of the screen- complex there is a potential set of cinematic data that supports it, accounts for it, enriches it, or explains it.

There is another factor too: the viewer must be kept always aware that it is his, her action on a particular image that has produced these new sounds or pictures, and techniques to foster this awareness must be developed. In my judgment, the most immediately available techniques can be found in the language of montage. A deliberate use of film editing strategies can keep reconvincing the viewer of the non-arbitrariness of connection between old and new elements, between the elements already there and those produced by viewer action. Once the interactive work has brought the viewer to the idea that his actions on the screen complex alwayq contribute to the continuing significance of the work, then the associations can roam more freely than in the city zoo of conventional narrative film. Now the fact that the viewer feels that he produces the new elemelts predisposes him towards finding links, associations, connections that may not have operated in his response to a conventional cinematic work.

Back to the Ocean

Tying Up & Getting Out

And yet. And yet.

However.

Butbutbut.

Real Timecannot be trashed with the need for closure like potato peel or an old jalopy. There is, there always will be, a Beginning, an End to a viewer's exposure to an Interactive Cinema work, and a Time Between. She walks up to the device, she interacts with it, she walks away. He walks up, sits down, stays a while, gets up. Do we place a viewer with an interactive work until it starts to repeat on him like rote learning or yesterday's overspiced entree? The Interactive Cinema will succeed only if, in retrospect, the experience seems substantial.

All and any loose narrative ends will never be knotted; this is one of the features (i.e., not bugs) of interactive cinema. If a viewer navigates through a mass of material, some of it will be seen and some won't, and surely some of what isn't seen earlier will raise issues that remain unresolved in what is seen later. But a system can be sensitized to repetition, either so as to avoid it, or so that as soon as repetition starts the viewer is offered the opportunity to enter a structurally different region, a territory of culmination or summary. In general terms, a map of territory covered can be kept by the system, and once a certain area has been explored, closure possibilities can be introduced.

In The Erl King, after certain segments have been repeated, a box with the work "END?" appears on the screen. If this box is touched, it produces a mildly interactive segment that starts with images of a few key production crew members touching the inside of the video screen from within the monitor, followed by a rapid series of production stills. A viewer can switch on or off two cardinal theoretical texts overlays--texts by Wittgenstein and Baudrillard that describe something of the theoretical underpinnings of the work--by touching different areas of the screen.

Sonata reserves two narrative segments that are acknowledged and indicated throughout the piece. If the viewer perseveres, following a story through to one of its climactic moments, the reward will be one of the two culminating murder scenes, one decorated with the blooming image of a blood-fountain, the other with the voluptuous sounds of a blade severing flesh and splintering bone. The possibility of viewing these scenes emerges when the viewer has covered a certain amount of the narrative ground of the piece. And after the murder the work ends or, more precisely, returns to the beginning.

All this is to say that despite its need for an opened narrative, closure cannot be banished from the Interactive Cinema. Remove the imminence of closure and we begin to drain cinema of desire. Closure must be recast in a more radical light.

The most fruitful possibility for me at this point, based on my interest in multiplying and intermingling narratives, is that several story lines continue until one, some, or all of them end. Here the idea is that numerous Diegetic Times are constantly flowing forward, many narratives operating in time simultaneously whether or not the viewer encounters any particular one. Narrative Time in this model always moves inflexibly on. This provides another picture of a form for the Interactive Fiction Cinema, a picture of multiple narrative streams not interconnected by a central image, theme or scene. The viewer navigates from one current to an adjacent one in a constantly flowing river, crossing between streams of story at moments of similarity or juncture. Or, to descend one level more, they might rather be thought of as potential narrative streams, elements themselves unformed or chaotic, but taking form as they intersect, gaining meaning in relation to one another.

In Sonata I attempt this by juxtaposing the stories of Podsnyeshev (the anti-hero wife-killer in Tolstoy's Kreutzer Sonata) and Judith (the Apocryphal heroine who decapitated the enemy general Holofernes). Both narratives progress, but it is at their connections, where the viewer can cross from one to the other, that they come into focus and take on meaning. A viewer will access an episode of one or the other narrative but not both, and their forms are similar enough that their plot movement can be seen as concurrent. Each of the two killers--Podsnyeshev and Judith--is reflected in the light of the other, since each emerges out of the story context of the other. And thus an act of interpretation is forced on the viewer: the morality of each character comes into question when they are placed together, especially because the former is presented as Evil and the latter as Good.

Without the act of interpretation, the stories are raw and problematic, but when clashed together at the points of interaction, a judge's role is forced on the viewer. As Eisenstein recognized explicitly, Griffith at least implicitly, and Kuleshov claimed as his own, meaning in cinema is determined by context--in the multiple narrative interactive cinema, context is in constant flux, the elements appearing always different as their surroundings shift.

As the viewer is drawn in by the act of interpretation, now the magnetic attraction of the Interactive Cinema can be felt, and the question of Experienced Time finally answered; for it is here that the Hacker mindset takes over--as we jack into Gibson's Cyberspace. Umberto Eco describes the state somewhat more suggestively than Gibson, though Eco is talking about the travels of a steel ball around an electric pinball surface, not a sprite in a graphic representation of a data environment. Parallel to Eco's pinball machine is a game like Tetris, in which the player arranges falling shapes into an unbroken plane, a theatre of geometry and spatial anticipation one often sees played on long airplane flights--as the time sense is held in abeyance, the magnified time of the cramped Atlantic crossing is compressed into a single moment of hypnotic focus. Tetris's hook of involvement is the desire for closure, for the completion of the pattern, an end that is always attainable but just out of reach, like Eco's "brink of orgasm." It is in this space that the machine absorbs time, providing in its place the never-quite-fulfilled promise of consummation.

Ending, Open Ending

There is the very central question of what function is left to Narrative in our Cybernetically Determined, Information Laden Era, as we travel along the Information Superhighways without stirring from our desks. Do we still need narrative to provide lessons in living and dying, do these lessons come to us through other channels ... or don't we need such lessons any longer? Then there is the very important and subtle idea, expressed by philosophers in the wake of Heidegger such as Paul Ricoeur, that the conceptual relationship of narrative and time is reversed: that we impose a (false) linearity on time because our stories about ourselves and others, our formation myths of what it is to be human, take shape as linear narratives, and upsetting this notion will change our understanding of temporality and hence our understanding of the world and ourselves.

But I must stop. It is late, my eyes hurt from looking too long at the CRT, and I'm afraid I'm getting the flu.

Printed in MFJ No. 28 (Spring 1995): Interactivities