topical media & game development

talk show tell print

guidelines for narrative construction(s)

Storytelling is vital to human existence. Whether to fall asleep during childhood, to dream about heroism in adolescence, to escape from the daily routine of work as an adult, ..., storytelling is ubiquitous, be it in books, movies, or (even) games. To think about storytelling, I have taken  [Film] as a guideline, since:

film as art

by still being read, the little treatise seems to prove that in spite of all the changes that have taken place in their form, content and function, films are still most genuinly effective when they rely on the basic properties of the visual medium.


With reference to our discussion in section 11.3, entitled immersion is not illusion, we may remark that we actually do not need visual realism to be taken in by a story:


... in film or theatre, so long as the essentials of any event are shown, the illusion takes place

As to the question of why film may be regarded as art,  [Film] observes:

patterns of light

... we can perceive objects and events as living and at the same time imaginary, as real objects and as simple patterns of light on the projection screen, and it is this fact that makes film art possible.

This, by extrapolation, also holds for video games, which may be regarded as a first-person variant/successor of films. As a consequence, both film and games may play with the relation between two distinct frames of reference. We repeat the characterization given in section 11.3:

frames of reference

it is one of the most important formal qualities of film that every object that is reproduced appears simultaneously in two entirely different frames of reference, namely the two-dimensional and the three-dimensional, and that as one identical object it fulfills two different functions in the two contexts.

Taking film as an example on which to model storytelling in games, we may distinguish between the following principles of montage:

principles of montage

  • cutting -- unit length, whole scenes, cuts within scenes
  • time relations -- synchronized, before/after, neutral
  • space relations -- same place (different time), different place
  • subject matter -- similarity and/or contrast
Technology plays an essential role in the production of movies. In particular, montage is enabled by technical properties of film, including:

film technique

  • camera -- position, focus, movement
  • transitions -- fading in/out, dissolving, stills
  • arrangement -- light/shade, color, sound
Film, or movies, being a dynamic medium, essentially involve motion. Following  [Film], we may characterize motion by making a distinction between:

cinematographic motion

  • movement of objects
  • effect of perspective
  • motion of camera
  • montage of scenes
Nowadays, with the rise of digital technology, editing film has changed significantly, and is within the reach of (almost) everybody. However, the basic principles of what was previously called motion pictures still seem to apply, even for CG movies or video games.

the meaning of composition

Where montage employs dynamic characteristics of the (moving) image, the static image itself may be said, following  [Semiotics], to have narrative implications as well:

narrative implication(s)

  • objects -- the items in the image
  • vectors -- (imaginary) lines suggesting interaction
  • gaze -- inward (offer) / outward (demand)
In other words, images may suggest a story, and the persona within an image may express a relation to us, either transactional, directly approaching us, or non-transactional, without the on-looker being ivolved directly.

Image composition plays an essential role in storytelling, since, as  [Semiotics] observe:


composition, ..., relates the representational and interactive meanings of the picture to eachother, through three interrelated systems.

Composition results in visual effects, since representational elements may be put together according to particular mechanisms or systems as they are called in  [Semiotics]:


  • information value -- left/right, top/bottom, centre/margin
  • salience -- foreground/background, relative size, contrast
  • framing -- connecting or dissolving lines
The effects of compositional mechanisms are partly due to physiological properties of the image and partly due to a codified interpretation in a particular social context. For example, information value is clearly determined by social context:

information value

  • left/right -- given versus new
  • top/bottom -- ideal versus real
  • centre/margin -- important versus marginal
In section 5.4, we discussed how we applied these interpretations in the design of information flows in PANORAMA, with, we must admit, with surprising effect!



example(s) -- edgecodes

The edgecodes documentary film by Phillip Daniels gives an inside account of film editing, a history of the evolution of editing conventions, as well as an account of the technological innovations of the late 20th century and their impact on film editing. It was shown at the documentary film festival IDFA 2004, in Amsterdam. Movies were, as Daniels staes, the new artform of the 20th century, which distinguishes itself from other artforms by ... editing!

The film begins with the statement such as the concept that a film is shot is entirely false, a film is not shot, its is built, continuing with the statements that the message of the movie medium is that of transition, and that a movie must have a beginning, middle and ending, but not necessarily in that order.

The documentary is highly visual, after all it is an editor's movie, and contains many fragments from wellknown movies and interviews with famous directors, among which George Lucas, who introduced the editoroid in the eighties, an editing machine built with at the time modern computing technology. George Lucas, image left above, explained the introduction of his editing machine by saying that he wanted to have a system, ... that is intuitive, obvious, ... and hihgly malleable, ... visual .... He wanted a machine that allowed him to use his moterskills, without the intervention of an engineer. But in the interview he admitted that they were on the bleeding edge in those days. Nowadays, real-time editing, with computer graphics (CG) support is (finally) feasible. See chapter 4.

research directions -- multimedia in context

In 1998, I organized a multimedia course for PhD. students, together with Lynda Hardman (CWI), entitled: multimedia in context. When defending the position of multimedia & game development in the computer science curriculum, it bacame once more necessary to reflect on the relation of multimedia to its (various) context(s).

First and for all, we may identify the scientific context of multimedia, which includes many seemingly unrelated areas of science:

scientific context

  • mathematics -- matrix algebra, transforms
  • physics -- game physic, particle systems
  • computer science -- technological infra-structure
  • information theory -- compression and delivery
  • media theory -- history of communication
  • semiotics -- theory of meaning
Secondly, we have the societal context. Given the (explosive) rise of the creative industry the list given here is rather poor, and should include at least the various communities that are coming into existence.

societal context

  • cultural heritage -- digital dissemination of art
  • education & communication -- presentation of concepts and examples
Obviously, there is a strong technological context of multimedia. Without the technology, multimedia and game development would not qualify to be seen as an academic discipline.

technological context

  • modelling -- objects, characters
  • interaction -- game programming
  • architecture -- game engine design
  • rendering -- programming the graphics hardware
Finally, there is a creative context. However, from my perspective this is intimately tied to a technological context, since at this stage there is still a great need for explorative development, to discover new applications and the aesthetics governing these apllications.

creative context

  • visual design -- style, models and attributes
  • story telling -- narrative structure
There should be no need to emphasize that all these contexts are itself one way or another strongly related. In other words, multimedia & game development is intrinsically a multi-disciplinary affair, which even cannot be so easlily taken out of a societal context, in that the value of new media applications is ultimately determined by its adoption in the new digital culture.

(C) Æliens 04/09/2009

You may not copy or print any of this material without explicit permission of the author or the publisher. In case of other copyright issues, contact the author.