topical media & game development

talk show tell print

print / present / tag(s)

preface -- topical media

This book provides a concise and comprehensive introduction to multimedia. It arose out of the need for material with a strong academic component, that is material related to scientific research.

Indeed, studying multimedia is not only fun. Compare it with obtaining a driver license. Before you are allowed to drive on the highway, you have to take a theory exam. So why not take such an exam before entering the multimedia circus.

Don't complain, and take the exam. After all it makes you aware of the rules governing the (broadband) digital highway.

themes and variations

So, who is this book meant for? It is meant for the student or reader who is looking for a quick introduction to the main topics in multimedia. The twelve chapters provide a concise overview of the themes and trends in current multimedia practice and research.

The themes and variations addressed in this book may be summarized as follows.

themes and variations

  • digital convergence -- all for one, one for all
  • broadband communication -- entertainment
  • multimedia information retrieval -- as an afterthought?
  • multimedia and game application(s) -- from design to development

To explain in somewhat more detail, digital convergence may be characterized as the coming together of data (including audio, video and information) in a possible multitude of platforms, to which these data are delivered by a variety of (broadband) communication channels. In fact, the increasingly powerful communication infrastructure due to the popularity of the Internet and the World Wide Web, leads to an almost universally accessible multimedia (information) repository, for which (unfortunately) the notion of (multimedia) information retrieval seems to have occurred only as an afterthought. Digital content design is only one step in the process of multimedia application development. Important issues in multimedia application development are, apart from project maanagement, data representation, navigation, presentation and usability.

An underlying thought that motivated the writing of this book is that somehow the gap between authoring and retrieval should be bridged.

In other words, either by developing the technology for extracting features or attributes from multimedia objects, or by applying content annotation for such objects, multimedia information retrieval should be considered as a necessary asset to make a multimedia web an effective information repository. In multimedia applications, such as the digital dossier we introduce in chapter 10, the data representation must accomodate meta-information, to support effective navigation and search.

Another line of thought, that became more clear during the writing of the book is concerned with the aesthetics of (interactive) applications. You will find more on this in chapters 11 and 12, that deal with game development.

what do you need to learn

When taking up multimedia as a subject of study, you may ask yourself what you need to know and learn about it. In general, what this book presents is

what do we have to learn?

  • concepts,
  • facts,
  • history,
  • applications
  • standards
  • technology
  • multimedia and games

Let me be frank with you. There is too much information to be digested in a first course. Nevertheless, after studying this book you will have an introduction to multimedia that should be viable for the rest of your (academic) career.

Now, don't hesitate, put yourself to the test and check which phrases and acronyms you are familiar with in the lists given for the subjects of digital convergence, broadband communication and information retrieval.

digital convergence

  • concepts -- digital revolution
  • facts -- from the entertainment industry
  • history -- from Pong to Big Brother
  • applications -- infotainment
  • standards -- MPEG, RM3D, SMIL
  • technology -- TV, PC, DVD


How did you succeed thus far? If you did well, try the second round and test yourself in what detail you have have knowledge about technologies mentioned.

broadband communication

  • concepts -- Quality of Service
  • facts -- compression is needed
  • history -- the internet
  • applications -- entertainment and communication
  • standards -- HTTP, TCP/IP, RTP
  • technology -- cable, (X)DSL


Finally, check to what extent you master the vocabulary of multimedia information retrieval.

multimedia information retrieval

  • concepts -- features, precision, recall
  • facts -- the problem is utterly complex
  • history -- from text to multimedia
  • applications -- digital libraries
  • standards <-- distance metrics
  • technology -- indexing & algorithms

If you are working online, you may click back to the text in the book that explains these notions. Just to make sure whether your impression of familiarity was justified.


I strongly believe that practical work is necessary, also for academics, to get a good grasp on multimedia and game development. Even if your interest is purely intellectual, it pays off to make your virtual hands dirty and indulge in making a compelling presentation.

As an assignment, consider making a presentation that offers an


Annotated Tour in Amsterdam

Amsterdam is the place where I live, and where our students take their courses. You may find it more convenient or natural to replace Amsterdam with a location of your choice.

Online, you will find an elaborated version of the assignment, including an extended description, a working plan, deliverables and hints. In essence though, the intent of the assignment is to make a compelling, not to say artistic, presentation, and to explore the realm of multimedia rethorics.

As a tool you may choose, for example, Flash or the flex 2 SDK, which is freely available.


Despite the fact that some consider the practical aspects of multimedia to be exclusively relevant, the intellectual aspects of multimedia should not be ignored.

Consider the following question, which is directly related to one of the themes underlying this book, that is the complementarity of authoring and retrieval:


This question can only be answered when the student has a sufficient level of experience, insight and knowledge of the field, and is able to relate theory and practice.

Each chapter contains a brief list of questions that may be used as a checklist, to see if you have sufficient knowledge of a particular area. These questions may also be used to prepare exams!

The questions are meant to test for insight, that is the ability to discuss a somewhat broader theme, and knowledge of concepts and technology, covering definitions, applications, historical facts, as well as the technological infrastructure enabling the deployment of multimedia applications.

In addition to the regular material, the book also contains a number of examples and sections indicating research directions. These sections are not meant to be part of the exam, but might provide the student with suggestions for projects or further research. Moreover, both the discussions in the research directions and the material in the appendices presents a vision on what multimedia should be. In effect, I have a strong preference for a programmatic approach to (intelligent) multimedia, as outlined in appendix E. Nevertheless, the bulk of the (regular) material is relevent also for readers with a rather different opinion on what consitutes the essence of multimedia.

how to use this book

The intended audience for this book is

intended audience(s)

The course notes were explicitly written for first year Computer Science and Information Science students. The Information Science students are expected to choose the specialisation Multimedia and Culture, a curriculum provided by the department of Mathematics and Computer Science of the Faculty of Sciences of the VU University Amsterdam. The course has a practical part and a theoretical part, which in combination takes 2-4 weeks, full time study. The book covers the theoretical part. The online version gives a skeleton assignment that may adapted by the one responsible for the course. The online version contains all the material needed for giving a multimedia course, that is

multimedia course

  • presentations for all chapters, including the preface in dynamic HTML slides
  • presentable versions of the MPEG-4 standard, and other relevant material
  • possible exam questions, with back links into the text for quick learning and review
  • seven sample lectures, with additional explanation for the instructor
One additional remark may be made. This is (so to speak) 'a book with an attitude'. It is slightly authorative and directive towards the students, telling them to learn the facts and 'do the exam'. Some students take refuge to learning the 'keywords and phrases'. They are even helped in this respect, since the text uses a 'graphic' layout to emphasize important points, and to allow for a quick recognition of chunks of relevant material.

the artwork

Although a book about multimedia does not need to be a multimedia artefact itself, it seemed better to include illustrations, to avoid the impression of a 'dry' book. Since I did not want to include any redundant diagrams or pictures, I decided to use a personal selection from the history of visual design, games, computer art and video art, not only to spice up the book but also to give the reader a collection of interesting samples. Each chapter starts with illustrations setting the visual theme of the chapter. All other illustrations are, in one way or another, related to the examples or the text of that chapter. Brief comments about the artwork, and an explanation of the visual theme, can be found at the end of each chapter.

about the author

At some point you may wonder whether the author is qualified or authorized to write about a particular subject and, in this particular case, to publish a book about such an elusive notion as multimedia.

Let me give you some personal history. Way back in the seventies, I did a degree in painting at the Gerrit Rietveld Academy in Amsterdam. At the same time, I did a master's philosophy, where I graduated in the field of aesthetics on a comparative study on theories of imagination and creativity, reading writers such as Kant, Husserl and Sartre. Then I got an interest in computer music, after listening to a concert of Xenakis in Paris, and started to work on a PDP-15 (with 4K of memory) at the Institute of Sonology in Utrecht. Leaving all philosophy and traditional art behind, I learned programming, studied AI and theoretical computer science. Some eight years later, I obtained my Ph.D. in computer science and started my academic career. After working in software engineering, and in particular object-oriented software development, I was asked, at the end of the millenium, to set up a collection of multimedia courses, since by then multimedia was coming in vogue as an academic subject. These courses, which include the introduction multimedia, Web3D authoring, intelligent virtual environments, a multimedia casus, and recently also visual design, are reflected in this book.

about the book

What started as a (not so) gentle introduction to multimedia, has grown into a rich (at times somewhat idiosyncratic) collection of topical material about multimedia and game development. Borrowing a phrase from the politics of the seventies, at some point, apparently, the professional became personal, and the personal professional. Nevertheless, the book may still be read as an introduction. It is written in a concise and compact manner, supported by the slides format, which allows for presentation of the material in class, and is illustrated by a variety of images, taken from the arts, design, and multimedia and game projects.

The book consists of the following parts:


  • part i -- digital convergence
  • part ii -- delivery & presentation
  • part iii -- multimedia information retrieval
  • part iv -- applications & tools
  • part v -- game development
Actually, as will become apparent when reading, the book is the result of a series of revisions. Since I started writing the book, the vision of digital convergence has become a reality, game research has become a respectable academic discipline, and attention has shifted from new media to cross media. Moreover, the role of media in our society is subject to change as well. With the Web 2.0, our information society is no loner passive, but part of a participatory culture in which users contribute and ultimately control content. And these developments are not likely to stop or slow down.

As concerns the history of this book, after writing chapters 1-8 (minus 4, which was added later), I extended the manuscript with with chapters 4, 9 and 10. After a thorough revision, I included the artwork, and then decided to subdivide the book in parts, adding chapter 11 and 12 in part v, on game development. The latest revision is the inclusion of another section with each chapter, discussing the latest development(s). Not visible in the printed book are the numerous technical examples, including ajax, flex and wiki experiments, that may help the reader in his/her exploration(s). Over the last year(s), it has become evident for me that the only viable method of learning multimedia is exploratory development. That is not to say that the theoretical material is superfluous. On the contrary. But only in the actual context of developing meaningful applications, meaningful for the individual that is, does the necessity of theory become an obvious truth.


This book is the result of developing the course notes for an introduction to multimedia for first year Computer Science and Information Science students. Hence, first of all, I like to thank the students that had to endure all the rough drafts of this material, and perhaps not to forget my experiment(s) with the presentation format of it.

Further I like to thank Harrie van der Lubbe and Sander Lammers for developing the manual for Director and their support in developing the practical assignment. Also, I like to thank Martin Kersten from CWI for allowing me to join his Multimedia Database Systems research group as a guest for a period of about two years, and Alex van Ballegooij for his active involvement in the RIF project and his coding effort for the slide PROTOs, used to produce the presentation slides for this book and described in appendix B. Also from CWI, I like to thank Lloyd Ruttledge, Lynda Hardman and Jacco van Ossenbruggen, for their effort in thinking about the multimedia course in its initial stages, and Lloyd and Jacco for their involvement in some of the practical work, and Jacco in particular for his knowledge of hypermedia systems that he shared with me during the period that he was my Ph.D. student. From CWI, I like to thank Zsofi Ruttkay for her general interest in 'my projects'. From the VU, I like to thank Andy Tanenbaum for allowing me to use his material on digital video, Gerrit van der Veer for taking the initiative for Multimedia and Culture, Zhisheng Huang for his excellent contributions to the WASP project, Johan Hoorn for our spirited cooperation, and Claire Dormann, for our discussions on the direction the Multimedia and Culture curriculum should take, and for sharing her thoughts on persuasive technology with me. I also like to thank Tatja Scholte and IJsbrand Hummelen from ICN (Netherlands Institute for Cultural Heritage) for their contributions to the multimedia casus, and Gaby Weijers and Bart Rutten from Montivideo for their cooperation and all the video material they so generously provided. Thanks are due to Mark Veldhuijzen van Zanten, Jaap Stahlie, Peter van Kessel, Federico Campanale and Katelijne Arts for providing me with material. Special thanks goes to the student members of the 2004 autumn multimedia casus group, for their collective work on the abramovic dossier, and to Rutger van Dijk for rekindling my interest in C++/DirectX programming by his youthful enthousiasm. I also should not forget two students from the Computer Science master Multimedia, who were exceptional in their dedication and skills, Winoe Bhikharie, who acted (among others) as the manager in the development of the VU-Life game described in 11.2, and Marco Bouterse, who excelled in both DirectX and Half-life shader programming. Also I need to mention Marek van de Watering, a student from Multimedia and Culture, who often surprised me by his sincerity and sensitivity, and Hugo Huurdeman, who convinced me of the viability of interactive (flash) video and who proved to be an extremely dedicated and creative worker. No doubt, I owe thanks to Gaynor Redvers-Mullon who made a serious attempt to encourage me to get the best out of this manuscript, even though at some point I decided to do it in my own way. Further, I also wish to thank Dhaval Vyas, at that time PhD. student at VU, for his interest in my projects and for sharing his thoughts on the panorama project, znd Zeljko Obrenovic, postdoc at CWI, for his enthusiasm, his technical skills and his willingness to assist me with my teaching. I thank Pier Vellinga, who I googled as the spin-doctor of climate change, for involving me in Clima Futura, and not in the least for the pleasant cameraderie we enjoyed during the effort to obtain the dutch national prize for academic communication, and Frans Feldberg, for involving me in the VU @ Second Life effort, an equally refreshing interdisciplinary partnership, with which we got into the 8 o'clock nation TV news, as planned! of course, I should not forget the guys from the communication department of the VU, Egon Compter and Merlijn Draaisma, who perhaps somewhat overestimated my expertise, in their enthusiasm to involve me in their projects. Last but not least, I owe Yiwen Wang (ηŽ‹δΎζ–‡) my deepest gratitude for providing the motivation to continue the work, and to strive for beauty and pleasure, in a serious way.

Finally, I must mention that I owe much insight and material to (among others) the following books and articles:  [Subrahmanian (1998)],  [Forman and Saint John (2000)],  [Chang and Costabile (1997)],  [Ossenbruggen (2001)],  [Vasudev and Li (1997)],  [Klabbers (2006)],  [Grau (2003)],  [Kress and van Leeuwen (1996)], and not to forget  [Zielinski (2006)]. As in any intellectual endeavor, intellectual ancestry can hardly be praised enough. So let me briefly indicate, for each chapter, some of the sources that provided me with inspiration, insight and material:


  1.  [Forman and Saint John (2000)],  [Davenport (2000)],  [Jain (2000)].
  2.  [Chang and Costabile (1997)],  [Ossenbruggen (2001)],  [Klabbers (2006)].
  3.  [Vasudev and Li (1997)],  [Koenen (2000)],  [Visser and Eliens (2000)].
  4.  [Luna (2003)],  [Adams (2003)],  [Fernando and Kilgard (2003)]
  5.  [Subrahmanian (1998)],  [Baeza-Yates and Ribeiro-Neto (1999)].
  6.  [Subrahmanian (1998)],  [McNab et al. (1997)],  [Kersten et al. (1998)].
  7.  [Subrahmanian (1998)],  [Fluckiger (1995)],
  8.  [Fluckiger (1995)],  [Ballegooij and Eliens (2001)],  [Huang et al. (2002)].
  9.  [McCuskey (2002)],  [Bolter and Grusin (2000)],
  10.  [Chapman and Chapman (2004a)],  [Chapman and Chapman (2004b)],  [Klabbers (2006)],
  11.  [Sherrod (2006)],  [Grau (2003)],
  12.  [Juul (2005)],  [Arnheim (1957)],  [Hawkins (2005)],  [Kress and van Leeuwen (1996)].
The material in sections 4.3, 7.1, 7.3, chapter 8, sections 9.3 and 10.2, and section 11.2 reflect my own research efforts. The other material has all been diligently collected from (among others) the sources mentioned.

(C) Æliens 04/09/2009

You may not copy or print any of this material without explicit permission of the author or the publisher. In case of other copyright issues, contact the author.