topical media & game development

# The concept of hypermedia

Due to the rapidly evolving software and hardware technology, including the mass storage provided by CD-ROM, hypermedia have become accessible to the general public. Hypermedia provide the technology to manage large amounts of information of various kinds by means of the computer. From a historical perspective, hypermedia may be understood as the combination of hypertext and (interactive) multimedia. Hypertext was conceived shortly after the second world war as a means by which to support mechanically the storage and retrieval of large amounts of information. Interactive multimedia have been developed as the result of integrating computing with media (notably audio and video display).

#### Hypertext -- nodes and links

• non-linear text -- As we may think
•  [Bush]

#### Machine-supported links -- text as a network

• virus $->$ infection

#### Hypermedia systems -- programmable media

• components -- text, graphics, audio, video
• links -- relations between components
• presentation -- structured display
##### document.write(' <a href=');if (!slidemode) { document.write('@slide-'); } else { document.write('quote-hypermedia.html#slide-'); } document.write('12-hypertext'); if (!slidemode) document.write('.html');document.write('>');

slide: The concept of hypermedia

A hypermedia system may, in a general way, be conceived of as consisting of a collection of stored components (containing text, graphics, audio or video fragments), a collection of links (which allow the user to retrieve components in an associative manner) and presentation facilities (to enable a structured display of the components selected by the user). The ease of use of a hypermedia system depends to a great extent upon the organization of the material (components) and the presentation of the associative relations (links) between the components. In particular, a powerful user interface is required to allow the user to navigate through the information without losing orientation.

#### Direct manipulation interfaces

Graphical user interfaces with buttons and menus, and windows for the display of text and graphics, have become the standard for a variety of applications, including text processing, spreadsheets and CAD systems. Originally menu-based, user interfaces have developed into what are often called object-oriented user interfaces, or more appropriately direct manipulation interfaces. Direct manipulation interfaces allow for the association of actions with the items displayed. For example, clicking on an icon may result in opening the window associated with that icon. Conversely, clicking on a specific part of the window may result in iconifying the window into a small graphical item on the screen. The reason for calling such interfaces object-oriented, obviously, is the analogy with sending a message or command to an object considered as the target. However, direct manipulation interfaces need not necessarily be implemented or designed in an object-oriented way, although in my opinion they had better be.

#### Direct manipulation interfaces -- object-oriented

• $target <- command$

#### Requirements

WYSIWYG

• documents -- open, close
• blocks -- select, copy, paste
• links -- $source -> destination$
##### document.write(' <a href=');if (!slidemode) { document.write('@slide-'); } else { document.write('quote-hypermedia.html#slide-'); } document.write('12-direct'); if (!slidemode) document.write('.html');document.write('>');

slide: Direct manipulation interfaces

Hypermedia systems may be used for browsing or authoring, or a combination of both. Browsing hypertext requires an interface resembling the interface of a WYSIWYG (What You See Is What You Get) text processor, in that it must support a readable display of the material and the facility to open and close documents. In addition, it must enable the selection of specific parts of the document, which may be blocks or single words, to activate links that lead to another block. See slide 12-direct. Often, as in the Intermedia system described in  [Meyro86], the user is allowed to copy parts into a private notebook, containing the material selected while browsing the system. See slide 12-navigation. Naturally, the display facilities of hypermedia as opposed to hypertext systems must be far more encompassing. These facilities and the issues involved in authoring will be discussed in section hypermedia-model.

One of the major problems in hypermedia interface design is the support for navigation. Experiments indicate that users have considerable trouble in maintaining a sense of orientation, and often rely on using a keyword index or the traditional hierarchical structure of the document instead of the associative access allowed by following links. See  [McK91].

Intermedia

• web = documents + links

#### Retrieval by attributes -- selection

• block -- to apply filters
##### document.write(' <a href=');if (!slidemode) { document.write('@slide-'); } else { document.write('quote-hypermedia.html#slide-'); } document.write('12-navigation'); if (!slidemode) document.write('.html');document.write('>');
To support navigation based on content characteristics or personal preference, the Intermedia system allows for the creation of webs. A web is a particular collection of documents and links, which is a subset of the total collection satisfying user-specific criteria, such as field of interest or level of aptitude. Moreover, the Intermedia system supports the graphical display of the structure of a web by means of maps, showing the blocks and the links between them. An example of a web is the private notebook mentioned above, which contains the blocks and links satisfying the condition selected.

To support the selection of (parts of) documents and links, the Intermedia system maintains attributes for both blocks and links. The value of block attributes determine whether a particular block is selected or filtered out, and the value of link attributes determine whether activating a link results in traversal or not.

The latter facilities, by the way, suggest that hypermedia offer significantly more than merely user interface gadgets. My opinion in this respect, following  [Wood90], is that hypermedia technology may be regarded as a unifying paradigm allowing the integration of disparate elements in a common presentation framework.

## Hypermedia applications

Hypermedia technology may best be thought of as technology that may be embedded in a variety of applications. This is reflected in the classification of hypertext and hypermedia systems given in  [Conklin87], pictured in slide 12-classification.

#### Classification of hypermedia systems

[Conklin87]

• macro-literary systems -- publishing, reading, criticism
• problem exploration tools -- authoring, outlining, programming
• browsing systems -- teaching, references, information
• general hypermedia technology -- authoring, browsing, collaboration

#### Applications -- embedded

• CASE, decision support, catalogs
##### document.write(' <a href=');if (!slidemode) { document.write('@slide-'); } else { document.write('quote-hypermedia.html#slide-'); } document.write('12-classification'); if (!slidemode) document.write('.html');document.write('>');

slide: Hypermedia systems -- classification

[Conklin87] makes a distinction between macro-literary systems which support publishing, reading and criticism and may serve as online annotated libraries, problem exploration tools which support authoring, outlining and programming and may be characterized as idea processors, browsing systems which support the display and content-based retrieval of information and may be used for teaching, reference or online help, and systems that explore general hypermedia technology which offer a combination of authoring, browsing and collaboration facilities (of which the Intermedia system is an example). In general, for systems that primarily support browsing, the presentation facilities will be most important, whereas for authoring systems the functional or programming capabilities will count most. Applications for which hypermedia technology is an essential ingredient encompass CASE tools, decision support systems and product catalogs. All these applications have in common that a large amount of material, that may come in various media formats, needs to be related, to allow the user flexible access both in a structured and associative manner. Despite the differences due to the particular application domains, we may discern a common hypermedia model underlying the associative structure of these applications. It is interesting to note that when looking at successful applications of object-oriented programming, such as those reported in  [Harmon93], these almost invariably include an embedded hypermedia component, as well as functionality that is derived from a rule-based reasoning facility.

## Structure versus presentation

Hypermedia technology supports the organization of a variety of material into an associative structure. In addition, hierarchical structuring facilities may be supported. The basic notions underlying the structuring facilities of hypertext have been expressed in the Dexter hypertext model. See  [Halasz94].

Dexter

#### Component

• content -- text, graphics, video, program
• attributes -- semantic description
• anchors -- links to other documents
• presentation -- display characteristics

#### Compound

• children -- subcomponents
##### document.write(' <a href=');if (!slidemode) { document.write('@slide-'); } else { document.write('quote-hypermedia.html#slide-'); } document.write('12-Dexter'); if (!slidemode) document.write('.html');document.write('>');

slide: The Dexter hypertext reference model

The Dexter model explains the structure of hypertext documents in terms of components, links and anchors. The notion of anchors is introduced to explain how to attach a link to either a source or destination component. An anchor is the indication of a particular spot in the document, or rather a component thereof, usually identified by some coordinate. The Dexter model distinguishes between three layers in a hypertext system, namely the document layer (defining the content and structure of documents), a storage layer (handling the storage and retrieval of components and links) and a presentation layer (handling the display of documents and the interaction with the user). A component, which is a part of a document is characterized by the following features: content (which may be text, graphics, audio, video or even a program), attributes (which give a semantic description that may be used for retrieval or selective display), anchors (which identify the places to which a link is attached), and presentation characteristics (which determine the display of the component). In addition, for compound components, a feature children may be defined, for storing the list of subcomponents.

#### Multimedia

The original Dexter hypertext model is strongly oriented towards text, despite the provision for multimedia content. Multimedia, in particular audio and video, are intrinsically time-based and require temporal primitives to synchronize the presentation of material from different sources. In the CMIF multimedia model described in  [Hardman94], channels (which are abstract output devices) are introduced to allow for the specification of abstract timing constraints by means of synchronization arcs between channels.

#### Multimedia model -- composition

CMIF

• data block -- atomic component
• channel -- abstract output device
• synchronization arc -- specifying timing constraints
• event -- actual presentation

#### Views -- authoring

• structure -- sequential and parallel composition
• channels -- presentation
##### document.write(' <a href=');if (!slidemode) { document.write('@slide-'); } else { document.write('quote-hypermedia.html#slide-'); } document.write('12-multimedia'); if (!slidemode) document.write('.html');document.write('>');

slide: The CMIF multimedia model

The notion of channels provides the abstraction needed to separate the contents of a presentation from the actual display characteristics. For example, text may be output through a text channel while, simultaneously, video may be output through a video channel. The screen layout and allocation of these channels may be determined independently. The actual presentation is determined by events, that may either arise as the result of a user action or as the result of the activation of a synchronization arc. For example, a synchronization constraint may specify that an audio fragment containing speech must be started 10 seconds after the beginning of a video sequence. Then, after 10 seconds, the video channel will issue an event that causes the audio channel to start presenting its contents. The CMIF model has been developed to allow for portable multimedia documents. In particular, the notion of channels allows for a platform-independent characterization of presentation characteristics and timing constraints. An important characteristic of the model, from an authoring perspective, is that it supports a compositional approach to authoring, since it allows us to compose a channel (specifying a sequential composition of components) with arbitrary many other channels, in parallel. In  [Hardman94], an extension of the CMIF multimedia model is developed to incorporate the associative structuring facilities defined by the Dexter hypertext model.

#### Hypermedia model -- components

• contents -- data block
• attributes -- semantic information
• anchors -- $(id, value) • presentation -- channel, duration, ... #### Compound • children --$( component, start-time $\right)$
• synchronization -- \$($source, destination$)
##### document.write(' <a href=');if (!slidemode) { document.write('@slide-'); } else { document.write('quote-hypermedia.html#slide-'); } document.write('12-model'); if (!slidemode) document.write('.html');document.write('>');

slide: A hypermedia reference model

In the combined model, a single component consists of contents containing the actual data blocks of the component, attributes that specify semantic information, a list of anchors (each specifying a symbolic name and a value, which in the case of an audio or video fragment is its time measured from the start), and presentation characteristics, which include the specification of a channel and the duration of the component. As in the Dexter model, compound components may have children attributes, specifying for each child a component and its start-time, and a number of synchronization arcs, each specifying a source (component and anchor) and destination (component and anchor). Synchronization arcs may cross channel boundaries. The reader is encouraged to specify a more detailed object model, based on the outline given above. Evidently, the incorporation of a variety of content types and display channels is a serious challenge. In particular, the notion of time-based active objects will probably be difficult to handle. For an abstract characterization of active time-based (media) object, the reader is referred to  [Nierstrasz].

## On the notion of links -- active documents

Hypermedia documents are often referred to as hyperdocuments, because of their associative structure imposed by (hyper) links. Links, in general, may be characterized as a possibly conditional connection between a source anchor and destination anchor. There has been an ongoing discussion as to whether links must lead from byte to byte or whether they must be defined at some higher level. On closer inspection, there appear to be a number of choices with respect to the kind of links that may be supported. See, for example, Halasz (1988, 1991).

• << source, conditions, destination >>

#### World Wide Web -- distributed hypermedia

• information retrieval -- HTML

#### Active documents

• text as scripts