multimedia @ VU
CV media links resources _ # @ !

talk show tell print

game @ VU / panorama @ VU


The PANORAMA interface is like a virtual gallery, where on the wall and on the floor information about social awareness is presented.
Both the frames on the wall and the floor are moving, to indicate the passing of time. The speed as well as the direction of movement may be adjusted. For example, when many casual encounters occur, the speed may be increased to indicate hectic activity.

The ViP system is a highly complex presentation system, based on DirectX9. It was originally developed for an augmented reality theatre production, to provide an overlay of text, images, 3D models and visual effects over live video feed, capturing the acts played on stage. For the PANORAMA interface, we use the extended version of ViP, that allows for the projection of multiple live video feeds, digital video clips, and sequences of images on objects in an immersive 3D space.

In the virtual gallery screenshot, these streaming images are represented by the nine squares aligned to the bottom of the screen. In the context of PANORAMA, we may regard the streaming image feeds as channels, that can, dependent on the design considerations discussed below, be mapped on arbitrary models in 3D space, of which the frames on the wall are just a simple example.

The ViP system allows for a variety of visual effects, including texture mapping of image feeds on 3D objects, overlays of multiple image textures, as well as for example particle systems with streaming image feeds projected on sprites. The ViP system also supports the use of vertex and pixel shaders for the application of both geometrical and color morphing.

For the control of what visual content is displayed and its behavior in time, such as for example the life time and velocity of elements in a paricle system, the ViP system provides a scripting and programming interface, as well as direct interaction with the screen. The interpretation of user actions is highly programmable. For example, clicking on one of the nine squares aligned with the bottom of the screen may result in displaying the content for that channel fullscreen, or may result in the activation of a command that displays an object or particle system with the content of that channel projected on it. Commands like these may also be given using a standard network messaging protocol, such as TCP/IP, thus allowing for the communication facilities needed to reflect social awareness in the PANORAMA interface.

Although there are only nine channels displayed in the virtual gallery, the ViP system allows for an arbitrary number of channels, that is streaming image feeds. Obviously, to control the complexity of the visual display, we are faced with a design problem, which may be summarized as follows:

  1. how do we allocate the channels to, respectively, occurrences of casual encounters and self-reflection,
  2. what representation do we select for projecting a channel in the PANORAMA space, and
  3. what desgn criteria do we apply for combining the simultaneous presentation of multiple channels.
Moreover, as will be discussed below, the design problem also includes the selection of image material, the 3D models and effects that populate the PANORAMA space, and possibly also the choice of what metaphor or type of immersive space to use for the PANORAMA interface.

alternative technology

We have chosen for the ViP system primarily because of its visual salience, providing us with an immersive 3D space with multiple (live) video feeds and a large variety of visual effects. As a disadvantage of ViP, however, we may mention the development effort it requires, that is it requires programming in C++, as well as shader programming in HLSL, and the use of the notoriously complex DirectX API.

For exploring the functionality of the PANORAMA system, or parts thereof, we will also consider the use of alternative technologies such as X3D/VRML, since it allows for fast prototyping, or of-the-shelf game engines such as the Half-Life 2 Source engine, which provides convenient tools for the construction of immersive virtual spaces as well as many built-in visual effects.

Broadly speaking, the PANORAMA system receives explicit and implicit awareness information from staff-members and transforms this information as an art display located in the staff room. It provides a continuous flow of information
[_] CV media links resources _ # @ !

(C) Æliens 2014