topical media & game development
representation & interaction
In re-thinking the abramovic dossier
we first needed to re-establish what are our goals in developing this application
and what are our primary data sources.
The goal, first of all, is to support conservators in their task of preserving
contemporary art, and to assist them with the re-installation of such artworks.
Our primary data sources are meta-information
, coming from the INCCA initiative,
and video-recorded artist interviews
, which were initiated by ICN as a means
to record information about contemporary art that would otherwise be lost.
In addition we media-material, including images and video,
that may be regarded as recordings
of the works of art,
as well as the textual interpretations
and classifications that exist, or may be
constructed from this material.
At this point, I may remark that one of the pitfalls in creating a dossier
is to get trapped in the visually salient features of the dossier,
the presentation of the artworks themselves, and forget about the
primary focus of the dossier, to make all information accessible in
an adequate manner.
For many cultural heritage applications, which aim to present art to the layman,
presenting the artwork is the primary focus, and giving access to the
information context generally comes second.
For our next generation of digital dossiers, we decided to take the following steps:
next generation dossier(s)
- adaptation of representation to Dublin Core (+ annotation needed for presentation)
- XML-based content management, with php forms (extending what we have now)
- there should also be a possibility to present the information and material in a 'plain' web format
- as well as in (a new version of) 3D dossiers
- we should think about the proper presentation parameters.
Dublin Core is the standard used in the INCCA initiative,
to record meta-information about existing information sources.
See section 3.3 for a description of the Dublin Core
element set and the Resource Description Framework (RDF) on which it is based.
For the abramovic dossier, a collection of record-like
structures was developed, together with a simple content-management tool,
written in PHP.
This content-management system must be adapted to be compatible with the
Dublin Core-based resource descriptions.
Further, we decided that, along with the 3D presentation of the dossier,
it would be worthwhile to develop a conversion tool that produces
standard web-technology based presentations as well.
This approach allows us to assess the usability merits of the 3D dossiers
in a comparative way.
Finally, as I indicated before,
an important issue that must be resolved concerns the proper presentation
What do we present to the user?
And how do we allow the user to interact with the material presented?
content management and data representation
For developing the abramovic dossier,
we have a fixed number of record-like structures:
- Video -- to display video fragment, including interviews
- Picture -- to present pictures of the artwork
- Artwork -- contains all information connected to a work of art
- TextItem -- to present text, from the interview or any other source
- MaterialItem -- to present information about material(s) used
- GroupNode -- to combine nodes in the concpet graph
- Information -- acts as the outer container for all nodes
All these structures support a set of common attributes,
including shortName, longName, ID, connectedNodesIDs,
In addition the Video, Picture and Picture have fields
allowing to show a preview image.
And the Video, Picture and TextItem, also have a url
field giving access to the actual information item.
The Information and GroupNode structures are used
for creating the top-levels of the concept graph, whereas the other structures,
such as the Video and TextItem give access to for example
a fragment of an interview and its transcription.
Below an example is given of the data underlying the concept graph of
the abramovic dossier:
description [ "Central information node" ]
connectedNodesIDs [ "ARTWORKS", "KEYWORDS",
"INTERVIEWS", "REPORT" ]
description [ "Node that connects to all the artworks" ]
connectedNodesIDs [ "MAIN", "TRANSITORY",
"ULAY", "VIDEOINSTALLATION", "ABRAMOVIC" ]
The Information node collects all available nodes,
and takes care of connecting the individual nodes, based on the information specified for each node.
As an example of an Artwork node, that is an element of the list
of nodes in the Information node, look at:
shortName "Terra degla Dea Madre"
longName "Terra degla Dea Madre"
description ["15:40 min, colour, sound."]
connectedNodesIDs ["VIDEOINSTALLATION", "DTV24",
"TTV24", "PV24", "CV24", "VV24", "G0"]
This node is connected to many other nodes, giving access to the information items
that belong to it, such as the video clips of the interview, shown below.
shortName "Interview clip Terra degla Dea Madre"
longName "Interview clip showing Terra degla Dea Madre"
connectedNodesIDs ["CLIP", "AV24"]
In the url field of this declaration, the actual video file
is indicated, which should be displayed at a resolution of 320x360,
as specified in the width and height fields.
And finally, as an example of a TextItem, consider:
longName "Green Dragon Lying instructions for the public."
description ["Text explaining the way the public has to interact with the artwork."]
connectedNodesIDs ["AO05", "INTERACTION"]
For constructing the abramovic dossier, Tim Verweij developed
the content management tool, that allows the user to browse
and edit existing nodes, and to insert new nodes into the graph.
integration with the Dublin Core
The Dublin Core is a general resource description formalism,
that allows for specifying resources in a variety of domains.
See section 3.3.
For INCCA the Dublin Core was chosen, not because it is the most suitable
formalism, but because it may serve as the least common denominator,
and agreement on anything else simply seemed to be impossible.
As a reminder, the Dublin Core provides the following elements:
- title -- name given to the resource
- creator -- entity primarily responsible for making the content of the resource
- subject -- topic of the content of the resource
- description -- an account of the content of the resource
- publisher -- entity responsible for making the resource available
- contributor -- entity responsible for making contributions to the content of the resource
- date -- date of an event in the lifecycle of the resource
- type -- nature or genre of the content of the resource
- format -- physical or digital manifestation of the resource
- identifier -- unambiguous reference to the resource within a given context
- source -- reference to a resource from which the present resource is derived
- language -- language of the intellectual content of the resource
- relation -- reference to a related resource
- coverage -- extent or scope of the content of the resource
- rights -- information about rights held in and over the resource
Descriptions of items in the digital dossier should incorporate
these elements, together with the attributes needed for
the insertion of items in the concept graph and the presentation parameters,
that are necessary for displaying the (media) material.
Technically, the namespaces supported by RDF does allow for merging these
different types of annotations.
However, the challenge here is to derive the presentation attributes
automatically, and to come up with a reasonable default for inserting these
items in the concept graph.
|location of Tower of Babel project|
intelligent guidance -- I-GUARD
Although digital archives or digital libraries
are by no means a new phenomenon,
our concept of digital dossiers contains a number
of innovative elements.
A digital dossier provides a unified information and
In this sense it differs significantly from
a digital archive with a traditional web interface,
where navigation and presentation are distinct.
Digital dossiers allow to a much greater extent for an immersive
experience of the information related to works of art.
As such it is reminiscient to explorations in
our to our notion of virtual context,
presented in section 8.1.
Working out the issues indicated above, that is the integration with the
Dublin Core and providing suitable content management,
is a matter of diligent software engineering.
But what can we further do to support the construction of digital
dossiers and improve the usability of such dossiers?
And what are the scientific issues, worth to be investigated?
To indicate the research issues, let me first expand the cope of our project
and re-define the goal of our research:
Contemporary art is an intrinsic part of our cultural heritage.
Installations, performances, video and other forms of media art,
as for example web art, have the interest of a small
group of adherents, but are in comparison with more traditional
art forms, far more difficult to present to a general audience.
Another problem presents itself, due to the type of materials
used and the context-specific aspects of these art forms,
in the conservation of the works.
In our research we address the issue of providing access
to these contemporary art forms from a wide variety
of perspectives, ranging from the interested layman
to the expert that has to deal with archiving,
conserving and the possible re-installation of the art works.
The acronym I-GUARD stands for Intelligent Guidance in Artist's Digital Dossiers,
and refers to a project the aim of which is
to arrive at a general framework for artist's digital
dossiers, that provide intelligent guidance
to both the expert user, responsible for the
future re-installation of the work(s),
and the interested layman, that wishes to get acquainted
with a particular work or collection of works.
In general, there are two techniques that we can apply to provide such
- filtering the information space according to the user's
- intelligent agents, that (pro) actively aid the user in searching
the information space.
Filtering the information space may be done
by using techniques from formal linguistics
to restrict the concept graph that defines the navigation
structure, that is
by stating assumptions with respect to the relevance
of particular (linguistic) categories or elements from
a user's perspective.
Intelligent agents is an approach stemming from artificial intelligence
which allows for providing guidance in a variety of ways,
possibly even in an embodied form using
a face or humanoid figure to give suggestions
to the user on what interactions to perform.
With the latter type of guidance we have already experimented in the
Marinus Boezem dossier, as described in section 9.2.
So let's look at what natural language technology has to offer.
Having a concept graph as a generic navigation device, it
still remains a problem how to fill the concept graph
with meaningful content, and how to indicate
meaningful relations between the concepts and aspects
covered by the nodes of the concept graph.
In the abramovic dossier
this was done by hand, based on information derived
from a transcripion of an interview with the artist.
(provided to us by ICN).
Interviews with artists is one of the means ICN deploys
to gain knowledge needed for the conservation of
Such interviews provide a rich source of textual information,
that includes both general viewpoints on the artist's oeuvre
as well as specific constraints that adhere to
the (re) installation of the work(s) of art.
What we should strive for is to derive
both structure and content of the concept graph
for a particular dossier (semi) automatically.
Using a basic lexicon of terms and phrases
related to contemporary art we should be able to
generate a representation of the textual information
that may serve as a basis for constructing the concept graph.
This representation must contain an enumeration
of the concepts, the relation between occurrences of concepts,
as well as a reference to the work(s) of art to which the
Natural language processing technology may not only serve
for the static analysis of the material, when the digital dossier
is created, but also dynamically when the dossier is being used,
to aid the user in finding relevant information.
Research issues here are, on the one hand,
the interpretation of user input (that is, loosely
structured natural language),
and on the other hand, filtering the concept graph
representing the information space in such a way
that it adequately reflects the user's interest or perspective.
In summary, from a research perspective, digital dossier(s) concern the following
- representation of information of one or multiple works
- presentation of that information in a
rich media presentation environment,
- intelligent navigation and interaction, and
- support for interaction with loosely-structured
And to conclude, digital dossiers will on the one hand contribute to making
contemporary art forms accessible to a larger audience
and on the other hand are explicitly meant
to support the complex task of the conservation
and re-installation of works of art in an effective manner.
|outside view of Tower of Babel project|
example(s) -- Tower of Babel
In the Tower of Babel project, shown above,
was projected from within buildings, on the windows,
to the outside.
Local citizens in a neighbourhood in Amsterdam
where approached to submit material
that expressed their emotions of daily life,
with the question what moves you.
and photograms could be submitted either by email or SMS.
Also workshops were held, during which particpants could
This material was then edited and prepared for projection,
using 40 carroussel dia-projectors,
taking about 2000 images, and six beamers projecting
images and video.
Also sound material, that was collected in the same manner, was
being used during the projection.
Inside view of Tower of Babel project.
The centre of the location, a somewhat impoverished neighbourhood near the
centre of Amsterdam,
is a building dating from 1926,
antroposofic temple, that once served as a cinema
for avant-garde movies, and is now being used as a library.
The buildings surrounding it are,
if not split up into apartments, being used as a local youth centre,
a city archive and another library.
research directions -- media art
In a recent symposium on the preservation of contemporary media art,
a number of institutions presented their projects,
ranging from more technical topics,
such as the
conservation of videotapes
mass storage of digital material
to the conceptual issues in
capturing new media,
the variety of
and the need to record and maintain
meta data about the artworks and related information.
To get an idea what the phrase media art encompasses,
have a look at the circumscription given in
new media art is a generic term used to describe art related to, or created with,
technology invented or made widely available since the mid-20th Century,
including technology stemming from
mass media and digital modes of delivery the artworks
Below, the disciplines that belong to this form of art are listed,
together with their entries in the WikiPedia, in an abbreviated form:
(new) media art
- audio art -- no definition available
- computer art -- any art in which computers played a role in production or display of the artwork.
- digital art -- art created on a computer in digital (that is, binary) form.
- electronic art -- entry to game producer,
should be Leonardo.
- generative art -- art or design generated, composed, or constructed through computer software algorithms,
or similar mathematical or mechanical autonomous processes
- hacktivism -- the writing of code, or otherwise manipulating bits, to promote political ideology
- interactive art -- a piece of art that involves the spectator in some way.
- internet art -- art or, more precisely, cultural production which uses the Internet
as its primary medium and, more importantly, its subject.
- performance art -- art where the actions of an individual or a group at
a particular place and in a particular time, constitute the work.
- robotic art -- page does not exist
- software art -- is an intersection of two almost non-overlapping realms: software and art.
- video art -- is a subset of artistic works which relies on "moving pictures" and is
comprised of video and/or audio data.
- video game art -- involves the use of a computer game for the creation of a digital artwork.
By the nature of the WikiPedia, to which every user can contribute entries,
this list nor the defining entries are by any means authorative.
Nevertheless, it does provide an overview and may serve as a starting point for further research.
You may not copy or print any of this material without explicit permission of the author or the publisher.
In case of other copyright issues, contact the author.