topical media & game development

# architectural issues

The notion of multimedia information system is sufficiently generic to allow for a variety of realizations. Let's have a look at the issues involved.

issues

• multimedia storage and retrieval -- homegrown, third-party and legacy sources
• information architecture -- common format, native format, hybrid
• media abstraction -- unified indexes, query relaxation
##### document.write(' <a href=');if (!slidemode) { document.write('@slide-'); } else { document.write('7-1.html#slide-'); } document.write('9-intro'); if (!slidemode) document.write('.html');document.write('>');

As concerns the database (that is the storage and rerieval facilities), we may have to deal with homegrown solution, commercial third party databases or (even) legacy sources. To make things worse, we will usually want to deploy a combination of these.

With respect to the information architecture, we may wish for a common format (which unifies the various media types), but in practice we will often have to work with the native formats or be satisfied with a hybrid information architecture that uses both media abstractions and native media types such as images and video.

The notion of media abstraction, introduced in  [MMDBMS], allows for uniform indexes over the multimedia information stored, and (as we will discuss in the next section) for query relaxation by employing hierarchical and equivalence relations.

Summarizing, for content organisation (which basically is the information architecture) we have the following options:

content organisation

• autonomy -- index per media type
• uniformity -- unified index
• hybrid -- media indexes + unified index
##### document.write(' <a href=');if (!slidemode) { document.write('@slide-'); } else { document.write('7-1.html#slide-'); } document.write('9-arch'); if (!slidemode) document.write('.html');document.write('>');
In  [MMDBMS], a clear preference is stated for a uniform approach, as expressed in the Principle of Uniformity:

Principle of Uniformity

... from a semantical point of view the content of a multimedia source is independent of the source itself, so we may use statements as meta data to provide a description of media objects.

• from a semantical point of view the content of a multimedia source is independent of the source itself.
• use statements as meta data
• $md\left(o\right)$ -- metadata associated with media object o

##### document.write(' <a href=');if (!slidemode) { document.write('@slide-'); } else { document.write('7-1.html#slide-'); } document.write('9-uniform'); if (!slidemode) document.write('.html');document.write('>');
Naturally, there are some tradeoffs. In summary,  [MMDBMS] claims that: metadata can be stored using standard relational and OO structures, and that manipulating metadata is easy, and moreover that feature extraction is straightforward.

• metadata can be stored using standard relational and OO structures
• manipulating metadata is easy
• feature extraction is (!) straightforward

is it?

##### document.write(' <a href=');if (!slidemode) { document.write('@slide-'); } else { document.write('7-1.html#slide-'); } document.write('mm-dbms-tradeoffs'); if (!slidemode) document.write('.html');document.write('>');
Now consider, is feature extraction really so straightforward as suggested here? I would believe not. Certainly, media types can be processed and analysis algorithms can be executed. But will this result in meaningful annotations? Given the current state of the art, hardly so!

## research directions -- the information retrieval cycle

When considering an information system, we may proceed from a simple generic software architecture, consisting of:

software architecture

• a database of media object, supporting
• operations on media objects, and offering
• logical views on media objects
##### document.write(' <a href=');if (!slidemode) { document.write('@slide-'); } else { document.write('7-1.html#slide-'); } document.write('r-6-1-arch'); if (!slidemode) document.write('.html');document.write('>');
However, such a database-centered notion of information system seems not to do justice to the actual support and information system must provide when considering the full information retrieval cycle:

information retrieval cycle

1. specification of the user's information need
2. translation into query operations
3. search and retrieval of media objects
4. ranking according to likelihood or relevance
5. presentation of results and user feedback
6. resulting in a possibly modified query
##### document.write(' <a href=');if (!slidemode) { document.write('@slide-'); } else { document.write('7-1.html#slide-'); } document.write('r-6-1-cycle'); if (!slidemode) document.write('.html');document.write('>');
When we look at older day information retrieval applications in libraries, we see more or less the automation of card catalogs, with search functionality for keywords and headings. Modern day versions of these systems, however, offer graphical userinterfaces, electronic forms and hypertext features.

When we look at the web and how it may support digital libraries, we see some dramatic changes with respect to the card catalogue type of applications. We can now have access to a variety of sources of information, at low cost, including geographically distributed resources, due to improved networking. And, everybody is free to make information available, and what is worse, everybody seems to be doing so. Hence, the web is a continuously growing repository of information of a (very) heterogeneous kind.

Considering the web as an information retrieval system we may observe, following  [IR], that:

• despite high interactivity, access is difficult;
• quick response is and will remain important!
##### document.write(' <a href=');if (!slidemode) { document.write('@slide-'); } else { document.write('7-1.html#slide-'); } document.write('r-6-1-webir'); if (!slidemode) document.write('.html');document.write('>');
So, we need better (user-centered) retrieval strategies to support the full information retrieval cycle. Let me (again) mention someof the relevant (research) topics: user interfaces, information visualisation, user-profiling and navigation.

(C) Æliens 04/09/2009

You may not copy or print any of this material without explicit permission of the author or the publisher. In case of other copyright issues, contact the author.