topical media & game development

talk show tell print


Rate, Recommend, Regret -- an Expert-based Approach to the Personalization of Guided Tours

Anton Eliëns, Yiwen Wang



In this paper we propose an approach to generate personalized guided tours based on a finite collection of tours obtained by tracking the navigation of expert users. Our proposal is based on a variant of decision theory, that uses a regret function to measure the difference between a proposed decision and a finite collection of expert decisions, generalized to a finite sequence of discrete choices. Personalization may then be seen as a minimization problem over a weighting scheme, expressing the relative importance of experts of which tours are available. We illustrate our approach by showing how we may obtain guided tours in 3D digital dossiers containing information on contemporary art installations, and discuss how our approach may be applied in other cultural heritage applications.
keywords: decision theory, personalization, guided tours, digital dossier, cultural heritage


Leaving all responsibility for interaction to the user is usually not a good choice, in particular when an information system contains complex, highly interrelated information. In  [Eliens et al. (2006b)],  [Wang et al. (2006)],  [van Riel et al. (2006)] we describe the 3D digital dossier format, in which we presented the information of respectively the Dutch-Serbian artist Marina Abramovic and the Australian artist Jeffrey Shaw, contemporary artists with a variety of work, ranging from video to art installations. The digital dossier supports navigation using a concept graph and allows for presenting media-rich material, including 3D models of artwork installations. The digital dossiers have been implemented using X3D/VRML to allow for deployment on the web.

Recently we have explored guided tours in digital dossiers,  [van Riel et al. (2006)], which actually automate user interaction, by mimicking user actions through events generated by a script. Although this provides an easy way to create guided tours, this does not solve the problem of what to select as elements in the guided tour, or how to personalize these tours in an intelligent manner.

In this paper, we discuss techniques from decision theory as a means to aid the construction of guided tours by consulting an advice function based on tracking the navigation behavior of expert users. We will also indicate how a similar advice function can be used for personalizing tours in cooperation with a recommender system for artworks, by altering the weight given to particular properties.


The structure of this paper is as follows. In section 2 we will briefly describe the abramovic dossier. In section 3 we will give a brief introduction to decision theory, and in sections 4 and 5 we will discuss how techniques from decision theory can be applied to the construction of guided tours in digital dossiers by using expert advice. In section 6, we will illustrate how to apply decision theory for the personalization of tours in a more conventional cultural heritage application and in section 7 we will discuss how to realize expert advice functions in digital dossiers. Finally, in section 8 we will give our conclusions and indicate directions for future research.

The abramovic dossier

As a user interface for navigating the abramovic dossier, we created a concept graph, fig. 1(a), that represents arbitrary information structures in a hierarchical way. The concept graph allows the user to detect relations and search for information. Unlike the 3D cone tree,  [Robertson and MacKinlay (1991)], where the complete hierarchical structure is presented, only a subset of the hierarchy is shown - three levels deep.


Fig. 1(a) Concept Graph(b) Presentation gadget

Presentation is an essential part of the digital dossier but is separated from navigation. The digital dossier contains different presentation facilities for 2D and 3D content. For 2D media content we need to be able to present video, images or textual information. This is implemented as a presentation gadget with three windows, fig 1(b). In each of the three windows the user can view either text, image or video content. The windows are positioned in such a way that the user can inspect the information simultaneously. In our experience, three views can be presented at the same time without much visual distortion.

usage scenario:

When starting the dossier, it loads the concept graph that is used to navigate through the available information. In the center of the concept graph, a shining star is shown to illustrate the root of the information hierarchy, which is used as the start object. When clicked, a star structure spreads and child objects appear surrounding the center star object. For the purpose of clarity the screenshots have been adapted. Not only is the originally black background color changed into white, also the textual information of the nodes has been modified.

Clicking on the Interviews node gives an overview of all interview fragments, then going back clicking on the information node Artworks and then on China Ring will bring the node for China Ring into focus. When clicking on the center node China Ring, a content presentation environment appears. which has three windows to present different types of information, grouped into the categories text, pictures and video. If desired, the user can focus on any window by using a zoom function. When the presentation of media content is finished, clicking on the close button will result in going back to the concept graph. Alternatively, the home function of the tool bar may be used to return directly to where we started: the original shining star.

Mathematical preliminaries -- decision theory

Before discussing how to realize guided tours in digital dossiers using user tracking and expert advice, we will give a very brief introduction to decision theory, more in particular a variant of decsion theory introduced in  [Cesa-Bianchi and Lugosi (2006)], that provides a mathematical foundation for our approach.

In classical prediction theory a prediction is a sequence of elements x_1, x_2, ... that results from a stationary stochastic process. The risk of the prediction is taken to be the expected value of the accumulated loss function, measuring the discrepancy between predicted values and actual outcomes.  [Cesa-Bianchi and Lugosi (2006)] introduce a variant of prediction theory in which no assumption is made with respect to the nature of the source of predictions. Instead, the forecaster is considered to be an entity that gives a prediction for an element based on advice of one or more experts. These experts might be actual sequences stored in a database. The deviation of the forecaster with the actual outcome is measured using a regret function, and the prediction task may hence be formulated as minimimizing the regret function by choosing the best expert for advice for each element of a prediction sequence.

For example, for the prediction of a bitstring of length n, the forecaster is a vector of n expert indices, that give advice for the bitvalue, 0 or 1, in that position. In the general case, in which we have no information on the error rate of the experts' advice, we may use a weighting factor $0 <= %b_{i} <= 1 for each expert i, to indicate the credibility of the experts' advice. After each prediction, obtained by taking the majority decision of the experts, according to the weighting scheme, we may verify which experts fail to give the right advice, and decrease their weight, thus eliminating the influence of their advice in the long run.

Guided tours in digital dossiers

In digital dossiers, we explored the use of guided tours as a means to present the information in a story-like way, relieving the user of the often cumbersome task to interact,  [van Riel et al. (2006b)]. Guided tours, in the digital dossier, may take one of the following forms:
  • automated (viewpoint) navigation in virtual space,
  • an animation explaining, for example, the construction of an artwork, or
  • the (narrative) presentation of a sequence of concept nodes.
In practice, a guided tour may be constructed as a combination of these elements, interweaving, for example, the explanation of concepts, or biographic material of the artist, with the demonstration of the positioning of an artwork in an exhibition space.

A pre-condition for the construction of guided tours based on user tracking is that navigation consists of a small number of discrete steps. This excludes the construction of arbitrary guided tours in virtual space, since it is not immediately obvious how navigation in virtual space may be properly discretized. In this case, as we will discuss in section 7, a guided tour may be constructed using a programmed agent showing the user around.

For navigation in the concept graph, as well as for the activation of the media presentation gadget, the discretization pre-condition holds, and a guided tour may be composed from a finite number of discrete steps, reflecting the choice of the user for a particular node or interaction with the presentation gadget.

For example, in the abramovic dossier, the user has the option to gode from the Main node to either Artworks, Video Installations or Interviews, and from there on further to any of the items under the chosen category. Tracking the actual sequences of choices of a user would suffice to create a guided tour, simply by re-playing all steps.

To obtain more interesting tours, we may track the navigation behavior of several experts for a particular task, for example retrieving information about the installation Terra della dea Madre. In case the experts disagree on a particular step in the tour, we may take the majority decision, and possibly correct this by adjusting the weight for one or more experts. When we have a database of tours from a number of experts, we may offer the user a choice of tours, and even allow to give priority to one or more of his/her favorite experts, again simply by adjusting the weighting scheme.

As a technical requirement, it must be possible to normalize interaction sequences, to eliminate the influence of short-cuts, and to allow for comparison between a collection of recordings. For the actual playback, as a guided tour, a decision mechanism is needed that finds the advice at each decision point, from each expert, to select the best step, according to a decision rule that takes the weighting scheme into account.

Personalization by expert rating

When offering guided tours for which several variants exists, we may allow the user to simply assign weights to each of the experts from which we have a tour, or allow for incrementally adjusting the weight of the experts, as feedback on the actual tour presented.

Incremental adaptation of recommendations

In the CHIP project (Cultural Heritage Information Personalization), the aim is to develop a recommender system that generates a collection of artworks in accordance with the users' preferences based on the rating of a small sample of artworks. The properties on which the recommendation is based include Period, Artist, and Genre. The recommender system will also be used to generate guided tours, where apart from the already mentioned properties the Location (the proximity in the actual museum) will be taken into account. Using a weighting scheme on the properties, that is a difference metric on the properties, a graph can be created, giving a prioritized accessibility relation between each artwork and a collection of related artworks. By changing the weight for one of the properties, for example Location, in case the tour is generated for the actual museum, the priority ordering may be changed, resulting in a different tour.

In a similar way as for the digital dossier, user tracking may be deployed to incrementally change the weight of the arcs of the graph, reflecting the actual preference of the user when deviating from an existing guided tour.

Intelligent guidance -- realization

Our aim is to arrive at a general framework for artist's digital dossiers, that provide intelligent guidance to both the expert user, responsible for the future re-installation of the work(s), and the interested layman, that wishes to get acquainted with a particular work or collection of works. In general, there are two techniques that we can apply to provide such guidance:
  • filtering the information space according to the user's perspective, and
  • intelligent agents, that (pro) actively aid the user in searching the information space.
Filtering the information space may be used to restrict the concept graph that defines the navigation structure, by stating assumptions with respect to the relevance of particular categories from a user's perspective. Intelligent agents is an approach stemming from artificial intelligence which allows for providing guidance in a variety of ways, possibly even in an embodied form using a face or humanoid figure to give suggestions to the user on what interactions to perform, an approach that we will discuss later on. For selecting the items to be presented in a guided tour, the most obvious way is to pre-define a sequence based on user profiles. Very likely this can be done in a more flexible way in a rule-based manner, applied to a template tour. More interesting, however, is to generate guided tours dynamically based on tracking actual user interaction of (expert) users, using techniques from prediction theory, as explained in sections 4 and 5.


Fig. 2: (a) Light(b) Material

A special case of a guided tour is the tool environment constructed for the Revolution installation of Jeffrey Shaw, which allows for experimenting with the (de-) construction of the installation, fig. 2(a), and exhibition parameters, fig. 2(b). Tracking interaction with such 3D models is, given the limitations imposed by the tool environment, relatively simple, and can be used for creating a repository of navigation sequences. More difficult, however, is to find proper normalizations for these interactions, and so in this case we may possibly have to rely on expert weighting only.


Fig. 3 (a) Overview(b) Interview(c) Agent

agent technology

In [_Agents] we have investigated the use of embodied agents in a digital dossier for the artist Marinus Boezem, fig. 3. To allow for a discrete mode of navigation we have used a map, displaying the interesting parts of the atelier, which contains locations where relevant information can be obtained, such as a filmprojector, for displaying interviews, a cabinet that contains biographical material and textual descriptions of the artworks, and an exhibition environment that displays (3D models of the) artworks. To construct a guided tour, we deployed a humanoid agent that shows the user around.

In a user evaluation test we found that humanoid agents where instrumental in providing information about the re-installation of artworks, but interestingly also that believability was positively affected by the degree of realism of the agent,  [Van Vugt et al. (2006)]. However, in creating guided tours for the current generation of digital dossiers, using concept graphs for navigation instead of a spatial metaphor, we will not use humanoid agents. Our agent technology, however, can be used in a fruitful way. In the I-GUARD project (Intelligent Guidance in Archives and Dossiers). we investigate how to realize advice functions, implemented using agent technology,  [Eliens et al. (2002)], based on actual navigation paths obtained by tracking expert users, that offer the user at any navigation point a choice of continuations and/or a selection of guided tours, focussing on a topic of interest.


In this paper we have sketched an approach for the construction or adaptation of guided tours, using techniques from decision theory, in particular the application of a weighting scheme determining the outcome of an advice function, based on stored preferences of (expert) users. This technique may also be applied in an incremental fashion to adapt an existing tour to personal preferences, reflecting the actual navigation behavior of users. The application of these techniques requires that choices are discrete and hence do not apply to arbitrary navigation in virtual environments, unless we find proper ways to encode such navigation as a small finite collection of discrete steps. Also in the discrete case, however, we must be able to normalize navigation paths, in order to compare and weigh the contribution of a collection of experts.


[CHIP] Aroyo, L., Rutledge, L., Brussee, R., de Bra, P., Gorgels, P., Stash,,
N., Veenstra, M. (2005). Personalized Presentation and Navigation of Cultural Heritage Content, Multimedia and Expo, ICME 2005. IEEE International Conference, 2005
[Prediction] Cesa-Bianchi N. and Lugosi G. (2006),
Prediction, Learning, and Games, Cambridge University Press
[Platform] Eliens A., Huang Z., and Visser C. (2002),
A platform for Embodied Conversational Agents based on Distributed Logic Programming, AAMAS Workshop -- Embodied conversational agents - let\'s specify and evaluate them!, Bologna 17/7/2002
[Navigate] Eliens A., van Riel C., Wang Y. (2006),
Navigating media-rich information spaces using concept graphs -- the abramovic dossier, Proc. InSciT2006, 25-28 Oct. 2006, Merida, Spain
[ Hoorn J., Eliens A., Huang Z., van Vugt H.C.,
Konijn E.A., Visser C.T. (2004). Agents with character: Evaluation of empathic agents in digital dossiers, Emphatic Agents, AAMAS 2004 New York 19 July - 23 July, 2004
[Guide] Riel C. van, Eliens A., Wang Y (2006),
Exploration and guidance in media-rich information spaces: the implementation and realization of guided tours in digital dossiers, Proc. InSciT2006, 25-28 Oct. 2006, Merida, Spain
[Maps] Riel C. van, Wang Y. and Eliens A. (2006b),
Concept map as visual interface in 3D Digital Dossiers: implementation and realization of the Music Dossier, CMC2006, Costa Rica, Sept 5-8 2006
[Cones] Robertson G.G. and MacKinlay J.D. (1991) ,
Cone trees: animated 3D visualizations of hierarchical information, Proc. of the SIGCHI Conference on Human factors in computing systems: Reaching through technology, 189 194, New Orleans, Louisiana, United States, March 1991.
[Present] Wang Y., Eliens A., van Riel C. (2006),
Content-oriented presentation and personalized interface of cultural heritage in digital dossiers, Proc. InSciT2006, 25-28 Oct. 2006, Merida, Spain
[Experiment] Van Vugt, H. C., Konijn, E. A., Hoorn, J. F., Keur, I.,
Eliens, A. (2006). Realism is not all! User Engagement with Task-Related Interface Characters, Interacting with Computers, 2006

(C) Æliens 27/08/2009

You may not copy or print any of this material without explicit permission of the author or the publisher. In case of other copyright issues, contact the author.