SIKS basic course

Learning and Reasoning

Locatie: Landgoed Huize Bergen te Vught, 

Maandag 25 mei 2009


theme: learning
9.30-10.30 (presentation)






Koen Hindriks & Dmytro Tykhonov (TU Delft)

Bilateral Negotiation. An overview of the theory of negotiation & negotiating software agents, with a focus on learning opponent models

Negotiation is a complex decision-making process aiming to reach an agreement to exchange goods or services. Current automated negotiation systems may significantly improve such agreements if the negotiation space is well-understood. In this presentation a brief overview of the state of the art and various research goals and challenges are discussed. In particular, we will present a generic framework based on Bayesian learning to learn an opponent model, i.e. the issue preferences as well as the issue priorities of an opponent. The algorithm proposed is able to effectively learn opponent preferences from bid exchanges by making some assumptions about the preference structure and rationality of the bidding process.

Material: slides.pdf




Martijn van Otterlo (Katholieke Universiteit Leuven)

Reinforcement learning


Reinforcement Learning (RL) is -- de facto -- the standard paradigm for learning in sequential decision making tasks. It is the main method of machine learning for tasks where i) an agent must learn to take a sequence of decisions, ii) the amount of feedback is very limited, iii) there is significant uncertainty in the environment, and iv) an exact model of the environment may not be available. RL stands in between supervised and unsupervised learning, and learning is guided by numerical rewards. Typical domains are games (e.g. backgammon, checkers, Tetris), robotics, multi-agent systems and various control tasks.

In this lecture I will introduce the core components and algorithms of RL and its model-based counterpart, dynamic programming (DP). I will explain concepts such as Markov decision processes, reward functions, policies and value (i.e. utility) functions and methods to compute (optimal) solutions. Furthermore, we will briefly touch upon more

advanced topics such as generalization. At the end of the lecture, I will give pointers to some of the current subfields of RL, such as

partially observable problems, hierarchical task decompositions and relational (i.e. object-based) environments.

Some part of the lecture consists of interacting with an educational (Java-based) demo of reinforcement learning for the card game Blackjack. The application allows for training various opponents and let's one play against them.

Material: readinglist , introduction in Reinforcement Learning , slides.pdf

15.15 - 17.00

Maarten van Someren (Universiteit van Amsterdam) 

Learning from sensory, language and structured data


In this talk I will discuss the problem of learning from different types of data, and outline basic methods and approaches to this problem. Building blocks are Machine Learning for Information Extraction, for Translation and for abstraction from sensory data.  Methods will be illustrated with examples of systems and we will make a brief excursion to philosophical issues.

Material: slides.pdf

Dinsdag 26 mei 2009


9.30 -12.00

Bert Bredeweg (Universiteit van Amsterdam)

‘Pragmatics’ in Qualitative Reasoning



Humans continuously reason about the physical-world that surrounds them.This kind of reasoning is sometimes referred to as common-sense reasoning, partly because it is often intuitive and because humans do not use any numerical information to do so. The key research question is to understand this kind of reasoning and to create means for automating it using computers.

Qualitative Reasoning is at the heart of this research. It is an

innovative approach within Artificial Intelligence that involves

non-numerical descriptions of systems and their behaviour, preserving all the important behavioural properties and distinctions. It has been applied successfully to problems in the automotive industry, to aeronautics and spacecraft, thermodynamics, and ecology. In addition to real-world

applications the research focusses on cognition and education; explaining human reasoning and developing means to support and enable this ability.

12.00 - 13.00



Henry Prakken (Universiteit Utrecht)



In recent years, argumentation has become an increasingly popular topic in the symbolic study of commonsense reasoning and inter-agent communication. In logical models of commonsense reasoning, the argumentation metaphor has proved to overcome some drawbacks of other formalisms. Many of these have a mathematical nature that is remote from how people actually perceive their everyday commonsense reasoning, which makes it difficult to understand and trust the behavior of an intelligent system. The argumentation approach bridges this gap by providing logical formalisms that are rigid enough to be formally studied and implemented, while at the same time being close enough to informal reasoning to be understood by designers and users. In the current course the fundamental concepts and structure of argumentation logics will be discussed.  

Furthermore, it will be discussed how formal multi-agent argumentation protocols can be specified in which agents discuss the validity of a claim. This enables designers of, for instance, multi-agent systems or discussion support systems to specify communication protocols that adhere to certain desirable logical properties.



Peter Lucas (Radboud University Nijmegen)

Probabilistic Reasoning


Modern work on uncertainty reasoning in artificial intelligence is

mostly based on probability theory, and, in particular, probabilistic

graphical models have had a major influence on this strand of

research. In the lecture some of the basic ideas of early uncertainty reasoning are briefly reviewed, as are the basics of conditional independence representation by graphical models, and the relationship between the early and modern work. Surprisingly, the relationship between the early work and later work is much stronger than most researchers think, which can be interpreted as further evidence of the strongly intuitive foundation of probability theory as an uncertainty calculus.  In addition, the subtleties of the various ways of representing conditional independence information will receive attention. Some recent research in merging logical and probabilistic representation and reasoning is also covered.

Material: slides.pdf