multimedia @ VU
On parallelism, synchronization and runtime performance
Complex humanoid gestures are of a highly parallel nature.
The STEP scripting language
supports a direct way of modelling parallel gestures
by offering a parallel construct
), which results in the simultaneous
execution of (possibly compound) actions.
To avoid unconstrained thread creation,
the STEP engine makes use of a thread pool,
containing a fixed number of threads,
from which threads are allocated to actions.
Once the action is finished, the thread is put back in
This approach works well for most examples.
However when many threads are needed,
as in the conductor example
(which requires approximately 60 threads),
problems may occur, in particular when there are may background jobs.
When using parallelism three types of potential
problems may be distingushed:
Synchronisation among gestures requires that gestures
that are meant to be executed simultaneously
do indeed start and end at the same time.
Unwanted delays however may occur, for example,
when there are nested parallel constructs, due to
the processing needed to unravel compound gestures.
One remedy here is to optimize the Prolog engine,
which is an ongoing effort.
Another, equally obvious, remedy is to eliminate
nested parallel constructs by some sort of flattening.
These solutions require no or only a minor change to the STEP engine
In effect, we have introduced a par_turn construct
that allows to declare the parallel rotation of an arbitrary
number of joints.
However, such a construct, when applied exclusively,
limits modelling to defining what can best be called successive
'frames' of a motion, which runs counter the approach advocated
by STEP, namely defining gestures in a natural way,
as concurrent motions of the human body.
To solve the problem of reliable timing in a more fundamental
way would require not
only a modification of the STEP engine,
but also a rather different implementation
of the DLP threads supporting the parallelism in STEP.
Currently, the implementation only allows for
best effort parallelism and does not provide
the means for deadline scheduling.
Synchronisation with external events is possible,
to the extent that such external events are supported
by the VRML/X3D environment.
That does at this stage, unfortunately, not include
the synchronisation with audio events,
unless an audio track is broken up in separate pieces.
One issue to work on, in the future,
is the synchronisation with voice,
using the events generated by a text-to-speech engine.
Summarizing, as concerns the use of parallelism on modeling
gestures we run into a number of problems which are either
caused by exhausting the resources of the machine,
inefficiencies in the execution of Prolog code,
or limitations of the thread model supported by DLP.
Partial remedies include, respectively,
reducing the number of threads needed,
eliminating nested parallel constructs,
optimizing the Prolog execution engine, and a push-down
of critical code of the STEP engine to the Java level.
These measures can be taken without affecting the
STEP engine or the underlying DLP thread model.
Nevertheless, there remains still some concern whether
the inherent inefficiencies of the Java platform and the VRML
Java External Authoring Interface can be overcome in a satisfying degree.
Therefore, we have recently started to develop a version of DLP
and STEP for the Microsoft .NET platform, which we believe
that in combination with Direct3D based implementations of
3D graphic models allow for better control of the issues
that we identified as problematic, in particular
thread control, synchronization and interaction with a dynamic environment.
As a preliminary result, we can mention that our Prolog implementation
in the .NET C# language already runs ten times faster that
our Java implementation on a standard set of Prolog programs.
- synchronisation among gestures,
- (reliable) timing of gestures, and
- synchronisation of gestures with external events, e;g.audio.