General
ICKEPS 2007 runs various simulators
to be manipulated via models in integrated planning systems. These
simulators will be used to evaluate plans produced by the competitors'
tools, providing an (indirect) way of evaluating the KE technology used
in each tool.
The interaction with the simulators is as follows:
- Step 1: Download documents describing each domain and
simulator API and build domain models offline. We continously
provide increasing levels of difficulty for each domain,
which might result in different domain models; when applicable, this
will be described in the documents.
- Step 2: Download problem instances. Competitors can
then build solutions for each problem instance (i.e. plans) for each
problem instance using their tools.
- Step 3: Competitors will then upload the plans for evaluation by the
simulator, which will respond with a report detailing any problems
encountered with the plans, or quality scores for valid plans.
Competitors can then revise their domain models and try producing new
plans, repeating steps 2 and 3 as they see fit.
Interactions with the simulation environment will be logged to provide
quantitative data used to evaluate the performance on the
simulated domains.
Infrastructure
Simulator providers do not need to
implement their own port listeners and dispatchers. They
just have to be extended to read and write agreed state and plan
data. As a consequence, simulators
can be executed individually.
The setting that we are working with and the consequences on the
hardware/software infrastructure are exposed
here.
We use a small c++ server that per communicates via TCP/IP via
port listening and dispatching.
The simulation server (see manual
for a description of its functionality)
- opens a port for TCP/IP listening
- for each client connecting to the server, it spawns a child process
- can deal with several simulators specified in config file
- starts a selected simulator and communicates
via pipes
- is based on a simple ASCII protocol which covers
PDDL and XML
- terminates child/dipatched processes on timeout
We provide a Java client as a support for
the competitors to perform the connection to the server
and the communication.
A small-sized client called (written in Java) is provided in
source code.
- selects the simulator (shortcut: first letter in
Manufactoring, PowerSupply, CyberSecurity,
Validator, Telescope, GraphTransformation)
- opens a port for TCP/IP listening
- can be started from the command line (Executable,
e.g. "./start_simtester P filename host port" for PSR,
filename refers to the tagged client's text file
containing all input information needed to feed the
simulator,
host and port are made available to the competitors)
- can be integrated inline in a Java
program (adapt Source)
It is up to the competitors to use the provided client or not, so
that they can participate without using it. The only thing is to open
a port to connect to the server and exchange text/file data. For this we
we have given a simple ASCII-protocol to send data and command line
parameter information through the net to call a simulator and receive
its output.
As a consequence, one can actually operate the server via telnet.
For experimenting with the server you may also download
the VEGA tool. This extended client includes
functionality for displaying geometrical objects, such that simulators
can - in principle - provide their internal state information to the
competitors (this implies that the simulator would send commands
like line and polygon)