The International Competition on Knowledge Engineering for Planning and Scheduling (ICKEPS) is a bi-annual event, hosted at the International Conference on Automated Planning and Scheduling (ICAPS). The objectives of the competition are to promote tools for knowledge aquisition and domain modeling, to accelerate knowledge engineering research in AI, and to encourage the development of software platforms that promise more rapid, accessible, and effective ways to construct reliable and efficient systems.

Knowledge Engineering (KE) for AI Planning has been defined as "the process that deals with the acquisition, validation and maintenance of planning domain models, and the selection and optimization of appropriate planning machinery to work on them. Hence, KE processes support the planning process: they comprise all of the off-line, knowledge-based aspects of planning that are to do with the application being built, and any on-line processes that cause changes in the planner's domain model". We expect the competition to encourage the development of tools across the whole KE area including domain modeling, heuristic acquisition, planner-domain matching, and domain knowledge validation.

The workshop time table is available.

The 2nd ICKEPS is hosted at ICAPS-2007. It builds on the previous competition, held in 2005.

Competition Format

Based on the small number of competitors we decided ICKEPS to take place as a showcase event, illustrating the state-of-the-art in knowledge engineering tools, different sorts of simulators and a discussion on how far current technology is from planning practice.

Area of Scope

The competition is open to authors of a tool, an integrated tools environment or tools platform (below we simply refer to the entry as "the tool") where the tool helps knowledge engineering for AI P&S purposes in at least one of the following categories: Stand-alone planners/schedulers are not eligible, but can be a part of a tool. The competition entries should be distributable without any associated fees; the competition is otherwise open to all participants. The authors should follow the competition structure. In particular, at least one of the authors must present the system during the conference.

Competition Structure

Phase 1: Pre-conference

Authors must submit a short paper describing the tool (no more than 5 pages including screenshots) to the workshop organizers by the submission deadline. Papers must be in US letter size, and use the AAAI Style template.

Authors are encouraged to make the tool available for download from the web prior to the competition in order for judges to evaluate tools independently prior to the conference. All tools will ultimately be posted on the ICKEP web page.

The short papers will be 'lightly' reviewed in order to

If the tool is accepted for the competition then the authors must submit a camera-ready copy of the paper to the workshop organizers by the camera-ready copy deadline and at least one author must register for the competition via the standard ICAPS registration.

Phase 2: Evaluation through Simulation

In addition to the pre-conference paper submission, the competition will make available planning and scheduling simulations that competitors will use to evaluate their tools. These simulations will be available via a web service. Competitors will read a short text description of the competition domain, including a description of the simulation API, retrieve problem instances, submit plans for each instance, and receive feedback describing the quality of the plan. Each participant.s interaction with simulators will be logged, and the logs will be used as part of the tool evaluation. The simulation server is up. Simulations are available to everybody. The simulation tester helps connecting to the server. Tools may evaluate their systems prior to the conference.

Phase 3: At the conference

At least one author per tool must come to the conference and demonstrate the system in person. This will entail giving a short talk during the competition workshop and being ready to demonstrate the system and answer questions during the demonstration session. The competitors must bring their tools installed on their own laptops. Competitors should tailor their presentations to discuss how their tools helped them solve the simulated domains, but should also discuss other facets of their tools not exercised by the simulations.

The competition will take place during a half day workshop, where competitors will present their systems, and system demonstration of simulation providers. The workshop and system demonstration will be open to everyone attending the conference.

The judges will decide the results using the reviews of the short papers, quantitative results from the competitors, interaction with the simulations, and the system presentations and demonstrations. The proposed criteria to be used by the judges are presented below; the final rules will be posted on the competition web page. There will be a presentation of the results and winners' prizes either during the conference reception or during/before the conference dinner. Final scores for each entry will also be posted on the competition web page after the competition.

Phase 4: After the conference

The simulation server will be accessible as a long-term challenge to the planning comnunity.

Important Dates:

Due to USB storage device the publishing time table for compiling the competition proceedings has been postponed. This gives room to include late coming showcases.


The competition was organized by Stefan Edelkamp, University of Bremen, and Jeremy Frank, NASA Research Center.

Program Committee

Comments on any part of the contents of this page are welcome. Send us mail.