Tool Competition

This year as well we are pleased to announce the twelfth edition of the testing tool competition. The competition has the goal to experiment with testing tools for a diversified set of traditional and emerging systems and domains.

Java Tool Competition

As for recent years, we invite researchers to participate in the competition with their unit test generation tool for Java. Tools will be evaluated against a benchmark with respect to code coverage and mutation score.

Cyber-physical systems (CPS) testing competition

In addition to the traditional Java tool competition, we also organize a CPS testing competition on self-driving cars simulation environments. Specifically, in collaboration with the BeamNG research team, this competition focuses on the generation of scenarios using BeamNG self-driving cars simulator. More information on how to participate, follow this link.

Fuzzing Competition

Continuing from previous year, we will hold a Fuzzing Tool competition. We will be using mutation analysis integrated into fuzzbench as a benchmarking tool for the fuzzers. The details on how to submit your fuzzer can be found here.

Python Competition

We welcome researchers to engage in our competition by submitting their Python unit test generation tools. These tools will undergo assessment based on a benchmark that assesses code coverage and mutation score performance.

CPS-UAV Tool Competition

We invite researchers to participate in the competition with their test generation tool for Unmanned Aerial Vehicles (UAVs). The UAV Testing Competition organized by the Search-Based and Fuzz Testing (SBFT) workshop is an initiative designed to inspire and encourage the Software Testing Community to direct their attention toward UAVs as a rapidly emerging and crucial domain. Tools will be evaluated against a benchmark with respect to relevant criteria, as done for other, previous competitions.

How to participate in the competitions?

To participate:

• Java: See instructions provided here.
• CPS: See instructions provided here.
• Fuzzing: Register your intention of participating by following the instructions provided here.
• Python: See instructions provided here.
• CPS-UAV: See instructions provided here.

Check out the contest infrastructure and make sure your tool works well within it.

Java Infrastructure:

Details on how to integrate your test generator in the benchmarking infrastructure can be found here.

CPS Infrastructure:

The infrastructure concerning the CPS tool competition can be found here: https://github.com/sbft-cps-tool-competition/cps-tool-competition.

Fuzzing Infrastructure:

Details on how to integrate your fuzzer in the benchmarking infrastructure can be found here.

Python Infrastructure:

Details on how to integrate your test generator in the benchmarking infrastructure can be found here. Notify the organizers by sending an email to the Python chairs:

• Nicolas Erni (Zurich University of Applied Sciences) - gann@zhaw.ch
• Christian Birchler (Zurich University of Applied Sciences) - birc@zhaw.ch
• Pouria Derakhshanfar (JetBrains) - pouria.derakhshanfar@jetbrains.com
• Stephan Lukasczyk (University of Passau) - stephan.lukasczyk@uni-passau.de
• Sebastiano Panichella (Zurich University of Applied Sciences) - panc@zhaw.ch

CPS-UAV Infrastructure:

Details on how to integrate your test generator in the benchmarking infrastructure can be found here. Notify the organizers by sending an email to the CPS-UAV chairs:

• Sajad Khatiri (Zurich University of Applied Sciences) - mazr@zhaw.ch
• Prasun Saurabh (Zurich University of Applied Sciences) - sarr@zhaw.ch
• Timothy Zimmermann (Verity) - timothy.zimmermann@outlook.com
• Charith Munasinghe (Zurich University of Applied Sciences) - mung@zhaw.ch
• Christian Birchler (Zurich University of Applied Sciences) - birc@zhaw.ch
• Sebastiano Panichella (Zurich University of Applied Sciences) - panc@zhaw.ch