Final Results

You can find your benchmark score in the “score.txt” file within your tool’s directory.

The final ranking is a follows:

  1. evosuite with 156.9459 points on average
  2. randoop with 101.8129 points on average
  3. t2 with 50.4938 points on average
  4. dsc with 43.3208 points on average
We do not have a ranking for the manual tests, since we do not know how much time the developers spent to develop the manual test cases. The benchmark score can thus only be given as a function of t_gen:  score_manual = 246.5554 – 5 * t_gen

The zip file also contains the score tool which aggregates the results of a single run as well as multiple runs.

./score ../evosuite/run1/results.txt      // score for a single run
./score ../evosuite      // final score averaged over all runs

Preliminary Results

We have the first results of the benchmark. However, be aware that these can still change. This is just to get a first impression of your results.

The zip file contains the test classes as well as the results from each run of a participant’s tool. Each run comprises the generated test cases, a log file of the run and the results.txt file with the metrics for each class.

The folder “scoretool” contains the tool “score” which can be used to calculate the benchmark score for each run. Just pass on of the results.txt files:

./score ../evosuite/run1/results.txt