CODEWITHSUNDEEP
javabenchmarkingoptaplanner
Ryan Moser
Ryan Moser

Reputation: 131

The OptaPlanner Benchmarker Report does not display correctly for huge runs

Versions:

I have the Benchmarker running a few huge sets on a remote server and the results do not display correctly.

Expanding the benchmark config, it yields 418 configs, subSingleCount == 10, and I'm using a problem set of size 5.

Each report tries to display results for 418 configs, but the legend of each plot ends up dominating the graph picture. I would attach a screenshot, but I think that feature is blocked.

I thought maybe the graphs where rendered on the spot, but the graphs are just pictures.

I know that I can regenerate the reports, so are there any power-user/hard-coded settings I can tweak to prevent this/increase the margins/etc.?

I couldn't find anything in the doc aside from maybe messing around with the BenchmarkAggregatorFrame, but that throws a NullPointerException in

org.optaplanner.benchmark.impl.ranking.ScoreSubSingleBenchmarkRankingComparator.compare(ScoreSubSingleBenchmarkRankingComparator.java:28)

on line 28, so that's a no-go unless I can figure that out.

Complete stack trace as requested is below. It may be worth mentioning that this does open the application window, too, but it just displays the uncaught exception.

Exception in thread "main" java.lang.NullPointerException
    at org.optaplanner.benchmark.impl.ranking.ScoreSubSingleBenchmarkRankingComparator.compare(ScoreSubSingleBenchmarkRankingComparator.java:28)
    at org.optaplanner.benchmark.impl.ranking.ScoreSubSingleBenchmarkRankingComparator.compare(ScoreSubSingleBenchmarkRankingComparator.java:24)
    at java.base/java.util.Collections$ReverseComparator2.compare(Collections.java:5231)
    at java.base/java.util.TimSort.countRunAndMakeAscending(TimSort.java:355)
    at java.base/java.util.TimSort.sort(TimSort.java:220)
    at java.base/java.util.Arrays.sort(Arrays.java:1515)
    at java.base/java.util.ArrayList.sort(ArrayList.java:1749)
    at org.optaplanner.benchmark.impl.result.SingleBenchmarkResult.determineRanking(SingleBenchmarkResult.java:403)
    at org.optaplanner.benchmark.impl.result.SingleBenchmarkResult.determineTotalsAndAveragesAndRanking(SingleBenchmarkResult.java:397)
    at org.optaplanner.benchmark.impl.result.SingleBenchmarkResult.accumulateResults(SingleBenchmarkResult.java:344)
    at org.optaplanner.benchmark.impl.result.ProblemBenchmarkResult.accumulateResults(ProblemBenchmarkResult.java:346)
    at org.optaplanner.benchmark.impl.result.PlannerBenchmarkResult.accumulateResults(PlannerBenchmarkResult.java:331)
    at org.optaplanner.benchmark.impl.aggregator.swingui.BenchmarkAggregatorFrame.initPlannerBenchmarkResultList(BenchmarkAggregatorFrame.java:262)
    at org.optaplanner.benchmark.impl.aggregator.swingui.BenchmarkAggregatorFrame.init(BenchmarkAggregatorFrame.java:157)
    at org.optaplanner.benchmark.impl.aggregator.swingui.BenchmarkAggregatorFrame.createAndDisplay(BenchmarkAggregatorFrame.java:126)
    at org.optaplanner.benchmark.impl.aggregator.swingui.BenchmarkAggregatorFrame.createAndDisplayFromFreemarkerXmlResource(BenchmarkAggregatorFrame.java:103)
    at com.lmco.victor.main.SatellitePlanningEngineBenchmarkerMain.main(SatellitePlanningEngineBenchmarkerMain.java:52)

Upvotes: 1

Views: 106

Answers (1)

Geoffrey De Smet
Geoffrey De Smet

Reputation: 27312

If I recall correctly, this is a known issue with large datasets: the graphs become unreadable.

Workaround

Run the Benchmark Aggregator (no need to rerun the benchmarks) to select a subset of all benchmarks so the graphs become readable.

I see you tried that, and got that NPE. That's weird, can you paste the entire stacktrace? This might be a regression. Normally, if older datasets (run with older optaplanner versions) are in the same directory, and loading them crashes, that doesn't crash the aggregator itself (it just reports those benchmarks as failed to load).

Upvotes: 1

Related Questions