Skip to content

Add option to regenerate report based on test cases (not reproduce) #177

@Hi-Timofey

Description

@Hi-Timofey

Is your feature request related to a problem? Please describe.
Yes. During a long fuzzing run (~4 hours, 700+ endpoints in my OpenAPI spec), CATS crashed with the following exception:

java.lang.ClassCastException: java.lang.Integer cannot be cast to java.util.Map
        at com.endava.cats.openapi.OpenAPIModelGeneratorV2.transformInJson(OpenAPIModelGeneratorV2.java:183)
        at com.endava.cats.openapi.OpenAPIModelGeneratorV2.generate(OpenAPIModelGeneratorV2.java:155)
        at com.endava.cats.factory.FuzzingDataFactory.generateSample(FuzzingDataFactory.java:521)
        at com.endava.cats.factory.FuzzingDataFactory.getResponsePayloads(FuzzingDataFactory.java:618)
        at com.endava.cats.factory.FuzzingDataFactory.getFuzzDataForHttpMethod(FuzzingDataFactory.java:250)
        at com.endava.cats.factory.FuzzingDataFactory.getFuzzDataForPost(FuzzingDataFactory.java:194)
        at com.endava.cats.factory.FuzzingDataFactory.fromPathItem(FuzzingDataFactory.java:99)
        at com.endava.cats.factory.FuzzingDataFactory_ClientProxy.fromPathItem(Unknown Source)
        at com.endava.cats.command.CatsCommand.fuzzPath(CatsCommand.java:403)
        at com.endava.cats.command.CatsCommand.startFuzzing(CatsCommand.java:323)
        at com.endava.cats.command.CatsCommand.doLogic(CatsCommand.java:259)
        at com.endava.cats.command.CatsCommand.run(CatsCommand.java:208)
        at picocli.CommandLine.executeUserObject(CommandLine.java:2045)
        at picocli.CommandLine.access$1500(CommandLine.java:148)
        at picocli.CommandLine$RunLast.executeUserObjectOfLastSubcommandWithSameParent(CommandLine.java:2469)
        at picocli.CommandLine$RunLast.handle(CommandLine.java:2461)
        at picocli.CommandLine$RunLast.handle(CommandLine.java:2423)
        at picocli.CommandLine$AbstractParseResultHandler.execute(CommandLine.java:2277)
        at picocli.CommandLine$RunLast.execute(CommandLine.java:2425)
        at picocli.CommandLine.execute(CommandLine.java:2174)
        at com.endava.cats.CatsMain.run(CatsMain.java:48)
        at com.endava.cats.CatsMain_ClientProxy.run(Unknown Source)
        at io.quarkus.runtime.ApplicationLifecycleManager.run(ApplicationLifecycleManager.java:141)
        at io.quarkus.runtime.Quarkus.run(Quarkus.java:80)
        at io.quarkus.runtime.Quarkus.run(Quarkus.java:51)
        at io.quarkus.runner.GeneratedMain.main(Unknown Source)
        at java.base@21.0.7/java.lang.invoke.LambdaForm$DMH/sa346b79c.invokeStaticInit(LambdaForm$DMH)

The problem: when this crash happens, CATS does not save the final index.html report. Although all individual test case files (Test1.json, Test1.html, …) exist in the output directory, the summary index.html is missing. As a result, I lost the ability to view the full report after a very long fuzzing session. This essentially canceled 4+ hours of testing.
And there's nothin I can find in docs about such sutiation.

Describe the solution you'd like

  • Add a mechanism to always generate the final index.html report from whatever test cases were already created, even if the fuzzing process crashes at some point.
  • Alternatively, provide a standalone command or flag to regenerate the final report (index.html + supporting files) from an already populated output folder that contains individual test case results.

Describe alternatives you've considered

  • Manually trying to parse all generated JSON/HTML test cases, but this is not practical when there are hundreds/thousands of cases.
  • Rerunning the entire fuzzing job is not feasible when it takes hours. Yes, I know about splitting functionality and I was using it correctly.

Additional context
CATS version 13.3.2
openjdk 24.0.2 2025-07-15
OpenJDK Runtime Environment (build 24.0.2)
OpenJDK 64-Bit Server VM (build 24.0.2, mixed mode, sharing)
Linux 6.16.4-zen1-1-zen #1 ZEN SMP PREEMPT_DYNAMIC Thu, 28 Aug 2025 19:49:40 +0000 x86_64 GNU/Linux

Metadata

Metadata

Assignees

Labels

enhancementNew feature or request

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions