Skip to content

Tyche logging support#258

Merged
vasumv merged 4 commits intomasterfrom
vasu/tyche_support
Sep 9, 2024
Merged

Tyche logging support#258
vasumv merged 4 commits intomasterfrom
vasu/tyche_support

Conversation

@vasumv
Copy link
Collaborator

@vasumv vasumv commented Aug 30, 2024

Resolves #257

Logging support for Tyche (feature flag enabled by -DjqfObservability). The output file will be target/fuzz-results/TEST_CLASS/TEST_METHOD/observations.jsonl. The output file can then be uploaded to
https://tyche-pbt.github.io/tyche-extension/

Modifies FuzzStatement to add logging at the end of the guidance loop if the flag is enabled. Adds the fuzz.util.Observability.event method that can be invoked in the fuzz target to log specific input features. Per-input coverage still not yet implemented. Since the coverage data in Tyche is per-input line coverage for each file, this should be done as a post processing step.

Example fuzz target (for ChocoPy):

@Fuzz
public void fuzzSemanticAnalysis(@From(ChocoPySemanticGenerator.class) String code) {
    Program program = RefParser.process(code, false);
    assumeTrue(!program.hasErrors());
    event("numStatements", program.statements.size());
    event("numDeclarations", program.declarations.size());
    Program typedProgram = RefAnalysis.process(program);
    event("numErrors", program.getErrorList().size());
    assumeTrue(!typedProgram.hasErrors());
}

Example visualization:
image

@vasumv vasumv self-assigned this Aug 30, 2024
@vasumv vasumv marked this pull request as ready for review September 5, 2024 14:54
@vasumv vasumv requested a review from aoli-al September 5, 2024 18:33
Copy link
Owner

@rohanpadhye rohanpadhye left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, thanks!

@vasumv vasumv merged commit cb44a7b into master Sep 9, 2024
@vasumv vasumv deleted the vasu/tyche_support branch September 9, 2024 19:32
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Add logging support for Tyche

3 participants