Skip to content

EvaluatorCSV produces unhelpful output when mixing various track versions in one ExecutionResultSet #140

@janothan

Description

@janothan

Describe the bug

Problem Description

  • An ExecutionResultSet (ERS) contains multiple ExecutionResults with test cases referring to the same track but to different track versions.
  • When this ERS is now evaluated by EvaluatorCSV, the trackPerformanceCube.csv evaluates the versions separately but it is not reconizable which track version was used.

To Reproduce

Reproducible in all MELT versions on all OS.

Full log output

Not relevant

Expected behavior

One of the following:

  1. Always print the track version.
  2. Automatically detect if multiple track versions are used and display the version in such cases in the produced CSV.
  3. Treat such cases as if this were only one track. Allow the user to manually set whether to treat track versions differently.

Metadata

Metadata

Assignees

Labels

bugSomething isn't working

Type

No type
No fields configured for issues without a type.

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions