Skip to content

add Elastic Stack and Elastic Package version to test results report#3070

Merged
teresaromero merged 10 commits intoelastic:mainfrom
teresaromero:991-render-version-tests
Nov 24, 2025
Merged

add Elastic Stack and Elastic Package version to test results report#3070
teresaromero merged 10 commits intoelastic:mainfrom
teresaromero:991-render-version-tests

Conversation

@teresaromero
Copy link
Contributor

@teresaromero teresaromero commented Nov 12, 2025 •
edited
Loading

Resolves #991

Elastic stack version and elastic-package running version are now logged into the running test output when elastic-stack test is run.

@teresaromero
Copy link
Contributor Author

teresaromero commented Nov 12, 2025

@efd6 should this information be across all the tests? (static, asset, pipeline, etc...)

Let me know if the elastic-package version format is ok or you have any other thought

@efd6
Copy link
Contributor

efd6 commented Nov 12, 2025

AFIUI the version is a constant over the table, correct? If that's the case, the additional verbosity is not a win.

In the output of benchstat for example, the constant data is provide as a preamble to the table.

$ benchstat new.bench old.bench 
goos: darwin
goarch: amd64
pkg: github.com/elastic/go-libaudit/v2/aucoalesce
cpu: Intel(R) Core(TM) i9-9980HK CPU @ 2.40GHz
                    │  new.bench  │             old.bench              │
                    │   sec/op    │   sec/op     vs base               │
CoalesceMessages-16   4.191µ ± 3%   4.594µ ± 4%  +9.62% (p=0.000 n=10)

                    │  new.bench   │              old.bench              │
                    │     B/op     │     B/op      vs base               │
CoalesceMessages-16   2.753Ki ± 0%   2.786Ki ± 0%  +1.21% (p=0.000 n=10)

                    │ new.bench  │             old.bench             │
                    │ allocs/op  │ allocs/op   vs base               │
CoalesceMessages-16   13.00 ± 8%   14.00 ± 0%  +7.69% (p=0.003 n=10)

In terms of other thought, I'd love it if the tool emitted JSON instead of a table (or at least optionally instead of), since then any format can by chosen by the user after processing the machine-readable format.

@elasticmachine
Copy link
Collaborator

elasticmachine commented Nov 13, 2025

💚 Build Succeeded

History

@teresaromero
Copy link
Contributor Author

teresaromero commented Nov 14, 2025

AFIUI the version is a constant over the table, correct? If that's the case, the additional verbosity is not a win.

As you said, its not a result perse, but a constant defining the test. I've added as a command log when running each test. To be part of the table/xml i guess it should be part of the result and redundant to each test case.

In terms of other thought, I'd love it if the tool emitted JSON instead of a table (or at least optionally instead of), since then any format can by chosen by the user after processing the machine-readable format.

i've added the option to make a json file/output with the test results

@teresaromero teresaromero marked this pull request as ready for review November 14, 2025 11:14
@teresaromero teresaromero requested review from a team and efd6 November 14, 2025 11:15
jsoriano
jsoriano reviewed Nov 19, 2025
View reviewed changes
cmd/testrunner.go Outdated
}

cmd.Println(version.Version())
cmd.Printf("elastic-stack: %s\n", esClientInfo.Version.Number)
Copy link
Member

@jsoriano jsoriano Nov 19, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As this is printed by each test action, this is printed multiple times, once for each type of test, when running elastic-package test without specifying a subcommand. Is this intended?

Copy link
Contributor Author

@teresaromero teresaromero Nov 20, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

it is intentional, as the version from the stack is taken from the clients / profile.

would it make sense to create a esClient right at the begining of the test cmd to get the version?

Copy link
Member

@jsoriano jsoriano Nov 20, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

would it make sense to create a esClient right at the begining of the test cmd to get the version?

I don't think so, as not all tests need it. Let's leave it here by now.

internal/testrunner/reporters/outputs/file.go Outdated
case formats.ReportFormatXUnit:
ext = "xml"
case formats.ReportFormatJSON:
ext = "json"
Copy link
Member

@jsoriano jsoriano Nov 19, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think it would be better to move this to a separate PR. Or was this report intended to include the version too? 🤔

Copy link
Contributor Author

@teresaromero teresaromero Nov 20, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Initially i thought this data should appear at the report (table, xml) but by the feedback i got i understood it needed to be printed as a log. The propsal for json y guess is independent as it still reflects the results of the test and not the "environment" (versions) that have been used for them.

eitherway, i am extracting the commit to a new branch and reverting from this PR

@efd6 could you describe with more detail what is the request for this changes? should the version still be part of the results or just be logged into the test output? should the new json option reflect the versions too? thanks

Copy link
Contributor Author

@teresaromero teresaromero Nov 24, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

opening #3101 to track json format changes

@teresaromero teresaromero merged commit d4a4d7a into elastic:main Nov 24, 2025
3 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@jsoriano jsoriano jsoriano approved these changes

@efd6 efd6 Awaiting requested review from efd6

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

feature: render package, stack and elastic-package version into test reports

4 participants

@teresaromero @efd6 @elasticmachine @jsoriano