Skip to content

Allow outputing test results as allure framework results.#1115

Merged
jmchilton merged 2 commits intogalaxyproject:masterfrom
jmchilton:allure
Dec 28, 2020
Merged

Allow outputing test results as allure framework results.#1115
jmchilton merged 2 commits intogalaxyproject:masterfrom
jmchilton:allure

Conversation

@jmchilton
Copy link
Member

@jmchilton jmchilton commented Dec 16, 2020 •
edited
Loading

Implements #1113.

xref http://allure.qatools.ru/

Quick Usage:

brew install allure
# Convert a Planemo/Ephemeris tool_test_outpt.json into a allure results directory 
planemo test_reports --test_output_allure allure_results/ ~/tool_test_outpt.json
# Generate a static site from that.
allure generate --clean allure_results/
# Serve and open with a browser (allure has a little shortcut for this)
allure open allure-repors/

Landing summary.

Screen Shot 2020-12-16 at 1 12 29 AM

Can dig into "Categories" and filter by exception vs failure:

Screen Shot 2020-12-16 at 1 26 34 AM

Category view group similar problems together:

Screen Shot 2020-12-16 at 1 28 21 AM

Screen Shot 2020-12-16 at 1 12 45 AM

Suite view allows digging into tools -> versions -> test index. Clicking a test allows access to a lot of data collected for debugging the problem.

Screen Shot 2020-12-16 at 1 15 46 AM

Pretty graphs that don't do a whole lot in our context.

Screen Shot 2020-12-16 at 1 15 57 AM

Pretty timeline that doesn't do a whole lot in our context.

Screen Shot 2020-12-16 at 1 16 26 AM

@bgruening
Copy link
Member

bgruening commented Dec 16, 2020

That is pretty cool!

John is this --test_output_allure allure not redundant? Should this be --test_output allure?

@mvdbeek
Copy link
Member

mvdbeek commented Dec 16, 2020

That is in line with --test_output_markdown, --test_output_xunit etc.

@bgruening
Copy link
Member

bgruening commented Dec 16, 2020

And is this not redundant? Having --test_output_allure and the positional argument allure seems unnecessary to me. Should I open another issue about that?

@jmchilton
Copy link
Member Author

jmchilton commented Dec 16, 2020

Here allure is the directory that is being created, I don't think things are redundant. I'll update the example commands to clarify that.

@bgruening
Copy link
Member

bgruening commented Dec 16, 2020

Thanks @jmchilton for the explanation!

@jmchilton jmchilton merged commit ce17645 into galaxyproject:master Dec 28, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

3 participants

@jmchilton @bgruening @mvdbeek