Add --detailed option for buildtest report by szuananwar · Pull Request #1567 · buildtesters/buildtest

szuananwar · 2023-07-19T00:38:34Z

This new feature will let the user to run
buildtest rt --format name,id,user,state,returncode,runtime,outfile,errfile,buildspec --count=1
By running
buildtest rt --detailed

codecov · 2023-07-19T00:43:58Z

Codecov Report

Patch coverage: 62.75% and project coverage change: +0.51% 🎉

Comparison is base (06888a8) 77.86% compared to head (eedb821) 78.36%.
Report is 46 commits behind head on devel.

Additional details and impacted files

@@            Coverage Diff             @@
##            devel    #1567      +/-   ##
==========================================
+ Coverage   77.86%   78.36%   +0.51%     
==========================================
  Files          57       56       -1     
  Lines        6621     6577      -44     
==========================================
- Hits         5155     5154       -1     
+ Misses       1466     1423      -43

Files Changed	Coverage Δ
buildtest/cli/config.py	`86.76% <ø> (ø)`
buildtest/executors/cobalt.py	`21.51% <0.00%> (-0.60%)`	⬇️
buildtest/executors/lsf.py	`19.28% <0.00%> (-0.72%)`	⬇️
buildtest/executors/pbs.py	`21.43% <0.00%> (-0.49%)`	⬇️
buildtest/executors/slurm.py	`20.51% <0.00%> (-0.74%)`	⬇️
buildtest/cli/report.py	`96.08% <90.91%> (+6.18%)`	⬆️
buildtest/builders/base.py	`89.46% <100.00%> (+0.13%)`	⬆️
buildtest/cli/__init__.py	`98.20% <100.00%> (-1.80%)`	⬇️
buildtest/executors/local.py	`98.00% <100.00%> (+3.77%)`	⬆️

... and 2 files with indirect coverage changes

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

…ry_test_report.rst

buildtest/cli/report.py

tests/cli/test_report.py

szuananwar · 2023-07-19T16:38:22Z

@shahzebsiddiqui Here is the buildtest report --detailed command result

buildtest/cli/__init__.py

buildtest/cli/report.py

shahzebsiddiqui · 2023-07-19T18:33:41Z

@szuananwar okay taking a step back and looking at the whole code base i think the way you implemented this method is not going to work with all the other options. We really need to pass detailed as an argument to Report class

Simply running the following command it shows 1 entry which is not correct it should show entries specified by --count or one specified in configuration file

(buildtest)  ~/Documents/github/buildtest/ [add_detailed_option] buildtest rt -d
                                                                                Report File: /Users/siddiq90/Documents/github/buildtest/var/report.json                                                                                 
┏━━━━━━━━━━━━┳━━━━━━━━━━┳━━━━━━━━━━┳━━━━━━━┳━━━━━━━━━━━━┳━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┓
┃ name       ┃ id       ┃ user     ┃ state ┃ returncode ┃ runtime ┃ outfile                                              ┃ errfile                                              ┃ buildspec                                            ┃
┡━━━━━━━━━━━━╇━━━━━━━━━━╇━━━━━━━━━━╇━━━━━━━╇━━━━━━━━━━━━╇━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┩
│ exit1_fail │ 47c2296e │ siddiq90 │ FAIL  │ 1          │ 0.32758 │ /Users/siddiq90/Documents/github/buildtest/var/tests │ /Users/siddiq90/Documents/github/buildtest/var/tests │ /Users/siddiq90/Documents/github/buildtest/tutorials │
│            │          │          │       │            │         │ /generic.local.bash/pass_returncode/exit1_fail/47c22 │ /generic.local.bash/pass_returncode/exit1_fail/47c22 │ /test_status/pass_returncode.yml                     │
│            │          │          │       │            │         │ 96e/exit1_fail.out                                   │ 96e/exit1_fail.err                                   │                                                      │
└────────────┴──────────┴──────────┴───────┴────────────┴─────────┴──────────────────────────────────────────────────────┴──────────────────────────────────────────────────────┴──────────────────────────────────────────────────────┘

Also we should make --detailed and --format be mutually exclusive options if that makes sense. In example below we should not have this happen because -d will expose pre-selected columns but when --format is specified then we want to specify our own columns

(buildtest)  ~/Documents/github/buildtest/ [add_detailed_option] buildtest rt -d --format name
                                                                                Report File: /Users/siddiq90/Documents/github/buildtest/var/report.json                                                                                 
┏━━━━━━━━━━━━┳━━━━━━━━━━┳━━━━━━━━━━┳━━━━━━━┳━━━━━━━━━━━━┳━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┓
┃ name       ┃ id       ┃ user     ┃ state ┃ returncode ┃ runtime ┃ outfile                                              ┃ errfile                                              ┃ buildspec                                            ┃
┡━━━━━━━━━━━━╇━━━━━━━━━━╇━━━━━━━━━━╇━━━━━━━╇━━━━━━━━━━━━╇━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┩
│ exit1_fail │ 47c2296e │ siddiq90 │ FAIL  │ 1          │ 0.32758 │ /Users/siddiq90/Documents/github/buildtest/var/tests │ /Users/siddiq90/Documents/github/buildtest/var/tests │ /Users/siddiq90/Documents/github/buildtest/tutorials │
│            │          │          │       │            │         │ /generic.local.bash/pass_returncode/exit1_fail/47c22 │ /generic.local.bash/pass_returncode/exit1_fail/47c22 │ /test_status/pass_returncode.yml                     │
│            │          │          │       │            │         │ 96e/exit1_fail.out                                   │ 96e/exit1_fail.err                                   │                                                      │
└────────────┴──────────┴──────────┴───────┴────────────┴─────────┴──────────────────────────────────────────────────────┴──────────────────────────────────────────────────────┴───────────────────────

buildtest/cli/report.py

docs/gettingstarted/query_test_report.rst

…together

docs/gettingstarted/query_test_report.rst

tests/cli/test_report.py

Add --detailed and -d options

afcdce3

pull-request-size bot added the size/XS label Jul 19, 2023

Add --detailed and -d to local options under report/rt command

06e3ad7

szuananwar force-pushed the add_detailed_option branch from 7fe465d to 06e3ad7 Compare July 19, 2023 00:51

szuananwar and others added 2 commits July 18, 2023 19:51

Merge branch 'buildtesters:devel' into add_detailed_option

73402c4

Added "Detailed Report (buildtest report --detailed" paragraph to que…

2c45a21

…ry_test_report.rst

pull-request-size bot added size/S and removed size/XS labels Jul 19, 2023

Add implementation for --detailed option

25080ce

pull-request-size bot added size/M and removed size/S labels Jul 19, 2023

szuananwar linked an issue Jul 19, 2023 that may be closed by this pull request

[FEATURE]: buildtest report --detailed #1456

Closed

7 tasks

szuananwar requested a review from shahzebsiddiqui July 19, 2023 01:07

szuananwar self-assigned this Jul 19, 2023

Add regression test for --detailed option

b127342

shahzebsiddiqui reviewed Jul 19, 2023

View reviewed changes

buildtest/cli/report.py Outdated Show resolved Hide resolved

shahzebsiddiqui reviewed Jul 19, 2023

View reviewed changes

tests/cli/test_report.py Outdated Show resolved Hide resolved

szuananwar added 2 commits July 19, 2023 11:33

Update the report_detailed method

c55e844

Update the test_report_detailed method

03bb642

szuananwar requested a review from shahzebsiddiqui July 19, 2023 16:42

szuananwar added 2 commits July 19, 2023 11:56

Fix style issue

0e6d2ff

Fix style issue

bf9002c

shahzebsiddiqui reviewed Jul 19, 2023

View reviewed changes

buildtest/cli/__init__.py Outdated Show resolved Hide resolved

shahzebsiddiqui reviewed Jul 19, 2023

View reviewed changes

buildtest/cli/report.py Outdated Show resolved Hide resolved

shahzebsiddiqui reviewed Jul 19, 2023

View reviewed changes

buildtest/cli/report.py Outdated Show resolved Hide resolved

shahzebsiddiqui reviewed Jul 19, 2023

View reviewed changes

buildtest/cli/report.py Outdated Show resolved Hide resolved

Add --detailed under optional arguments.

5c981f0

shahzebsiddiqui reviewed Jul 28, 2023

View reviewed changes

buildtest/cli/report.py Outdated Show resolved Hide resolved

shahzebsiddiqui reviewed Jul 28, 2023

View reviewed changes

docs/gettingstarted/query_test_report.rst Outdated Show resolved Hide resolved

shahzebsiddiqui reviewed Jul 28, 2023

View reviewed changes

docs/gettingstarted/query_test_report.rst Outdated Show resolved Hide resolved

shahzebsiddiqui reviewed Jul 28, 2023

View reviewed changes

docs/gettingstarted/query_test_report.rst Outdated Show resolved Hide resolved

szuananwar added 3 commits July 28, 2023 14:40

Fix indentation and double ticks

05aaea1

Raise an exception if both format and detailed options are specified …

bb018c3

…together

Add regression test for buildtest rt -d/--detailed

7f7a030

shahzebsiddiqui reviewed Jul 31, 2023

View reviewed changes

docs/gettingstarted/query_test_report.rst Show resolved Hide resolved

shahzebsiddiqui reviewed Jul 31, 2023

View reviewed changes

tests/cli/test_report.py Outdated Show resolved Hide resolved

shahzebsiddiqui reviewed Jul 31, 2023

View reviewed changes

tests/cli/test_report.py Show resolved Hide resolved

szuananwar added 2 commits July 31, 2023 13:20

Add regression test for buildtest rt -d/--detailed --format "name"

62467b3

Fix doc rendering issue

7ac315e

szuananwar requested a review from shahzebsiddiqui July 31, 2023 18:39

Fix doc rendering issue

8cab76f

shahzebsiddiqui reviewed Jul 31, 2023

View reviewed changes

tests/cli/test_report.py Outdated Show resolved Hide resolved

szuananwar added 4 commits July 31, 2023 16:01

Create a class variable format_fields_detailed

ac93439

raise an exception to capture specifying both --detailed and --format

91dbb51

Fix style check error

931e000

Fix style check error

619b9ce

pull-request-size bot added size/L and removed size/M labels Jul 31, 2023

Fix style check error by removing ()

5ad52a1

pull-request-size bot added size/M and removed size/L labels Jul 31, 2023

szuananwar and others added 5 commits July 31, 2023 16:51

Remove ['py311'] under tool.black

f00a896

Add comments to test --detailed --format in test_report_detailed method

3cedce2

Add 'py311' under tool.black

012f879

Add detailed args to report path method

28bf2c1

fix issue with documentation

eedb821

shahzebsiddiqui merged commit 6ee4a47 into buildtesters:devel Aug 1, 2023

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Add --detailed option for buildtest report#1567

Add --detailed option for buildtest report#1567
shahzebsiddiqui merged 42 commits intobuildtesters:develfrom
szuananwar:add_detailed_option

szuananwar commented Jul 19, 2023

Uh oh!

codecov bot commented Jul 19, 2023 •

edited

Loading

Uh oh!

Uh oh!

Uh oh!

szuananwar commented Jul 19, 2023

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

shahzebsiddiqui commented Jul 19, 2023

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Uh oh!

Conversation

szuananwar commented Jul 19, 2023

Uh oh!

codecov bot commented Jul 19, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

Uh oh!

Uh oh!

szuananwar commented Jul 19, 2023

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

shahzebsiddiqui commented Jul 19, 2023

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

codecov bot commented Jul 19, 2023 •

edited

Loading