qa: do not report errors on stderr as cluster log findings#66366
qa: do not report errors on stderr as cluster log findings#66366
Conversation
Fixes: https://tracker.ceph.com/issues/73953 Signed-off-by: Patrick Donnelly <pdonnell@ibm.com>
/teuthology/pdonnell-2025-11-30_03:13:50-fs-wip-pdonnell-testing-20251126.180742-debug-distro-default-smithi/8630995/teuthology.log works as expected |
| stdout = r.stdout.getvalue().decode() | ||
| if stdout: | ||
| return stdout | ||
| stderr = r.stderr.getvalue() |
There was a problem hiding this comment.
2025-11-20T19:44:37.966 INFO:teuthology.orchestra.run.smithi045.stderr:grep: /var/log/ceph/573c76ee-c649-11f0-877f-adfe0268badd/ceph.log: No such file or directory
It was added in #48539 specifically to catch cases where the cluster log file doesn't exist for some reason ;) Unfortunately this was the case for all cephadm-based jobs for years before it was noticed and fixed in #54312 and caused high-profile issues like https://tracker.ceph.com/issues/63389 to get missed.
There was a problem hiding this comment.
My main problem with it is that it obscures the real error for teuthology but that's more of a problem with teuthology I suppose.
There was a problem hiding this comment.
It's more with this bit in the task than with teuthology itself IMO -- the failure could certainly be signaled in a more sophisticated way than just returning grep's stderr. But I'd argue that an obscure error is better than no error and a false-pass result.
There was a problem hiding this comment.
Sure, but the error is only valid if it's from a test which would otherwise pass, no? In this case, cephadm task didn't even start successfully.
Anyway, I will probably update this to give a better error so moving this to draft.
There was a problem hiding this comment.
Sure, but the error is only valid if it's from a test which would otherwise pass, no?
Yes, that is the part where it's a problem with teuthology itself.
|
This pull request has been automatically marked as stale because it has not had any activity for 60 days. It will be closed if no further activity occurs for another 30 days. |
|
This pull request has been automatically closed because there has been no activity for 90 days. Please feel free to reopen this pull request (or open a new one) if the proposed change is still appropriate. Thank you for your contribution! |
Fixes: https://tracker.ceph.com/issues/73953
Contribution Guidelines
To sign and title your commits, please refer to Submitting Patches to Ceph.
If you are submitting a fix for a stable branch (e.g. "quincy"), please refer to Submitting Patches to Ceph - Backports for the proper workflow.
When filling out the below checklist, you may click boxes directly in the GitHub web UI. When entering or editing the entire PR message in the GitHub web UI editor, you may also select a checklist item by adding an
xbetween the brackets:[x]. Spaces and capitalization matter when checking off items this way.Checklist
Show available Jenkins commands
jenkins test classic perfJenkins Job | Jenkins Job Definitionjenkins test crimson perfJenkins Job | Jenkins Job Definitionjenkins test signedJenkins Job | Jenkins Job Definitionjenkins test make checkJenkins Job | Jenkins Job Definitionjenkins test make check arm64Jenkins Job | Jenkins Job Definitionjenkins test submodulesJenkins Job | Jenkins Job Definitionjenkins test dashboardJenkins Job | Jenkins Job Definitionjenkins test dashboard cephadmJenkins Job | Jenkins Job Definitionjenkins test apiJenkins Job | Jenkins Job Definitionjenkins test docsReadTheDocs | Github Workflow Definitionjenkins test ceph-volume allJenkins Jobs | Jenkins Jobs Definitionjenkins test windowsJenkins Job | Jenkins Job Definitionjenkins test rook e2eJenkins Job | Jenkins Job DefinitionYou must only issue one Jenkins command per-comment. Jenkins does not understand
comments with more than one command.