Skip to content

[loganalyzer] Fix the IOError of opening match file#994

Merged
qiluo-msft merged 2 commits intosonic-net:masterfrom
wangxin:la-ioerror
Jul 9, 2019
Merged

[loganalyzer] Fix the IOError of opening match file#994
qiluo-msft merged 2 commits intosonic-net:masterfrom
wangxin:la-ioerror

Conversation

@wangxin
Copy link
Copy Markdown
Collaborator

@wangxin wangxin commented Jul 5, 2019

Description of PR

Summary:
Fixes # (issue)

After PR #970 was merged, log analyzer may fail with IOError: [Errno 2] No such file or directory: ''

It was caused by an extra comma appended to the common match file name. In the below ansible log, we can see that the match file list option -m loganalyzer_common_match.txt, has an unnecessary comma suffix. While parsing argument, loganalyzer.py would try to open a list of match files separated by comma. The second file in the list has empty file name and will cause the error.

Adjust location of the comma can fix the issue.

04:17:18 TASK [test : debug] ************************************************************
04:17:18 task path: /builds2/jenkins2/workspace/SONiC_Test-CRM@2/ansible/roles/test/files/tools/loganalyzer/loganalyzer_analyze.yml:122
04:17:18 Thursday 04 July 2019  20:17:18 +0000 (0:00:00.181)       0:05:49.878 ********* 
04:17:18 ok: [arc-switch1025-t1-lag] => {
04:17:18     "msg": "python /tmp/loganalyzer.py --action analyze --logs /tmp/syslog --run_id crm.2019-07-04-20:17:01 --out_dir /tmp/crm.2019-07-04-20:17:01 -m loganalyzer_common_match.txt, -i loganalyzer_common_ignore.txt -e expect_crm_th_exceeded -v"
04:17:18 }
04:17:18 
04:17:18 TASK [test : Invoke loganalyzer analyse crm] ***********************************
04:17:18 task path: /builds2/jenkins2/workspace/SONiC_Test-CRM@2/ansible/roles/test/files/tools/loganalyzer/loganalyzer_analyze.yml:124
04:17:18 Thursday 04 July 2019  20:17:18 +0000 (0:00:00.166)       0:05:50.045 ********* 
04:17:18 <arc-switch1025> ESTABLISH SSH CONNECTION FOR USER: admin
04:17:18 <arc-switch1025> SSH: ansible.cfg set ssh_args: (-o)(ControlMaster=auto)(-o)(ControlPersist=120s)(-o)(UserKnownHostsFile=/dev/null)
04:17:18 <arc-switch1025> SSH: ANSIBLE_HOST_KEY_CHECKING/host_key_checking disabled: (-o)(StrictHostKeyChecking=no)
04:17:18 <arc-switch1025> SSH: ANSIBLE_REMOTE_USER/remote_user/ansible_user/user/-u set: (-o)(User=admin)
04:17:18 <arc-switch1025> SSH: ANSIBLE_TIMEOUT/timeout set: (-o)(ConnectTimeout=30)
04:17:18 <arc-switch1025> SSH: PlayContext set ssh_common_args: ()
04:17:18 <arc-switch1025> SSH: PlayContext set ssh_extra_args: ()
04:17:18 <arc-switch1025> SSH: found only ControlPersist; added ControlPath: (-o)(ControlPath=/root/.ansible/cp/ansible-ssh-%h-%p-%r)
04:17:18 <arc-switch1025> SSH: EXEC sshpass -d14 ssh -C -vvv -o ControlMaster=auto -o ControlPersist=120s -o UserKnownHostsFile=/dev/null -o StrictHostKeyChecking=no -o User=admin -o ConnectTimeout=30 -o ControlPath=/root/.ansible/cp/ansible-ssh-%h-%p-%r arc-switch1025 '/bin/sh -c '"'"'sudo -H -S  -p "[sudo via ansible, key=aggfyvhmnxaghuntbeecioaicggthgeg] password: " -u root /bin/sh -c '"'"'"'"'"'"'"'"'echo BECOME-SUCCESS-aggfyvhmnxaghuntbeecioaicggthgeg; LANG=C LC_ALL=C LC_MESSAGES=C /usr/bin/python'"'"'"'"'"'"'"'"''"'"''
04:17:18 fatal: [arc-switch1025-t1-lag]: FAILED! => {"changed": true, "cmd": "python /tmp/loganalyzer.py --action analyze --logs /tmp/syslog --run_id crm.2019-07-04-20:17:01 --out_dir /tmp/crm.2019-07-04-20:17:01 -m loganalyzer_common_match.txt, -i loganalyzer_common_ignore.txt -e expect_crm_th_exceeded -v", "delta": "0:00:00.051856", "end": "2019-07-04 20:17:18.733369", "failed": true, "invocation": {"module_args": {"_raw_params": "python /tmp/loganalyzer.py --action analyze --logs /tmp/syslog --run_id crm.2019-07-04-20:17:01 --out_dir /tmp/crm.2019-07-04-20:17:01 -m loganalyzer_common_match.txt, -i loganalyzer_common_ignore.txt -e expect_crm_th_exceeded -v", "_uses_shell": true, "chdir": "/tmp", "creates": null, "executable": null, "removes": null, "warn": true}, "module_name": "command"}, "rc": 1, "start": "2019-07-04 20:17:18.681513", "stderr": "Traceback (most recent call last):\n  File \"/tmp/loganalyzer.py\", line 645, in <module>\n    main(sys.argv[1:])\n  File \"/tmp/loganalyzer.py\", line 625, in main\n    match_messages_regex, messages_regex_m = analyzer.create_msg_regex(match_file_list)\n  File \"/tmp/loganalyzer.py\", line 181, in create_msg_regex\n    with open(filename, 'rb') as csvfile:\nIOError: [Errno 2] No such file or directory: ''", "stdout": "[LogAnalyzer][diagnostic]:log file:/tmp/syslog, place marker end-LogAnalyzer-crm.2019-07-04-20:17:01\n[LogAnalyzer][diagnostic]:processing match file:loganalyzer_common_match.txt\n[LogAnalyzer][diagnostic]:[diagnostic]:processing row:0\n[LogAnalyzer][diagnostic]:row:['r', '\\\\.ERR', '\\\\.WARN', 'crash']\n[LogAnalyzer][diagnostic]:[diagnostic]:processing row:1\n[LogAnalyzer][diagnostic]:row:['r', 'kernel:.*Oops', 'kernel:.*hung', 'kernel.*oom\\\\s']\n[LogAnalyzer][diagnostic]:[diagnostic]:processing row:2\n[LogAnalyzer][diagnostic]:row:['r', 'kernel:.*scheduling', 'kernel:.*atomic', 'kernel:.*panic']\n[LogAnalyzer][diagnostic]:[diagnostic]:processing row:3\n[LogAnalyzer][diagnostic]:row:['r', 'kernel:.*\\\\serr', 'kernel:.*allocation', 'kernel:.*kill', '']\n[LogAnalyzer][diagnostic]:[diagnostic]:processing row:4\n[LogAnalyzer][diagnostic]:row:['r', 'kernel:.*kmemleak.*', 'kernel:.* Err:']\n[LogAnalyzer][diagnostic]:[diagnostic]:processing row:5\n[LogAnalyzer][diagnostic]:row:['s', 'ERR']\n[LogAnalyzer][diagnostic]:Built error string: ERR\n[LogAnalyzer][diagnostic]:processing match file:", "stdout_lines": ["[LogAnalyzer][diagnostic]:log file:/tmp/syslog, place marker end-LogAnalyzer-crm.2019-07-04-20:17:01", "[LogAnalyzer][diagnostic]:processing match file:loganalyzer_common_match.txt", "[LogAnalyzer][diagnostic]:[diagnostic]:processing row:0", "[LogAnalyzer][diagnostic]:row:['r', '\\\\.ERR', '\\\\.WARN', 'crash']", "[LogAnalyzer][diagnostic]:[diagnostic]:processing row:1", "[LogAnalyzer][diagnostic]:row:['r', 'kernel:.*Oops', 'kernel:.*hung', 'kernel.*oom\\\\s']", "[LogAnalyzer][diagnostic]:[diagnostic]:processing row:2", "[LogAnalyzer][diagnostic]:row:['r', 'kernel:.*scheduling', 'kernel:.*atomic', 'kernel:.*panic']", "[LogAnalyzer][diagnostic]:[diagnostic]:processing row:3", "[LogAnalyzer][diagnostic]:row:['r', 'kernel:.*\\\\serr', 'kernel:.*allocation', 'kernel:.*kill', '']", "[LogAnalyzer][diagnostic]:[diagnostic]:processing row:4", "[LogAnalyzer][diagnostic]:row:['r', 'kernel:.*kmemleak.*', 'kernel:.* Err:']", "[LogAnalyzer][diagnostic]:[diagnostic]:processing row:5", "[LogAnalyzer][diagnostic]:row:['s', 'ERR']", "[LogAnalyzer][diagnostic]:Built error string: ERR", "[LogAnalyzer][diagnostic]:processing match file:"], "warnings": []}

Type of change

  • Bug fix
  • [] Testbed and Framework(new/improvement)
  • [] Test case(new/improvement)

Approach

How did you do it?

Move the comma to correct location.

How did you verify/test it?

Tested on Mellanox platform

Any platform specific information?

N/A

Supported testbed topology if it's a new test case?

Documentation

Signed-off-by: Xin Wang <xinw@mellanox.com>
match_file_list: "{{ match_file_list | default([]) }} + {{ [test_match_file] }}"
when: test_match_file is defined

- name: Add mmatch file option
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

mmatch

@qiluo-msft qiluo-msft merged commit ce11c62 into sonic-net:master Jul 9, 2019
@wangxin wangxin deleted the la-ioerror branch September 26, 2019 12:36
sdszhang pushed a commit to sdszhang/sonic-mgmt that referenced this pull request Mar 23, 2026
<!--
Please make sure you've read and understood our contributing guidelines;
https://github.com/sonic-net/SONiC/blob/gh-pages/CONTRIBUTING.md

Please provide following information to help code review process a bit easier:
-->
### Description of PR
<!--
- Please include a summary of the change and which issue is fixed.
- Please also include relevant motivation and context. Where should reviewer start? background context?
- List any dependencies that are required for this change.
-->

Summary:
This add params for the qos sai tests for t0-f2-d40u8

### Type of change

<!--
- Fill x for your type of change.
- e.g.
- [x] Bug fix
-->

- [ ] Bug fix
- [ ] Testbed and Framework(new/improvement)
- [ ] New Test case
 - [ ] Skipped for non-supported platforms
- [ ] Test case improvement

### Back port request
- [ ] 202205
- [ ] 202305
- [ ] 202311
- [ ] 202405
- [ ] 202411
- [ ] 202505
- [ ] 202511

### Approach
#### What is the motivation for this PR?

#### How did you do it?

#### How did you verify/test it?

#### Any platform specific information?

#### Supported testbed topology if it's a new test case?

### Documentation
<!--
(If it's a new feature, new test case)
Did you update documentation/Wiki relevant to your implementation?
Link to the wiki page?
-->
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

6 participants