Skip to content

qa/tasks/cbt.py: do not unzip in specified directory#35683

Closed
tchaikov wants to merge 2 commits intoceph:masterfrom
tchaikov:wip-qa-cbt
Closed

qa/tasks/cbt.py: do not unzip in specified directory#35683
tchaikov wants to merge 2 commits intoceph:masterfrom
tchaikov:wip-qa-cbt

Conversation

@tchaikov
Copy link
Contributor

@tchaikov tchaikov commented Jun 20, 2020

Checklist

  • References tracker ticket
  • Updates documentation if necessary
  • Includes tests for new functionality or reproducer for bug
Show available Jenkins commands
  • jenkins retest this please
  • jenkins test classic perf
  • jenkins test crimson perf
  • jenkins test signed
  • jenkins test make check
  • jenkins test make check arm64
  • jenkins test submodules
  • jenkins test dashboard
  • jenkins test dashboard backend
  • jenkins test docs
  • jenkins render docs
  • jenkins test ceph-volume all
  • jenkins test ceph-volume tox

tchaikov added 2 commits June 20, 2020 20:41
@tchaikov
qa/tasks/cbt.py: do not unzip in specified directory
b84d760 
before this change, we have:

$ mkdir -p -m0755 -- /home/ubuntu/cephtest/cosbench-0.4.2.c3.1
$ cd /home/ubuntu/cephtest && \
  wget http://drop.ceph.com/qa/cosbench-0.4.2.c3.1.zip && \
  unzip cosbench-0.4.2.c3.1.zip -d cosbench-0.4.2.c3
...
cosbench-0.4.2.c3.1/0.4.2.c3/stop-driver.sh
...
$ cd /home/ubuntu/cephtest && ln -s cosbench-0.4.2.c3.1 cos
$ cd /home/ubuntu/cephtest/cos && chmod +x *.sh
chmod: cannot access '*.sh': No such file or directory

after this change, cosbench-0.4.2.c3.1.zip is unzipped
right in /home/ubuntu/cephtest without "-d" option.

Signed-off-by: Kefu Chai <kchai@redhat.com>
@tchaikov
qa/tasks/cbt.py: pass "-q" to unzip
9baac24 
less noise in the log message this way

Signed-off-by: Kefu Chai <kchai@redhat.com>
@tchaikov
Copy link
Contributor Author

tchaikov commented Jun 20, 2020

this change should address the failure of http://pulpito.ceph.com/kchai-2020-06-20_03:37:44-rados-wip-kefu2-testing-2020-06-19-2355-distro-basic-smithi/5166727/

This was referenced Jun 20, 2020
Merged
Merged
Merged
Merged
Merged
@tchaikov
Copy link
Contributor Author

tchaikov commented Jun 20, 2020

http://pulpito.ceph.com/kchai-2020-06-20_12:58:10-rados-wip-kefu2-testing-2020-06-19-2355-distro-basic-smithi/

@tchaikov tchaikov requested a review from neha-ojha June 20, 2020 13:29
@tchaikov
Copy link
Contributor Author

tchaikov commented Jun 21, 2020 

2020-06-20T13:14:21.934 INFO:teuthology.orchestra.run.smithi087:> cd /home/ubuntu/cephtest/cos && chmod +x *.sh
2020-06-20T13:14:21.982 INFO:tasks.cbt:start cosbench
2020-06-20T13:14:21.983 INFO:teuthology.orchestra.run.smithi087:> cd /home/ubuntu/cephtest && cd cos && sh start-all.sh
2020-06-20T13:14:22.069 INFO:teuthology.orchestra.run.smithi087.stdout:Launching osgi framwork ...
2020-06-20T13:14:23.071 INFO:teuthology.orchestra.run.smithi087.stdout:Successfully launched osgi framework!
2020-06-20T13:14:23.072 INFO:teuthology.orchestra.run.smithi087.stdout:Booting cosbench driver ...
2020-06-21T01:00:54.794 DEBUG:teuthology.exit:Got signal 15; running 2 handlers...
2020-06-21T01:00:54.810 DEBUG:teuthology.task.console_log:Killing console logger for smithi087
2020-06-21T01:00:54.811 DEBUG:teuthology.task.console_log:Killing console logger for smithi087

@tchaikov
Copy link
Contributor Author

tchaikov commented Jun 22, 2020

jenkins test make check

@tchaikov
Copy link
Contributor Author

tchaikov commented Jun 23, 2020 •
edited
Loading

@neha-ojha the test still hangs. i added set -ex in the involved scripts:

:~/cephtest/cos$ bash start-driver.sh
+ ip=127.0.0.1
+ num=1
+ base_port=18088
+ '[' 0 -eq 0 ']'
+ num=1
+ SERVICE_NAME=driver
++ cat VERSION
+ VERSION=0.4.2
+ OSGI_BUNDLES='cosbench-log_0.4.2 cosbench-tomcat_0.4.2 cosbench-config_0.4.2 cosbench-http_0.4.2 cosbench-cdmi-util_0.4.2 cosbench-core_0.4.2 cosbench-core-web_0.4.2 cosbench-api_0.4.2 cosbench-mock_0.4.2 cosbench-ampli_0.4.2 cosbench-swift_0.4.2 cosbench-keystone_0.4.2 cosbench-httpauth_0.4.2 cosbench-s3_0.4.2 cosbench-librados_0.4.2 cosbench-scality_0.4.2 cosbench-cdmi-swift_0.4.2 cosbench-cdmi-base_0.4.2 cosbench-driver_0.4.2 cosbench-driver-web_0.4.2'
+ rm -f ip-port.list
++ seq 1 1
+ for i in $(seq 1 $num)
+ name=
+ url=
+ cd conf
+ let 'x=(1-1)*100+18088'
+ rm -f driver.conf
+ rm -f driver_1.conf
+ cp driver_template.conf driver_1.conf
+ name=127.0.0.1:18088
+ url='http:\/\/127.0.0.1:18088\/driver'
+ sed -i 's/^name=.*$/name=127.0.0.1:18088/' driver_1.conf
+ sed -i 's/^url=.*$/url=http:\/\/127.0.0.1:18088\/driver/' driver_1.conf
+ ln -s driver_1.conf driver.conf
+ rm -f driver-tomcat-server.xml
+ rm -f driver-tomcat-server_1.xml
+ cp driver-tomcat-server_template.xml driver-tomcat-server_1.xml
+ sed -i 's/\(.*Connector port="\)[^"]*\(.*\)/\118088\2/' driver-tomcat-server_1.xml
+ let x=18088+1
+ ln -s driver-tomcat-server_1.xml driver-tomcat-server.xml
+ cd ../
+ OSGI_CONSOLE_PORT=18089
+ sh cosbench-start.sh driver 'cosbench-log_0.4.2 cosbench-tomcat_0.4.2 cosbench-config_0.4.2 cosbench-http_0.4.2 cosbench-cdmi-util_0.4.2 cosbench-core_0.4.2 cosbench-core-web_0.4.2 cosbench-api_0.4.2 cosbench-mock_0.4.2 cosbench-ampli_0.4.2 cosbench-swift_0.4.2 cosbench-keystone_0.4.2 cosbench-httpauth_0.4.2 cosbench-s3_0.4.2 cosbench-librados_0.4.2 cosbench-scality_0.4.2 cosbench-cdmi-swift_0.4.2 cosbench-cdmi-base_0.4.2 cosbench-driver_0.4.2 cosbench-driver-web_0.4.2' 18089
+ SERVICE_NAME=driver
+ BOOT_LOG=log/driver-boot.log
+ OSGI_BUNDLES=cosbench-log_0.4.2 cosbench-tomcat_0.4.2 cosbench-config_0.4.2 cosbench-http_0.4.2 cosbench-cdmi-util_0.4.2 cosbench-core_0.4.2 cosbench-core-web_0.4.2 cosbench-api_0.4.2 cosbench-mock_0.4.2 cosbench-ampli_0.4.2 cosbench-swift_0.4.2 cosbench-keystone_0.4.2 cosbench-httpauth_0.4.2 cosbench-s3_0.4.2 cosbench-librados_0.4.2 cosbench-scality_0.4.2 cosbench-cdmi-swift_0.4.2 cosbench-cdmi-base_0.4.2 cosbench-driver_0.4.2 cosbench-driver-web_0.4.2
+ OSGI_CONSOLE_PORT=18089
+ OSGI_CONFIG=conf/.driver
+ TOMCAT_CONFIG=conf/driver-tomcat-server.xml
+ TOOL=nc
+ TOOL_PARAMS=
+ rm -f log/driver-boot.log
+ mkdir -p log
+ echo Launching osgi framwork ...
Launching osgi framwork ...
+ [ 0 -ne 0 ]
+ sleep 1
+ /usr/bin/nohup java -Dcosbench.tomcat.config=conf/driver-tomcat-server.xml -server -cp main/org.eclipse.equinox.launcher_1.2.0.v20110502.jar org.eclipse.equinox.launcher.Main -configuration conf/.driver -console 18089
+ echo Successfully launched osgi framework!
Successfully launched osgi framework!
+ echo Booting cosbench driver ...
Booting cosbench driver ...
+ succ=1
+ which nc
+ [ 0 -ne 0 ]
+ ready=0
+ attempts=60
+ [ 0 -ne 1 ]
+ echo ss -s ACTIVE cosbench
+ nc 0.0.0.0 18089
+ grep cosbench-log_0.4.2

@neha-ojha
Copy link
Member

neha-ojha commented Jun 24, 2020

@neha-ojha the test still hangs. i added set -ex in the involved scripts:

:~/cephtest/cos$ bash start-driver.sh
+ ip=127.0.0.1
+ num=1
+ base_port=18088
+ '[' 0 -eq 0 ']'
+ num=1
+ SERVICE_NAME=driver
++ cat VERSION
+ VERSION=0.4.2
+ OSGI_BUNDLES='cosbench-log_0.4.2 cosbench-tomcat_0.4.2 cosbench-config_0.4.2 cosbench-http_0.4.2 cosbench-cdmi-util_0.4.2 cosbench-core_0.4.2 cosbench-core-web_0.4.2 cosbench-api_0.4.2 cosbench-mock_0.4.2 cosbench-ampli_0.4.2 cosbench-swift_0.4.2 cosbench-keystone_0.4.2 cosbench-httpauth_0.4.2 cosbench-s3_0.4.2 cosbench-librados_0.4.2 cosbench-scality_0.4.2 cosbench-cdmi-swift_0.4.2 cosbench-cdmi-base_0.4.2 cosbench-driver_0.4.2 cosbench-driver-web_0.4.2'
+ rm -f ip-port.list
++ seq 1 1
+ for i in $(seq 1 $num)
+ name=
+ url=
+ cd conf
+ let 'x=(1-1)*100+18088'
+ rm -f driver.conf
+ rm -f driver_1.conf
+ cp driver_template.conf driver_1.conf
+ name=127.0.0.1:18088
+ url='http:\/\/127.0.0.1:18088\/driver'
+ sed -i 's/^name=.*$/name=127.0.0.1:18088/' driver_1.conf
+ sed -i 's/^url=.*$/url=http:\/\/127.0.0.1:18088\/driver/' driver_1.conf
+ ln -s driver_1.conf driver.conf
+ rm -f driver-tomcat-server.xml
+ rm -f driver-tomcat-server_1.xml
+ cp driver-tomcat-server_template.xml driver-tomcat-server_1.xml
+ sed -i 's/\(.*Connector port="\)[^"]*\(.*\)/\118088\2/' driver-tomcat-server_1.xml
+ let x=18088+1
+ ln -s driver-tomcat-server_1.xml driver-tomcat-server.xml
+ cd ../
+ OSGI_CONSOLE_PORT=18089
+ sh cosbench-start.sh driver 'cosbench-log_0.4.2 cosbench-tomcat_0.4.2 cosbench-config_0.4.2 cosbench-http_0.4.2 cosbench-cdmi-util_0.4.2 cosbench-core_0.4.2 cosbench-core-web_0.4.2 cosbench-api_0.4.2 cosbench-mock_0.4.2 cosbench-ampli_0.4.2 cosbench-swift_0.4.2 cosbench-keystone_0.4.2 cosbench-httpauth_0.4.2 cosbench-s3_0.4.2 cosbench-librados_0.4.2 cosbench-scality_0.4.2 cosbench-cdmi-swift_0.4.2 cosbench-cdmi-base_0.4.2 cosbench-driver_0.4.2 cosbench-driver-web_0.4.2' 18089
+ SERVICE_NAME=driver
+ BOOT_LOG=log/driver-boot.log
+ OSGI_BUNDLES=cosbench-log_0.4.2 cosbench-tomcat_0.4.2 cosbench-config_0.4.2 cosbench-http_0.4.2 cosbench-cdmi-util_0.4.2 cosbench-core_0.4.2 cosbench-core-web_0.4.2 cosbench-api_0.4.2 cosbench-mock_0.4.2 cosbench-ampli_0.4.2 cosbench-swift_0.4.2 cosbench-keystone_0.4.2 cosbench-httpauth_0.4.2 cosbench-s3_0.4.2 cosbench-librados_0.4.2 cosbench-scality_0.4.2 cosbench-cdmi-swift_0.4.2 cosbench-cdmi-base_0.4.2 cosbench-driver_0.4.2 cosbench-driver-web_0.4.2
+ OSGI_CONSOLE_PORT=18089
+ OSGI_CONFIG=conf/.driver
+ TOMCAT_CONFIG=conf/driver-tomcat-server.xml
+ TOOL=nc
+ TOOL_PARAMS=
+ rm -f log/driver-boot.log
+ mkdir -p log
+ echo Launching osgi framwork ...
Launching osgi framwork ...
+ [ 0 -ne 0 ]
+ sleep 1
+ /usr/bin/nohup java -Dcosbench.tomcat.config=conf/driver-tomcat-server.xml -server -cp main/org.eclipse.equinox.launcher_1.2.0.v20110502.jar org.eclipse.equinox.launcher.Main -configuration conf/.driver -console 18089
+ echo Successfully launched osgi framework!
Successfully launched osgi framework!
+ echo Booting cosbench driver ...
Booting cosbench driver ...
+ succ=1
+ which nc
+ [ 0 -ne 0 ]
+ ready=0
+ attempts=60
+ [ 0 -ne 1 ]
+ echo ss -s ACTIVE cosbench
+ nc 0.0.0.0 18089
+ grep cosbench-log_0.4.2

@tchaikov don't think anything has changed in cosbench since we use https://github.com/ceph/ceph/blob/master/qa/tasks/cbt.py#L124, but using TOOL_PARAMS="-q 0" takes me further

@neha-ojha
Copy link
Member

neha-ojha commented Jun 30, 2020

We don't need this PR, see https://tracker.ceph.com/issues/46222#note-4 for details.

@neha-ojha neha-ojha closed this Jun 30, 2020
@tchaikov tchaikov deleted the wip-qa-cbt branch July 1, 2020 01:45
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@neha-ojha neha-ojha neha-ojha approved these changes

Assignees

No one assigned

Labels

needs-qa tests wip-kefu-testing

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

2 participants

@tchaikov @neha-ojha