run systemtests on installed packages by pstorz · Pull Request #400 · bareos/bareos

pstorz · 2020-01-24T12:30:40Z

This pull request enables the system tests to be optionally run on installed binaries like from packages.

arogge

Does a bit more than the description of the PR suggest, but OK.
The cmake formatting needs a quick review though.

arogge · 2020-01-24T12:55:26Z

systemtests/bin/bareos

 . ./environment
 ${scripts}/bareos-config initialize_database_driver
-${scripts}/bareos $@
+${rscripts}/bareos $@


was that an intentional change?

Yes, that was intended. When running on installed files, some of the scripts are taken from the installation (scripts) and some are not part of the packages so need to be taken from the source tree (rscripts).

core/scripts/bareos-config-lib.sh.in

arogge · 2020-01-24T12:59:04Z

core/src/cats/CMakeLists.txt

-  bareossql
-  PROPERTIES VERSION "${BAREOS_NUMERIC_VERSION}" SOVERSION
-                                                 "${BAREOS_VERSION_MAJOR}"
+  bareossql PROPERTIES VERSION "${BAREOS_NUMERIC_VERSION}" SOVERSION


something seems to be weird with cmake-format here, it seems to have formatted in a different way for you

I verified that my cmake-format does it that way.

franku

Looks good. However, some missing quotes here and there.

franku · 2020-01-24T14:42:10Z

core/src/tests/catalog.cc

 static std::string getenv_std_string(std::string env_var)
 {
-  auto v{std::getenv(env_var.c_str())};
+  const char* v=(std::getenv(env_var.c_str()));


Suggested change

const char* v=(std::getenv(env_var.c_str()));

const char* v = (std::getenv(env_var.c_str()));

franku · 2020-01-24T14:42:22Z

core/src/tests/catalog.cc

 {
  std::vector<char> stime;
-  auto jcr{directordaemon::NewDirectorJcr()};
+  auto jcr=directordaemon::NewDirectorJcr();


Suggested change

auto jcr=directordaemon::NewDirectorJcr();

auto jcr = directordaemon::NewDirectorJcr();

franku · 2020-01-24T14:42:57Z

systemtests/bin/bareos

@@ -2,4 +2,4 @@
 # script to start and stop daemons for individual test
 . ./environment
 ${scripts}/bareos-config initialize_database_driver


Suggested change

${scripts}/bareos-config initialize_database_driver

"${scripts}"/bareos-config initialize_database_driver

franku · 2020-01-24T14:43:42Z

systemtests/bin/bconsole

 # wrapper script to start/stop bconsole for each individual test
 . ./environment
-${bin}/bconsole -c ${conf} "$@"
+${BAREOS_BCONSOLE_BINARY} -c ${conf} "$@"


Suggested change

${BAREOS_BCONSOLE_BINARY} -c ${conf} "$@"

"${BAREOS_BCONSOLE_BINARY}" -c "${conf}" "$@"

BAREOS_BCONSOLE_BINARY and others should all be quoted accordingly.

franku · 2020-01-24T14:44:43Z

systemtests/environment.in

 PROJECT_BINARY_DIR=@PROJECT_BINARY_DIR@
-bin=${PROJECT_BINARY_DIR}/bin
-sbin=${PROJECT_BINARY_DIR}/sbin
+bin=@bin@


Suggested change

bin=@bin@

bin="@bin@"

franku · 2020-01-24T14:45:05Z

systemtests/environment.in


 dumps=${current_test_directory}/dumps
-plugindir=${PROJECT_BINARY_DIR}/plugins
+plugindir=@plugins@


Suggested change

plugindir=@plugins@

plugindir="@plugins@"

franku · 2020-01-24T14:48:03Z

systemtests/scripts/setup

-echo "running $0"
 ${scripts}/bareos-config initialize_database_driver
-. ${scripts}/functions
+. ${rscripts}/functions


Suggested change

. ${rscripts}/functions

. "${rscripts}/functions"

franku · 2020-01-24T14:49:07Z

systemtests/tests/bconsole-pam/testrunner

 END_OF_DATA

-if ! ${bin}/bconsole -c "${conf}" -p etc/user1.cred < $tmp/bconcmds >${tmp}/log1.out 2>${tmp}/err1.out; then
+if ! ${BAREOS_BCONSOLE_BINARY} -c "${conf}" -p etc/user1.cred < $tmp/bconcmds >${tmp}/log1.out 2>${tmp}/err1.out; then


Suggested change

if ! ${BAREOS_BCONSOLE_BINARY} -c "${conf}" -p etc/user1.cred < $tmp/bconcmds >${tmp}/log1.out 2>${tmp}/err1.out; then

if ! "${BAREOS_BCONSOLE_BINARY}" -c "${conf}" -p etc/user1.cred < $tmp/bconcmds >${tmp}/log1.out 2>${tmp}/err1.out; then

franku · 2020-01-24T14:49:49Z

systemtests/tests/bconsole-pam/testrunner

 print_debug "OK: login as user1 succeeded."

-if ${bin}/bconsole -c "${conf}" -p etc/user2.cred < $tmp/bconcmds >${tmp}/log2.out 2>${tmp}/err2.out; then
+if ${BAREOS_BCONSOLE_BINARY} -c "${conf}" -p etc/user2.cred < $tmp/bconcmds >${tmp}/log2.out 2>${tmp}/err2.out; then


Suggested change

if ${BAREOS_BCONSOLE_BINARY} -c "${conf}" -p etc/user2.cred < $tmp/bconcmds >${tmp}/log2.out 2>${tmp}/err2.out; then

if "${BAREOS_BCONSOLE_BINARY}" -c "${conf}" -p etc/user2.cred < $tmp/bconcmds >${tmp}/log2.out 2>${tmp}/err2.out; then

check other occurences as well, please

franku · 2020-01-24T14:51:06Z

systemtests/tests/bconsole-pam/testrunner

-
-
-if ! ${scripts}/bareos-ctl-dir status >/dev/null; then
+if ! ${rscripts}/bareos-ctl-dir status >/dev/null; then


Suggested change

if ! ${rscripts}/bareos-ctl-dir status >/dev/null; then

if ! "${rscripts}"/bareos-ctl-dir status >/dev/null; then

- moved common reload-xxx-test functions into one script - quoting of variables - enable shellcheck to follow sourced files - added missing function parameters - fix order of output redirection - set /bin/bash as shell where required

arogge · 2020-01-28T13:22:04Z

systemtests/bin/bconsole

 . ./environment
-${BAREOS_BCONSOLE_BINARY} -c ${conf} "$@"
+
+"${BAREOS_BCONSOLE_BINARY}" -c ${conf} "$@"


${conf} -> "${conf}"

arogge · 2020-01-28T13:25:09Z

systemtests/scripts/cleanup

+rm -rf "${working:?}"/CLEANUPMARKER
+rm -rf "${working:?}"/plugins/*
+rm -rf "${archivedir:?}"/*
+rm -f  "${config_directory_dir_additional_test_config}"/*


arogge · 2020-01-28T13:28:31Z

systemtests/scripts/functions

   fi

-   if [ $bstat != 0 -o $rstat != 0 ] ; then
+   if [ $bstat != 0 ] || [ $rstat != 0 ] ; then


"!=" does a string-comparison, numerically not-equal is "-ne"

arogge · 2020-01-28T13:30:25Z

systemtests/scripts/functions

      exit 1
   fi
-   if [ "$dstat" != "0" -o "$bstat" != "0" -o "$rstat" != "0" ] ; then
+   if [ "$dstat" != "0" ] || [ "$bstat" != "0" ] || [ "$rstat" != "0" ] ; then


same as above. Should be -ne instead of !=

arogge · 2020-01-28T13:30:29Z

systemtests/scripts/functions

      echo "  !!!!! $TestName failed!!! $ENDDATE !!!!! " >>test.out
      echo "   Status: zombie=$zstat backup=$bstat restore=$rstat diff=$dstat" >>test.out
-      if [ $bstat != 0 -o $rstat != 0 ] ; then
+      if [ $bstat != 0 ] || [ $rstat != 0 ] ; then


same as above. Should be -ne instead of !=

arogge · 2020-01-28T13:42:38Z

systemtests/tests/bconsole-status-client/testrunner

+#shellcheck source=../environment.in
 . ./environment
+
+JobName=backup-bareos-fd


is this correct? The original JobName was "bconsole-status-client"

JobName is irrelevant here as no job is run at all in this test.

arogge · 2020-01-28T13:43:39Z

systemtests/tests/copy-bscan/testrunner

 num_eos=$(grep -c '^End Job Session Record:$' "$tmp/bscan.out")
 if [ "$num_eos" -ne 1 ]; then
-  echo "Found $eum_sos end of session records instead of 1"
+  echo "Found $num_sos end of session records instead of 1"


same as above, must be $num_eos

arogge · 2020-01-28T13:44:22Z

systemtests/tests/copy-remote-bscan/testrunner

 num_eos=$(grep -c '^End Job Session Record:$' "$tmp/bscan.out")
 if [ "$num_eos" -ne 1 ]; then
-  echo "Found $eum_sos end of session records instead of 1"
+  echo "Found $num_sos end of session records instead of 1"


same as above, must be $num_eos

arogge · 2020-01-28T13:44:56Z

systemtests/tests/multiplied-device-test/testrunner

-grep "Using Device \"MultiFileStorage0003\"" ${tmp}/log1.out 2>&1 >/dev/null
+grep "Using Device \"MultiFileStorage0001\"" ${tmp}/log1.out >/dev/null 2>&1 &&
+grep "Using Device \"MultiFileStorage0002\"" ${tmp}/log1.out >/dev/null 2>&1 &&
+grep "Using Device \"MultiFileStorage0003\"" ${tmp}/log1.out >/dev/null 2>&1


$tmp should be quoted

arogge · 2020-01-28T13:48:28Z

systemtests/tests/virtualfull-bscan/testrunner

 num_eos=$(grep -c '^End Job Session Record:$' "$tmp/bscan.out")
 if [ "$num_eos" -ne 1 ]; then
-  echo "Found $eum_sos end of session records instead of 1"
+  echo "Found $num_sos end of session records instead of 1"


$num_eos again

arogge requested changes Jan 24, 2020

View reviewed changes

pstorz added 24 commits January 24, 2020 15:01

systemtests: backup-bareos-test: run installed binaries

3e7069d

systemtests: adapt python-fd-percona-xtrabackup to test installed files

3ea9691

bareos-config-lib.sh: remove logging of useless information

bb99eee

cats scripts: psql: suppress hints when doing database operations

40f7e9b

systemtests: cleanup CMakeLists, adapt testrunner to use new variables

b7fa571

systemtests: split off python-pam from bconsole-pam test

64852c2

webui system tests: fix testrunner

0bf34c2

systemtests: python-fd-percona-xtrabackup-test: removed mariadb for now

f901e9c

systemtests: fix test dependencies for catalog and pam tests

b1dd0c5

systemtests: fix detection of BACKEND_DIR

290fd38

systemtests: disable catalog test when gtest is not available

178be66

systemtests: disable python-pam when installed packages are tested

c5f8c1d

all source tree: apply cmake-format to all files

d02de50

systemtests: pam_exec_check.sh: cleanup trailing space

40e6bc4

tests: make catalog.cc build on old compilers

c169918

webui tests: only run one instance of webdriver

b4c0729

webui tests: add --no-sandbox option to enable testing as root

57f165f

systemtests: remove log line from scripts/setup

b59199d

system tests: skip catalog test if gtest not available

4c50d96

webui tests: enable running on installed files from package

ac7e7bc

tests: introduce custom pretest script

a603ed0

system tests: give info about test configuration in pretest

f4482f1

tests: cleanup pretest script

6de22d5

bareos-config-lib.sh.in: fix review problem

f31fb90

pstorz force-pushed the dev/pstorz/master/run-systemtests-on-installed-packages branch from bb088e3 to f31fb90 Compare January 24, 2020 14:04

franku suggested changes Jan 24, 2020

View reviewed changes

pstorz added 2 commits January 26, 2020 10:41

applied clang-format to catalog.cc

a11e3c6

pstorz force-pushed the dev/pstorz/master/run-systemtests-on-installed-packages branch from 28d4137 to f98f608 Compare January 28, 2020 07:57

arogge requested changes Jan 28, 2020

View reviewed changes

systemtests: fixes resulting from review

ce44023

pstorz merged commit ee457bb into master Jan 28, 2020

pstorz deleted the dev/pstorz/master/run-systemtests-on-installed-packages branch January 28, 2020 14:36

	const char* v=(std::getenv(env_var.c_str()));
	const char* v = (std::getenv(env_var.c_str()));

	auto jcr=directordaemon::NewDirectorJcr();
	auto jcr = directordaemon::NewDirectorJcr();

	${scripts}/bareos-config initialize_database_driver
	"${scripts}"/bareos-config initialize_database_driver

	${BAREOS_BCONSOLE_BINARY} -c ${conf} "$@"
	"${BAREOS_BCONSOLE_BINARY}" -c "${conf}" "$@"

	if ! ${BAREOS_BCONSOLE_BINARY} -c "${conf}" -p etc/user1.cred < $tmp/bconcmds >${tmp}/log1.out 2>${tmp}/err1.out; then
	if ! "${BAREOS_BCONSOLE_BINARY}" -c "${conf}" -p etc/user1.cred < $tmp/bconcmds >${tmp}/log1.out 2>${tmp}/err1.out; then

	if ${BAREOS_BCONSOLE_BINARY} -c "${conf}" -p etc/user2.cred < $tmp/bconcmds >${tmp}/log2.out 2>${tmp}/err2.out; then
	if "${BAREOS_BCONSOLE_BINARY}" -c "${conf}" -p etc/user2.cred < $tmp/bconcmds >${tmp}/log2.out 2>${tmp}/err2.out; then



		if ! ${scripts}/bareos-ctl-dir status >/dev/null; then
		if ! ${rscripts}/bareos-ctl-dir status >/dev/null; then

	if ! ${rscripts}/bareos-ctl-dir status >/dev/null; then
	if ! "${rscripts}"/bareos-ctl-dir status >/dev/null; then

Conversation

pstorz commented Jan 24, 2020

Uh oh!

arogge left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

franku left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants