Skip to content

Update build files to work with more modern autotools#358

Merged
SpamapS merged 4 commits intomasterfrom
autoupdates
Nov 19, 2022
Merged

Update build files to work with more modern autotools#358
SpamapS merged 4 commits intomasterfrom
autoupdates

Conversation

@SpamapS
Copy link
Member

@SpamapS SpamapS commented Nov 15, 2022

This also enables a build target on Ubuntu 22.04 with the latest gcc version.

Tested to still work at least as far back as Ubuntu 16.04.

Closes #351

@SpamapS
Copy link
Member Author

SpamapS commented Nov 15, 2022

Note that this is stacked on top of #356 so merging this merges that too.

@SpamapS
Copy link
Member Author

SpamapS commented Nov 16, 2022

gearmand-1.1.19.1-79-g1161d2c.tar.gz

Uploading a tarball produced on my 22.04 machine that seems to work including building with all queue plugins enabled. It would help me if folks tried it on various older platforms.

@esabol
Copy link
Member

esabol commented Nov 16, 2022

Changes to the autotools stuff make me nervous, especially the changes to the time.h and sys/time.h includes, tbh. I'm pleasantly surprised the CI tests on Ubuntu 14.04 pass, and that's reassuring. We should be sure to test on CentOS 7 and Alpine and macOS, I think. I'll try to do my part, but, as I mentioned previously, I'm extremely busy this week at work, so it might not be until this coming weekend.

@SpamapS
Copy link
Member Author

SpamapS commented Nov 17, 2022

Indeed, the autotools stuff is my least favorite thing about gearmand and libgearman.

But I'm pretty confident the CI is simple and straightforward.

There's no rush to get this released, but this weekend would be great. Maybe join the Matrix chat again so we can collab in realtime if we bump into each other.

@SpamapS SpamapS merged commit 4213a11 into master Nov 19, 2022
@esabol esabol deleted the autoupdates branch November 19, 2022 21:03
@esabol
Copy link
Member

esabol commented Nov 19, 2022

I guess you couldn't wait for my testing this weekend, but I'll post my findings anyway.

CentOS 7.9: Configure and build went well. make test failed on the first attempt in t/tokyocabinet with the following:

libtest/client.cc:273: in start() pid(8718) localhost:43654 ping(libtest/client.cc:273: Connection refused), additionally pid: 8822 is alive: true waited: 17 server started. exec: /path/to/src/gearmand-1.1.19.1-79-g1161d2c/libtool --mode=execute /path/to/src/gearmand-1.1.19.1-79-g1161d2c/./gearmand/gearmand --verbose=INFO --log-file=var/log/gearmand.logLhGmCI --pid-file=var/run/gearmand.pidp8p2fC --port=43654 --libtokyocabinet-file=var/tmp/gearman.tcb --queue-type=libtokyocabinet --listen=localhost stderr:

tests/tokyocabinet_test.cc:142: in lp_1054377_TEST() pid(8718) Assertion 'server_startup(servers, "gearmand", first_port, argv)'
tokyocabinet.queue restart.lp:1054377 [ failed ]

That seemed like a transient sort of thing (we'd see similar transient test failures like this in Travis CI a lot), so I tried again. This time I got a disconcerting backtrace in a different test (t/protocol, I think):

*** Error in `/path/to/src/gearmand-1.1.19.1-79-g1161d2c/./gearmand/gearmand': double free or corruption (!prev): 0x000055c928482710 ***
======= Backtrace: =========
/lib64/libc.so.6(+0x81329)[0x7ff89db0e329]
/path/to/src/gearmand-1.1.19.1-79-g1161d2c/./gearmand/gearmand(+0x21f0d)[0x55c9271c8f0d]
/path/to/src/gearmand-1.1.19.1-79-g1161d2c/./gearmand/gearmand(+0x1e1d5)[0x55c9271c51d5]
/path/to/src/gearmand-1.1.19.1-79-g1161d2c/./gearmand/gearmand(+0xd34e)[0x55c9271b434e]
/lib64/libc.so.6(__libc_start_main+0xf5)[0x7ff89daaf555]
/path/to/src/gearmand-1.1.19.1-79-g1161d2c/./gearmand/gearmand(+0xdb92)[0x55c9271b4b92]
======= Memory map: ========
55c9271a7000-55c9271f2000 r-xp 00000000 00:2b 78893113                   /path/to/src/gearmand-1.1.19.1-79-g1161d2c/gearmand/gearmand
55c9273f2000-55c9273f4000 r--p 0004b000 00:2b 78893113                   /path/to/src/gearmand-1.1.19.1-79-g1161d2c/gearmand/gearmand
55c9273f4000-55c9273f5000 rw-p 0004d000 00:2b 78893113                   /path/to/src/gearmand-1.1.19.1-79-g1161d2c/gearmand/gearmand
55c9273f5000-55c9273f7000 rw-p 00000000 00:00 0
55c92846e000-55c92848f000 rw-p 00000000 00:00 0                          [heap]
7ff878000000-7ff878021000 rw-p 00000000 00:00 0
7ff878021000-7ff87c000000 ---p 00000000 00:00 0
7ff880000000-7ff880021000 rw-p 00000000 00:00 0
7ff880021000-7ff884000000 ---p 00000000 00:00 0
7ff884000000-7ff884021000 rw-p 00000000 00:00 0
7ff884021000-7ff888000000 ---p 00000000 00:00 0
7ff888000000-7ff888021000 rw-p 00000000 00:00 0
7ff888021000-7ff88c000000 ---p 00000000 00:00 0
7ff88c000000-7ff88c23b000 rw-p 00000000 00:00 0
7ff88c23b000-7ff890000000 ---p 00000000 00:00 0
7ff892ffe000-7ff892fff000 ---p 00000000 00:00 0
7ff892fff000-7ff8937ff000 rw-p 00000000 00:00 0
7ff8937ff000-7ff893800000 ---p 00000000 00:00 0
7ff893800000-7ff894000000 rw-p 00000000 00:00 0
7ff894000000-7ff894021000 rw-p 00000000 00:00 0
7ff894021000-7ff898000000 ---p 00000000 00:00 0
7ff89873e000-7ff89873f000 ---p 00000000 00:00 0
7ff89873f000-7ff898f3f000 rw-p 00000000 00:00 0
7ff898f3f000-7ff898f40000 ---p 00000000 00:00 0
7ff898f40000-7ff899740000 rw-p 00000000 00:00 0
7ff899740000-7ff899741000 ---p 00000000 00:00 0
7ff899741000-7ff899f41000 rw-p 00000000 00:00 0
7ff899f41000-7ff899f4d000 r-xp 00000000 fd:02 18317520                   /usr/lib64/libnss_files-2.17.so
7ff899f4d000-7ff89a14c000 ---p 0000c000 fd:02 18317520                   /usr/lib64/libnss_files-2.17.so
7ff89a14c000-7ff89a14d000 r--p 0000b000 fd:02 18317520                   /usr/lib64/libnss_files-2.17.so
7ff89a14d000-7ff89a14e000 rw-p 0000c000 fd:02 18317520                   /usr/lib64/libnss_files-2.17.so
7ff89a14e000-7ff89a154000 rw-p 00000000 00:00 0
7ff89a955000-7ff89a957000 r-xp 00000000 fd:02 16783484                   /usr/lib64/libfreebl3.so
7ff89a957000-7ff89ab56000 ---p 00002000 fd:02 16783484                   /usr/lib64/libfreebl3.so
7ff89ab56000-7ff89ab57000 r--p 00001000 fd:02 16783484                   /usr/lib64/libfreebl3.so
7ff89ab57000-7ff89ab58000 rw-p 00002000 fd:02 16783484                   /usr/lib64/libfreebl3.so
7ff89ab58000-7ff89abb8000 r-xp 00000000 fd:02 16783823                   /usr/lib64/libpcre.so.1.2.0
7ff89abb8000-7ff89adb8000 ---p 00060000 fd:02 16783823                   /usr/lib64/libpcre.so.1.2.0
7ff89adb8000-7ff89adb9000 r--p 00060000 fd:02 16783823                   /usr/lib64/libpcre.so.1.2.0
7ff89adb9000-7ff89adba000 rw-p 00061000 fd:02 16783823                   /usr/lib64/libpcre.so.1.2.0
7ff89adba000-7ff89adc2000 r-xp 00000000 fd:02 16960199                   /usr/lib64/libcrypt-2.17.so
7ff89adc2000-7ff89afc1000 ---p 00008000 fd:02 16960199                   /usr/lib64/libcrypt-2.17.so
7ff89afc1000-7ff89afc2000 r--p 00007000 fd:02 16960199                   /usr/lib64/libcrypt-2.17.so
7ff89afc2000-7ff89afc3000 rw-p 00008000 fd:02 16960199                   /usr/lib64/libcrypt-2.17.so
7ff89afc3000-7ff89aff1000 rw-p 00000000 00:00 0
7ff89aff1000-7ff89b015000 r-xp 00000000 fd:02 16783856                   /usr/lib64/libselinux.so.1
7ff89b015000-7ff89b214000 ---p 00024000 fd:02 16783856                   /usr/lib64/libselinux.so.1
7ff89b214000-7ff89b215000 r--p 00023000 fd:02 16783856                   /usr/lib64/libselinux.so.1
7ff89b215000-7ff89b216000 rw-p 00024000 fd:02 16783856                   /usr/lib64/libselinux.so.1
7ff89b216000-7ff89b218000 rw-p 00000000 00:00 0
7ff89b218000-7ff89b252000 r-xp 00000000 fd:02 16784268                   /usr/lib64/libnspr4.so
7ff89b252000-7ff89b452000 ---p 0003a000 fd:02 16784268                   /usr/lib64/libnspr4.so
7ff89b452000-7ff89b453000 r--p 0003a000 fd:02 16784268                   /usr/lib64/libnspr4.so
7ff89b453000-7ff89b455000 rw-p 0003b000 fd:02 16784268                   /usr/lib64/libnspr4.so
7ff89b455000-7ff89b457000 rw-p 00000000 00:00 0
7ff89b457000-7ff89b45b000 r-xp 00000000 fd:02 17262190                   /usr/lib64/libplc4.so
7ff89b45b000-7ff89b65a000 ---p 00004000 fd:02 17262190                   /usr/lib64/libplc4.so
7ff89b65a000-7ff89b65b000 r--p 00003000 fd:02 17262190                   /usr/lib64/libplc4.so
7ff89b65b000-7ff89b65c000 rw-p 00004000 fd:02 17262190                   /usr/lib64/libplc4.so
7ff89b65c000-7ff89b65f000 r-xp 00000000 fd:02 17262212                   /usr/lib64/libplds4.so
7ff89b65f000-7ff89b85e000 ---p 00003000 fd:02 17262212                   /usr/lib64/libplds4.so
7ff89b85e000-7ff89b85f000 r--p 00002000 fd:02 17262212                   /usr/lib64/libplds4.so
7ff89b85f000-7ff89b860000 rw-p 00003000 fd:02 17262212                   /usr/lib64/libplds4.so
7ff89b860000-7ff89b889000 r-xp 00000000 fd:02 17344012                   /usr/lib64/libnssutil3.so
7ff89b889000-7ff89ba88000 ---p 00029000 fd:02 17344012                   /usr/lib64/libnssutil3.so
7ff89ba88000-7ff89ba8f000 r--p 00028000 fd:02 17344012                   /usr/lib64/libnssutil3.so
7ff89ba8f000-7ff89ba90000 rw-p 0002f000 fd:02 17344012                   /usr/lib64/libnssutil3.so
7ff89ba90000-7ff89bbc1000 r-xp 00000000 fd:02 17557976                   /usr/lib64/libnss3.so
7ff89bbc1000-7ff89bdc1000 ---p 00131000 fd:02 17557976                   /usr/lib64/libnss3.so
7ff89bdc1000-7ff89bdc6000 r--p 00131000 fd:02 17557976                   /usr/lib64/libnss3.so
7ff89bdc6000-7ff89bdc9000 rw-p 00136000 fd:02 17557976                   /usr/lib64/libnss3.so
7ff89bdc9000-7ff89bdca000 rw-p 00000000 00:00 0
7ff89bdca000-7ff89bdef000 r-xp 00000000 fd:02 17557977                   /usr/lib64/libsmime3.so

I didn't see anything in the test-suite.log file for that, so that's why I'm not sure which test triggered it.

A third attempt at make test passed.

I'm not sure if this is any cause for concern. I think we could blame it on CentOS 7.9 or (more likely) gcc version 4.8.5 which ships with CentOS 7.9. Should I open this as an issue?

@esabol
Copy link
Member

esabol commented Nov 19, 2022

Built and tested the 1.1.20 release tarball on CentOS 7.9, and tested fine the first, second, and third tries. Guess it was a fluke. :-/

macOS 1.1.20 release tarball build failed with configure: error: Unable to find libevent. Hmmm, there's no DarwinPort for libevent (although there is one for gearmand, so not sure how that's done without libevent), and I don't have Homebrew installed on this Mac, so I'm punting on this right now....

@esabol
Copy link
Member

esabol commented Nov 19, 2022

Built and tested 1.1.20 release tarball on Alpine 3.13. The build went OK, except for this message:

  CXX      libgearman-server/plugins/protocol/http/libgearman_server_la-protocol.lo
libgearman-server/plugins/protocol/http/protocol.cc: In member function 'virtual size_t HTTPtext::pack(const gearmand_packet_st*, gearman_server_con_st*, void*, size_t, gearmand_error_t&)':
libgearman-server/plugins/protocol/http/protocol.cc:153:25: warning: this statement may fall through [-Wimplicit-fallthrough=]
  153 |       gearmand_log_debug(GEARMAN_DEFAULT_LOG_PARAM,
      |       ~~~~~~~~~~~~~~~~~~^~~~~~~~~~~~~~~~~~~~~~~~~~~
  154 |                          "Bad packet command: gearmand_command_t:%s",
      |                          ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
  155 |                          gearman_strcommand(packet->command));
      |                          ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
libgearman-server/plugins/protocol/http/protocol.cc:157:5: note: here
  157 |     case GEARMAN_COMMAND_WORK_FAIL:
      |     ^~~~

I'll look into cleaning up that warning.

All tests passed.

I used the following Dockerfile, a variation on the one in the repo except it downloads the tarball instead of using the git repo:

ARG version=3.13
FROM alpine:${version}

ARG version
ARG GEARMAN_TARBALL=https://github.com/gearman/gearmand/releases/download/1.1.20/gearmand-1.1.20.tar.gz

LABEL description="Gearman Job Server Image (Alpine ${version})"
LABEL maintainer="Gearmand Developers https://github.com/gearman/gearmand"
LABEL version="${version}"

# Install packages
RUN apk add --no-cache \
        make \
        gcc \
        g++ \
        autoconf \
        automake \
        m4 \
        libtool \
        bash \
        file \
        wget \
        tar \
        py3-sphinx \
        util-linux-dev \
        libuuid \
        libevent-dev \
        gperf \
        boost-dev \
        openssl-dev

# Switch to a non-root user
RUN adduser --disabled-password --shell /bin/bash gearman
USER gearman

# Clone the GitHub repository master branch
RUN cd /tmp \
 && wget --quiet ${GEARMAN_TARBALL} \
 && tar xzvf gearmand-*.tar.gz \
 && rm -f gearmand-*.tar.gz \
 && mv gearmand-* gearmand

# Configure, make, and make test
WORKDIR /tmp/gearmand
RUN ./configure --enable-ssl 2>&1 | tee ./configure.log
RUN make 2>&1 | tee ./build.log
RUN make test 2>&1 | tee ./test.log

@esabol
Copy link
Member

esabol commented Nov 19, 2022

Ditto for Alpine 3.17, which is the latest.

I wish we had Dockerfiles for Debian....

@SpamapS
Copy link
Member Author

SpamapS commented Nov 20, 2022 via email

@esabol
Copy link
Member

esabol commented Nov 21, 2022

I was able to get libevent (and libboost) installed on my Mac in /opt/local/ and was finally able to get the 1.1.20 release tarball to compile like so:

./configure LDFLAGS="-L/opt/local/lib" CFLAGS="-I/opt/local/include" LIBS="-L/opt/local/lib" --with-boost=/opt/local CPLUS_INCLUDE_PATH="/opt/local/include"
make CPLUS_INCLUDE_PATH="/opt/local/include"

Some of those configure arguments might be overkill. Regardless, build was clean! But make test fails repeatedly in t/unittest. All other tests pass. The following is printed to stderr:

./build-aux/test-driver: line 112: 64812 Killed: 9               "$@" >> "$log_file" 2>&1

Nothing amiss in test-suite.log. It shows all the tests in t/unittest as either passed OK or skipped. Not sure what to make of that. Any ideas? My MacOS is pretty out of date, so that could be the reason, I suppose.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

bootstrap.sh -a fails on Ubuntu 22.04

2 participants