Skip to content

Conversation

@jimklimov
Copy link
Member

@jimklimov jimklimov commented Jan 23, 2024

Bump the copy of 42ity fork contents to minimize difference with the master (as they got to differ considerably over the past 2 years of main trunk's evolution). Hopefully this would ease eventual import of features from the fork and facilitate comparisons of intentional differences.

Also letting NUT CI chew it in many more scenarios than I can run locally :)

After all, it is very possible that difftool/mergetool duplicated or ate away some lines (fixed some cases like that which did break local builds soon after initial conflict resolution).

@jimklimov jimklimov added DMF NUT Data/Dynamic Mapping File/Format/Functionality feature impacts-release-2.8.1 Issues reported against NUT release 2.8.1 (maybe vanilla or with minor packaging tweaks) labels Jan 23, 2024
@aquette
Copy link
Member

aquette commented Jan 23, 2024

Side comment: as you remember, that fork is not used anymore nowadays, with the cons of missing dmf support, and the pros of being 2x aligned upstream (using official Debian 12)

@jimklimov
Copy link
Member Author

Yes, but a lot of treasures lurk there waiting to be upstreamed (and feed back to benefit FTY from the other side). I fear the day when I'd have to review the 50k lines of changes though, and extract that into specific features (DMF, nutconf, vanilla packages, etc.) and more so if trying to keep the 3K items of commit trail :D

@jimklimov
Copy link
Member Author

jimklimov commented Jan 24, 2024

@aquette : FWIW, I've hopefully brought nutconf here up to speed, at least in regard to building without warnings and passing the self-test (locally so far, letting NUT CI loose on it now). It is probably behind the modern code in terms of keyword support, but that's iterable if someone were so inclined :)

Also, finally had a sort of bright idea how to avoid those ugly ifdef WITH_...FUNC in all the SNMP mapping and lookup tables, to add explicit NULL initializers for optional fields: made a series of macros to apply these defaults (or set function pointers in a few cases). So the *-mib.c files are now much more readable and somewhat smaller, similar to master-branch code. The next step is to apply similar macros on master to minimize the diff with DMF-capable codebase :)

@jimklimov jimklimov added the nutconf NUT configuration library and tool label Jan 24, 2024
@jimklimov
Copy link
Member Author

jimklimov commented Jan 24, 2024

Hm, while assigning labels and looking for historic evidence, found the https://github.com/networkupstools/nut/tree/libconf branch among the dozens of loose threads right in this repo. Didn't go that deep into the cellar before :D

At least, this gives me a chance to cleanly PR the feature separately of DMF etc. :)

Cross-referencing to 42ity#161, #6, #18 for completeness :)

@jimklimov jimklimov marked this pull request as draft January 24, 2024 19:55
@jimklimov jimklimov force-pushed the FTY-remerge-20240123 branch from 82d3287 to 6233365 Compare January 25, 2024 02:07
@jimklimov jimklimov marked this pull request as ready for review January 25, 2024 02:08
@jimklimov
Copy link
Member Author

Hm, C++ obsoletions strike again in clang :\

@jimklimov jimklimov force-pushed the FTY-remerge-20240123 branch from 6233365 to 6168ce6 Compare January 25, 2024 16:49
@jimklimov jimklimov marked this pull request as draft January 29, 2024 13:45
@jimklimov jimklimov marked this pull request as ready for review February 16, 2024 00:28
@jimklimov jimklimov changed the title FTY remerge from master as of 2024-01-23 FTY remerge from master as of 2024-02-16 Feb 16, 2024
@jimklimov
Copy link
Member Author

PR updated with master that separately included the IPSS Unix derived installer #2288, and the updated nutconf effort #2290 among others.

@jimklimov jimklimov force-pushed the FTY-remerge-20240123 branch 3 times, most recently from 258274d to 7d0b025 Compare February 17, 2024 21:29
…s first

Signed-off-by: Jim Klimov <jimklimov+nut@gmail.com>
…ep-src-docs, avoid sub-makes where we can

Signed-off-by: Jim Klimov <jimklimov+nut@gmail.com>
…pared by "git difftool" for comparisons

Signed-off-by: Jim Klimov <jimklimov+nut@gmail.com>
jimklimov and others added 4 commits March 2, 2024 16:17
Signed-off-by: Jim Klimov <jimklimov+nut@gmail.com>
… and ExecStartPost to monitor that it was applied, from FTY branch

Originates from FTY commits:

* 062d186 (Jim Klimov    2018-01-16 13:11:08 +0100)
* 0e28cc8 (Jim Klimov    2018-01-16 13:37:39 +0100)
* e9a6710 (Jim Klimov    2018-01-16 13:41:52 +0100)

Signed-off-by: Jim Klimov <jimklimov+nut@gmail.com>
jimklimov and others added 4 commits March 2, 2024 17:47
…artLimitInterval for broader compatibility

Should help avoid this warning and unachieved functionality on systemd-229 and older:
* systemd[1]: [/lib/systemd/system/nut-driver-enumerator-daemon-activator.service:30] Unknown lvalue 'StartLimitIntervalSec' in section 'Service'
…to FTY-remerge-20240123

Signed-off-by: Jim Klimov <jimklimov+nut@gmail.com>
…om-FTY

Bump NUT `master` branch from FTY
Signed-off-by: Jim Klimov <jimklimov+nut@gmail.com>
@jimklimov
Copy link
Member Author

Regarding several builds of libssl/libcrypto conflicting on FreeBSD builds...

From warnings analysis summary at https://ci.networkupstools.org/job/nut/job/nut/job/PR-2275/lastBuild/aggregated-analysis/ :

libssl.so.111, needed by /usr/local/lib/libneon.so, may conflict with libssl.so.11
libcrypto.so.111, needed by /usr/local/lib/libneon.so, may conflict with libcrypto.so.11
libcrypto.so.111, needed by /usr/local/lib/libnetsnmp.so, may conflict with libcrypto.so.11

Files are indeed differently named and sized:

$ ls -la /usr/lib32/libssl.so.111 /usr/local/lib/libssl.so.11 /usr/local/lib/libssl.so /usr/lib/libssl.so.111 /usr/lib/libssl.so /usr/lib32/libssl.so
lrwxr-xr-x  1 root  wheel      13 Oct 23  2020 /usr/lib/libssl.so -> libssl.so.111
-r--r--r--  1 root  wheel  609520 Oct 23  2020 /usr/lib/libssl.so.111
lrwxr-xr-x  1 root  wheel      13 Oct 23  2020 /usr/lib32/libssl.so -> libssl.so.111
-r--r--r--  1 root  wheel  512604 Oct 23  2020 /usr/lib32/libssl.so.111
lrwxr-xr-x  1 root  wheel      12 Jul  2  2021 /usr/local/lib/libssl.so -> libssl.so.11
-rw-r--r--  1 root  wheel  603904 Jul  2  2021 /usr/local/lib/libssl.so.11

$ file /usr/lib/libssl.so.111 /usr/lib32/libssl.so.111 /usr/local/lib/libssl.so.11
/usr/lib/libssl.so.111:      ELF 64-bit LSB shared object, x86-64, version 1 (FreeBSD), dynamically linked, stripped
/usr/lib32/libssl.so.111:    ELF 32-bit LSB shared object, Intel 80386, version 1 (FreeBSD), dynamically linked, stripped
/usr/local/lib/libssl.so.11: ELF 64-bit LSB shared object, x86-64, version 1 (FreeBSD), dynamically linked, stripped
  • Why does it not pop out in standard NUT builds (probably DMF recipes hardcode something as a shortcut? or just got a different combo of libs needed together by some binary?)
  • Is it possible to portably configure detection of library search paths to accommodate the whole chain of used libs or poke exact preferred filenames (e.g. to do use /usr/lib/libssl.so.111 for /usr/local/lib/libneon.so and ignore the nearby /usr/local/lib/libssl.so.11)
  • Where did /usr/local/lib/libssl.so.11 come from? Why is libneon/libsnmp not in /usr/lib if it wants standard libraries? Is it some curiosity of the build agent and not a systemic distro problem - maybe the core fix is in some consistent use of packages vs. local builds, not in NUT recipes imagining workarounds for sysadmin errors?

@jimklimov
Copy link
Member Author

jimklimov commented Mar 3, 2024

So... it seems that the openssl binary is the only consumer of that library, and the symlink is not needed:

# { find /usr/local . -type f -perm +111 | while read F ; do  ldd $F | grep /usr/local/lib/libssl.so && echo "^^^ $F" ; done ; }  2>/dev/null

        libssl.so.11 => /usr/local/lib/libssl.so.11 (0x8002e1000)
^^^ /usr/local/bin/openssl

However the "local" one is the package-delivered one:

# pkg which /usr/{local/,}lib*/libssl.so*
/usr/local/lib/libssl.so was installed by package openssl-1.1.1k_1,1
/usr/local/lib/libssl.so.11 was installed by package openssl-1.1.1k_1,1
/usr/lib/libssl.so was not found in the database
/usr/lib/libssl.so.111 was not found in the database
/usr/lib32/libssl.so was not found in the database
/usr/lib32/libssl.so.111 was not found in the database

Same as libneon and libnetsnmp, which are packaged and only "local":

# pkg which /usr/{local/,}lib*/libneon.so*
/usr/local/lib/libneon.so was installed by package neon-0.31.2
/usr/local/lib/libneon.so.27 was installed by package neon-0.31.2
/usr/local/lib/libneon.so.27.4.2 was installed by package neon-0.31.2
/usr/lib*/libneon.so* was not found in the database

# pkg which /usr/{local/,}lib*/libnetsnmp.so*
/usr/local/lib/libnetsnmp.so was installed by package net-snmp-5.9_3,1
/usr/local/lib/libnetsnmp.so.40 was installed by package net-snmp-5.9_3,1
/usr/local/lib/libnetsnmp.so.40.0.0 was installed by package net-snmp-5.9_3,1
/usr/lib*/libnetsnmp.so* was not found in the database

...and use system-provided libssl and libcrypto (note the latter is in /lib):

# ldd /usr/local/lib/libnetsnmp.so /usr/local/lib/libneon.so | grep -E 'libssl|libcrypto'
        libcrypto.so.111 => /lib/libcrypto.so.111 (0x800e00000)
        libssl.so.111 => /usr/lib/libssl.so.111 (0x8006b1000)
        libcrypto.so.111 => /lib/libcrypto.so.111 (0x800e82000)

NUT-wise, the warnings seem (parallel build) to appear in these linkages (only for this branch but not for master):

--- netxml-ups ---
/usr/local/bin/ld: warning: libssl.so.111, needed by /usr/local/lib/libneon.so, may conflict with libssl.so.11
/usr/local/bin/ld: warning: libcrypto.so.111, needed by /usr/local/lib/libneon.so, may conflict with libcrypto.so.11
--- snmp-ups ---
/usr/local/bin/ld: warning: libcrypto.so.111, needed by /usr/local/lib/libnetsnmp.so, may conflict with libcrypto.so.11
--- snmp-ups-dmf ---
/usr/local/bin/ld: warning: libcrypto.so.111, needed by /usr/local/lib/libnetsnmp.so, may conflict with libcrypto.so.11
/usr/local/bin/ld: warning: libssl.so.111, needed by /usr/local/lib/libneon.so, may conflict with libssl.so.11
--- nut-scanner ---
/usr/local/bin/ld: warning: libcrypto.so.111, needed by /usr/local/lib/libnetsnmp.so, may conflict with libcrypto.so.11
--- nutconf ---
/bin/sh ../../libtool  --tag=CXX    --mode=link g++ -I../../include  -I../../include -DWITH_NUTSCANN
ER -I../../tools/nut-scanner  -isystem /usr/local/include -g -O2 -Wno-reserved-identifier -fdiagnost
ics-color=always -Wno-unknown-warning-option -std=gnu++11 -Wno-system-headers -Wall -Wextra -Werror
-O0 -g3 -gdwarf-2   -o nutconf nutconf-nutconf-cli.o ../../common/libcommon.la  ../../common/libnutc
onf.la  ../../tools/nut-scanner/libnutscan.la -lpthread
libtool: link: g++ -I../../include -I../../include -DWITH_NUTSCANNER -I../../tools/nut-scanner -isys
tem /usr/local/include -g -O2 -Wno-reserved-identifier -fdiagnostics-color=always -Wno-unknown-warni
ng-option -std=gnu++11 -Wno-system-headers -Wall -Wextra -Werror -O0 -g3 -gdwarf-2 -o .libs/nutconf
nutconf-nutconf-cli.o  ../../common/.libs/libcommon.a ../../common/.libs/libnutconf.a ../../tools/nu
t-scanner/.libs/libnutscan.so -L/usr/local/lib -L/usr/lib -lrt -lutil -lnetsnmp -lxml2 -lz -llzma /u
sr/local/lib/libltdl.so -llua-5.1 -lm -lssl -lcrypto -lpthread -pthread -Wl,-rpath -Wl,/usr/local/up
s/lib -Wl,-rpath -Wl,/usr/local/lib
/usr/local/bin/ld: warning: libcrypto.so.111, needed by /usr/local/lib/libnetsnmp.so, may conflict with libcrypto.so.11

The command above does include -L/usr/local/lib -L/usr/lib in this order...

In both branch builds, libnetsnmp and libneon details were found via pkg-config, so both claim e.g. checking for Net-SNMP libs... -L/usr/local/lib -lnetsnmp per *.pc file instructions:

# cat /usr/local/libdata/pkgconfig/netsnmp.pc
prefix=/usr/local
exec_prefix=${prefix}
includedir=${prefix}/include
libdir=${exec_prefix}/lib

Name: netsnmp (Net-SNMP)
Description: SNMP (Simple Network Management Protocol) daemon and applications.
URL: http://www.net-snmp.org
Version: 5.9
Cflags: -I${includedir}
Libs: -L${libdir} -lnetsnmp
Libs.private: -L/usr/lib -lm -lkvm -ldevstat -L/lib -L/usr/local/lib    -lcrypto -lelf -lpkg -lssp_nonshared -lregex   -Wl,-R/usr/local/lib/perl5/5.32/mach/CORE

One notable difference of config.log files of the two builds is that DMF version has LIBXML2_LIBS='-L/usr/local/lib -lxml2 -lz -L/usr/lib -llzma -L/usr/lib -lm' which I guess upsets the search path, if it goes before SNMP/NEON macros?..

…t clauses, to ease maintenance and navigation

Signed-off-by: Jim Klimov <jimklimov+nut@gmail.com>
…p rules

Signed-off-by: Jim Klimov <jimklimov+nut@gmail.com>
@jimklimov
Copy link
Member Author

jimklimov commented Mar 3, 2024

Tested, libxml2 was not the culprit (makes sense, little use for crypto there).

Found that DMF branch also includes this sort of difference:

+
+if WITH_SSL
+  snmp_ups_CFLAGS += $(LIBSSL_CFLAGS)
+  snmp_ups_LDADD += $(LIBSSL_LIBS)
+
+  snmp_ups_dmf_CFLAGS += $(LIBSSL_CFLAGS)
+  snmp_ups_dmf_LDADD += $(LIBSSL_LIBS)
+endif

 # NEON XML/HTTP
 netxml_ups_SOURCES = netxml-ups.c mge-xml.c
 netxml_ups_LDADD = $(LDADD_DRIVERS) $(LIBNEON_LIBS)
+netxml_ups_CFLAGS = $(AM_CFLAGS)
+
+if WITH_SSL
+  netxml_ups_CFLAGS += $(LIBSSL_CFLAGS)
+  netxml_ups_LDADD += $(LIBSSL_LIBS)
+endif

and the link commands in build logs reflect that (snmp-ups build):

STD: ... ../common/libparseconf.la -L/usr/local/lib -lnetsnmp -lm -lpthread
DMF: ... ../common/libparseconf.la -L/usr/local/lib -lnetsnmp -lm  -L/usr/local/lib -lssl -lcrypto -lpthread

(they were built into nut-scanner and inherited into nutconf earlier, in master-branch as well; the DMF difference is about also pulling the "conflicting" libneon and libnetsnmp builds into orbit).

The openssl-1.1.1 libraries are honestly found via pkg-config, so NUT recipes probably should not intrude too rudely with their opinion. Something to solve on the build agent (pkg remove openssl helped; it was added per common docs/config-prereqs.txt suggestions and nothing else depended on it; the system-provided OpenSSL was found and used afterwards, claiming no conflicts). Maybe eventually in distro, if it is not a one-off problem with this somewhat older worker at FreeBSD 12... Just in case, CCing @gdt :)

checking for OpenSSL version via pkg-config... 1.1.1k found
checking for OpenSSL cflags... -I/usr/local/include
checking for OpenSSL ldflags... -L/usr/local/lib -lssl -lcrypto
checking for openssl/ssl.h... yes

… "pkg add openssl" in fact [networkupstools#2275]

Signed-off-by: Jim Klimov <jimklimov+nut@gmail.com>
… "pkg add openssl" in fact [networkupstools#2275]

Signed-off-by: Jim Klimov <jimklimov+nut@gmail.com>
…fy the unit description (it does not strictly restart the daemon, can reload if running too)

Signed-off-by: Jim Klimov <jimklimov+nut@gmail.com>
…om-FTY-systemd

Bump NUT `master` branch from FTY: systemd integration
Signed-off-by: Jim Klimov <jimklimov+nut@gmail.com>
…s to running the ./configure script

Signed-off-by: Jim Klimov <jimklimov+nut@gmail.com>
Signed-off-by: Jim Klimov <jimklimov+nut@gmail.com>
@jimklimov jimklimov merged commit 839287b into networkupstools:FTY Mar 4, 2024
@jimklimov jimklimov deleted the FTY-remerge-20240123 branch March 4, 2024 20:21
jimklimov added a commit to jimklimov/nut that referenced this pull request Jul 24, 2024
…lkp_default(), snmp_info_sentinel and info_lkp_sentinel macros to simplify the codebase [networkupstools#2275]

Signed-off-by: Jim Klimov <jimklimov+nut@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

DMF NUT Data/Dynamic Mapping File/Format/Functionality feature impacts-release-2.8.1 Issues reported against NUT release 2.8.1 (maybe vanilla or with minor packaging tweaks) nut-scanner nutconf NUT configuration library and tool

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants