Skip to content

Conversation

@jimklimov
Copy link
Member

Also test added NUT CI farm worker with NetBSD

CC @gdt

…l for GitHub to work

Signed-off-by: Jim Klimov <jimklimov+nut@gmail.com>
…AX+1 to allow for ending NUL char [networkupstools#823]

Signed-off-by: Jim Klimov <jimklimov+nut@gmail.com>
… to unsigned char when using as array subscript [networkupstools#823]

This may be an implicit array, such as tolower() implementation
as a ctype(3) macro rather than function.

Signed-off-by: Jim Klimov <jimklimov+nut@gmail.com>
…profile edits

Signed-off-by: Jim Klimov <jimklimov+nut@gmail.com>
@jimklimov jimklimov added documentation refactor/fightwarn PR or issue proposal to improve code maintainability without functional changes, or to fix warnings NetBSD portability We want NUT to build and run everywhere possible labels Mar 31, 2025
@jimklimov jimklimov added this to the 2.8.3 milestone Mar 31, 2025
@jimklimov jimklimov added the C-str Issues and PRs about C/C++ methods, headers and data types dealing with strings and memory blocks label Mar 31, 2025
@gdt
Copy link
Contributor

gdt commented Mar 31, 2025

More or less read the code and looks good.

A nit is that the unigned char is necessary not because it might be an array but because POSIX says it's UB to pass a value that is other than EOF and those that are legal for unsigned char. (It's implicit that implementations may optimize and are free to produce UB when the spec says UB.) Not a big deal, but so many people fail to understand the spec/UB issue.

@jimklimov
Copy link
Member Author

Why is NetBSD's shell sort program so broken? :)

Usual alphanumeric sort: decent

-bash-5.1$ printf '5\n55\n8\n4\n1\nqwe\nabc\nabc3\nabc1\na2\n' | sort
1
4
5
55
8
a2
abc
abc1
abc3
qwe

With numeric argument: broken - pure numeric strings are sorted as numbers go (e.g. 55 after 8), but other strings just dumped in same order as they were, at start of output (numbers after letters ignored and not sorted either):

$ printf '5\n55\n8\n4\n1\nqwe\nabc\nabc3\nabc1\na2\n' | sort  -n
qwe
abc
abc3
abc1
a2
1
4
5
8
55
  • Same on all other systems I saw so far:
$ printf '5\n55\n8\n4\n1\nqwe\nabc\nabc3\nabc1\na2\n' | sort  -n
a2
abc
abc1
abc3
qwe
1
4
5
8
55

@AppVeyorBot
Copy link

@gdt
Copy link
Contributor

gdt commented Mar 31, 2025

Why is NetBSD's shell sort program so broken? :)

You didn't show that it was broken, just not what you expected. As I read POSIX

https://pubs.opengroup.org/onlinepubs/9799919799/utilities/sort.html

It's quite clear "restrict to initial numeric" and therefore all the non-numeric are empty/0 and thus can be sorted arbitrarily while complying with POSIX.

@jimklimov
Copy link
Member Author

I'd invoke principle of least surprise: output of a sort program should be reasonably sorted :)

I agree pedantically it is not incorrect, e.g. pure-numeric sort is not defined for non-numeric tokens, so any order of those including original or random is not-wrong. But it is also not a useful result. Like in those fun tales about mathematicians' replies.

Do you know OTOH a POSIX CLI option combo to sort alphanumerically and number-aware for punctuation/whitespace surrounded numbers with leading parts of the strings being equal, at the same time?

@gdt
Copy link
Contributor

gdt commented Mar 31, 2025

I'd invoke principle of least surprise: output of a sort program should be reasonably sorted :)

I agree pedantically it is not incorrect, e.g. pure-numeric sort is not defined for non-numeric tokens, so any order of those including original or random is not-wrong. But it is also not a useful result. Like in those fun tales about mathematicians' replies.

Do you know OTOH a POSIX CLI option combo to sort alphanumerically and number-aware for punctuation/whitespace surrounded numbers with leading parts of the strings being equal, at the same time?

I see your point, but you're basically arguing that the spec should have said more than it does, and surely they argued about it (that's what POSIX committees do) and it says what it says. The other side of the coin is that it takes CPU time to do things that aren't required, and the view that there is merit in the least-cost conforming output.

You can use -k, and specify multiple keys; the later one only matters if the first compare equal. This sorts the non-numeric keys, and they end up as a group sorted as 0. Should get the same answer on all complying systems. "punctuation" is hard as numeric is defined to handle -, radix, thousands separator. Unfortunately sort's design expects "fields" with a separator (space, :, |) and within each field to be sorted either charset-wise or numeric, more or less.

$ printf '5\n55\n8\n4\n1\nqwe\nabc\nabc3\nabc1\na2\n' | sort -k1n -k1
a2
abc
abc1
abc3
qwe
1
4
5
8
55

@jimklimov
Copy link
Member Author

Valid points. And your example is intriguing. I figured expression of complex desires would go through key specs (how else), but did not realize you can specify the same key twice - always did them for different numbers...

"Live for a century, learn for a century!" indeed :)

jimklimov added a commit to jimklimov/nut that referenced this pull request Apr 1, 2025
…numeric+alpha sort [networkupstools#2870]

Signed-off-by: Jim Klimov <jimklimov+nut@gmail.com>
@gdt
Copy link
Contributor

gdt commented Apr 1, 2025

I had never thought about specifying a key twice, but the spec clearly doesn't say you can't, and sure enough it worked.

…numeric+alpha sort [networkupstools#2870]

Thanks to Greg Troxel @gdt for the hint and fruitful discussion.

Signed-off-by: Jim Klimov <jimklimov+nut@gmail.com>
…c and its pkgin tool

Signed-off-by: Jim Klimov <jimklimov+nut@gmail.com>
"make" package is absent, `/usr/bin/make` appears from something
I could not track down; however there is a separate but similar
"bmake" as `/usr/pkg/bin/bmake` that can be installed. Probably
a difference of system vs. userland tooling variants.

Signed-off-by: Jim Klimov <jimklimov+nut@gmail.com>
…endencies for NUT-Monitor

Signed-off-by: Jim Klimov <jimklimov+nut@gmail.com>
@jimklimov
Copy link
Member Author

jimklimov commented Apr 1, 2025

One more issue cropped up, similar to #2782 seen in OmniOS builds: the pkgsrc libraries are located under /usr/pkg/lib which is not in the compilers'/linker's default search path.

However their pkg-config metadata does not suggest -R options to link them, e.g.:

:; cat /usr/pkg/lib/pkgconfig/libssl.pc
prefix=/usr/pkg
exec_prefix=${prefix}
libdir=${exec_prefix}/lib
includedir=${prefix}/include

Name: OpenSSL-libssl
Description: Secure Sockets Layer and cryptography libraries
Version: 3.3.2
Requires.private: libcrypto
Libs: -L${libdir} -lssl
Cflags: -I${includedir}

As a result, during NUT build the toolkits see them (thanks to -L) and link the binaries, but at run-time they fail to start with messages like:

upsd: Shared object "libssl.so.3" not found

Or, to be fair, SOME libraries do report these bits and do get found "out of the box":

:; cat /usr/pkg/lib/pkgconfig/gdlib.pc
prefix=/usr/pkg
exec_prefix=${prefix}
libdir=${exec_prefix}/lib
includedir=${prefix}/include

Name: gd
Description: GD graphics library
Version: 2.3.3
Requires:
Requires.private:   zlib  libpng  freetype2 >= 9.8.3  raqm  fontconfig  libjpeg  libtiff-4  libwebp >= 0.2.0
Cflags: -I${includedir}
Libs.private:  -L/usr/pkg/lib  -limagequant -fopenmp
Libs: -Wl,-R${libdir} -L${libdir} -lgd

In OmniOS this was accepted as a distro error, maybe was fixed since.

As a hot-fix, hacking LDFLAGS="-R/usr/pkg/lib" to NUT configure runs helps, but is not ideal.

…workupstools#2870]

Just renamed the private enum values.
Earlier code complained with NetBSD clang-18:

	mge-xml.c:76:2: error: identifier '_UNEXPECTED' is reserved because it starts with '_' followed by a capital letter [-Werror,-Wreserved-identifier]
	   76 |         _UNEXPECTED,
	      |         ^
	mge-xml.c:77:2: error: identifier '_PARSEERROR' is reserved because it starts with '_' followed by a capital letter [-Werror,-Wreserved-identifier]
	   77 |         _PARSEERROR,
	      |         ^
	2 errors generated.

Signed-off-by: Jim Klimov <jimklimov+nut@gmail.com>
Signed-off-by: Jim Klimov <jimklimov+nut@gmail.com>
…etBSD pkgsrc dependencies to be found out of the box

Signed-off-by: Jim Klimov <jimklimov+nut@gmail.com>
…r now [networkupstools#2870]

Signed-off-by: Jim Klimov <jimklimov+nut@gmail.com>
…etBSD libltdl dependency to be found out of the box

Signed-off-by: Jim Klimov <jimklimov+nut@gmail.com>
Signed-off-by: Jim Klimov <jimklimov+nut@gmail.com>
…elp NUT at the moment

Signed-off-by: Jim Klimov <jimklimov+nut@gmail.com>
@jimklimov
Copy link
Member Author

jimklimov commented Apr 1, 2025

Despite having the packages installed, NUT failed to make use of:

  • libltdl => no nut-scanner : lacks a pkgconfig file so is not found in /usr/pkg/lib (works when libs/cflags options are added explicitly though)
  • net-snmp : net-snmp/net-snmp-config.h: No such file or directory (which package can that be in? find / -name '*snmp*.h' indeed finds nothing here)
    • UPDATE: Ach, perhaps my bad - it refused to install
# pkgin install net-snmp
calculating dependencies...done.

2 packages to install:
  net-snmp-5.9.4nb1 osabi-NetBSD-9.0

0 to remove, 0 to refresh, 0 to upgrade, 2 to install
0B to download, 14M of additional disk space will be used

proceed ? [Y/n] y
[1/2] installing osabi-NetBSD-9.0...
The Operating System version (9.2) does not match 9.0
To force installation of this package, add CHECK_OSABI=no to pkg_install.conf
[2/2] installing net-snmp-5.9.4nb1...
The Operating System version (9.2) does not match 9.0
To force installation of this package, add CHECK_OSABI=no to pkg_install.conf
pkg_install warnings: 0, errors: 3
pkg_install error log can be found in /var/db/pkgin/pkg_install-err.log
reading local summary...
processing local summary...
  • some more may have appeared as detected after I added blanket LDFLAGS=-R... per above

@AppVeyorBot
Copy link

…t also net-snmp, packages need CHECK_OSABI=no in /etc/pkg_install.conf [networkupstools#2870]

Signed-off-by: Jim Klimov <jimklimov+nut@gmail.com>
Signed-off-by: Jim Klimov <jimklimov+nut@gmail.com>
@gdt
Copy link
Contributor

gdt commented Apr 1, 2025

I agree that a pkg-config file with -L and not -R is a bug and should be patched in pkgsrc (and probably fixed upstream, but many upstreams take an "of course you are using Linux and installing in /usr" attitude).

As for things not having pkg-config, it is historically normal to add CPPFLAGS/LDFLAGS to a program build to tell it where dependencies are. Then, various programs grew foo-config which you could call with --cflags etc and then those were abstracted into pkg-config which has a standard interface and data files. But we have not yet arrived at "there is a rule that every program must have pkg-config and if not it's defective".

Overall, I would say that when building a program where dependencies are in some prefix, one has to pass CPPFLAGS/LDFLAGS, and it's a special case on most GNU/Linux that everything is in /usr. When pkgsrc builds programs, it does pass CPPFLAGS/LDFLAGS. This is almost certainly why the failure to have -R in openssl has not been noticed and fixed.

@AppVeyorBot
Copy link

@jimklimov
Copy link
Member Author

jimklimov commented Apr 1, 2025

Some hiccup in tests against Mozilla NSS - does not seem to honour the -R in linker flags so does not find libssl3.so during test case start.

UPDATE: Must be going blind. It is not a problem of linker -R as such, it is the dynamic SO loading in nut-scanner specifically, and NSS delivering into separate dirs on many platforms:

================================
Tue Apr  1 20:51:02 UTC 2025 [INFO] [testcase_sandbox_nutscanner_list] Call libupsclient test suite: nut-scanner on localhost:34981
Tue Apr  1 20:51:02 UTC 2025 [INFO] [testcase_sandbox_nutscanner_list] Preparing LD_LIBRARY_PATH='/var/shm/jenkins-nutci-netbsd-92-amd64/nut_nut_PR-2870@2/obj/clients:/var/shm/jenkins-nutci-netbsd-92-amd64/nut_nut_PR-2870@2/obj/clients/.libs'
/var/shm/jenkins-nutci-netbsd-92-amd64/nut_nut_PR-2870@2/obj/tools/nut-scanner/.libs/nut-scanner: Shared object "libssl3.so" not found
/var/shm/jenkins-nutci-netbsd-92-amd64/nut_nut_PR-2870@2/obj/tools/nut-scanner/.libs/nut-scanner: Shared object "libssl3.so" not found
Tue Apr  1 20:51:02 UTC 2025 [INFO] [testcase_sandbox_nutscanner_list] findings from nut-scanner:

Tue Apr  1 20:51:02 UTC 2025 [INFO] [testcase_sandbox_nutscanner_list] inspecting these findings from nut-scanner...
Tue Apr  1 20:51:02 UTC 2025 [ERROR] [testcase_sandbox_nutscanner_list] nut-scanner complained or did not return all expected data, check above

...

================================
Wed Apr  2 08:05:20 UTC 2025 [INFO] OVERALL: PASSED=17 FAILED=1
   1 testcase_sandbox_nutscanner_list
[FATAL] Some test scenarios failed!
*** Error code 1
:; find / -name libssl3.so 2>/dev/null
/usr/pkg/lib/nss/libssl3.so

UPDATE2: Or maybe it is, the crypto libs are wanted by the nut-scanner binary directly, if built with them:

:; ldd tools/nut-scanner/.libs/nut-scanner
tools/nut-scanner/.libs/nut-scanner:
        -lnutscan.2 => not found
        -lltdl.7 => /usr/pkg/lib/libltdl.so.7
        -lc.12 => /usr/lib/libc.so.12
        -lssl3 => not found
        -lsmime3 => not found
        -lnss3 => not found
        -lnssutil3 => not found
        -lplds4 => not found
        -lplc4 => not found
        -lnspr4 => not found
        -lpthread.1 => /usr/lib/libpthread.so.1

Oddly, e.g. upsd and upsc do know where to find them, so gotta be a NUT recipe bug then:

:; ldd server/upsd
server/upsd:
        -lwrap.1 => /usr/lib/libwrap.so.1
        -lc.12 => /usr/lib/libc.so.12
        -lssl3 => /usr/pkg/lib/nss/libssl3.so
        -lplc4 => /usr/pkg/lib/nspr/libplc4.so
        -lnspr4 => /usr/pkg/lib/nspr/libnspr4.so
        -lpthread.1 => /usr/lib/libpthread.so.1
        -lplds4 => /usr/pkg/lib/nspr/libplds4.so
        -lrt.1 => /usr/lib/librt.so.1
        -lnss3 => /usr/pkg/lib/nss/libnss3.so
        -lnssutil3 => /usr/pkg/lib/nss/libnssutil3.so
        -lsmime3 => /usr/pkg/lib/nss/libsmime3.so

:;  ldd clients/.libs/upsc
clients/.libs/upsc:
        -lupsclient.6 => not found
        -lssl3 => /usr/pkg/lib/nss/libssl3.so
        -lplc4 => /usr/pkg/lib/nspr/libplc4.so
        -lnspr4 => /usr/pkg/lib/nspr/libnspr4.so
        -lpthread.1 => /usr/lib/libpthread.so.1
        -lc.12 => /usr/lib/libc.so.12
        -lplds4 => /usr/pkg/lib/nspr/libplds4.so
        -lrt.1 => /usr/lib/librt.so.1
        -lnss3 => /usr/pkg/lib/nss/libnss3.so
        -lnssutil3 => /usr/pkg/lib/nss/libnssutil3.so
        -lsmime3 => /usr/pkg/lib/nss/libsmime3.so

Signed-off-by: Jim Klimov <jimklimov+nut@gmail.com>
@jimklimov
Copy link
Member Author

jimklimov commented Apr 2, 2025

Will separate spellcheck revision commits as not NetBSD-specific. => #2871

@jimklimov
Copy link
Member Author

jimklimov commented Apr 2, 2025

So indeed, the linking calls go like this:

  • nut-scanner (note just -L/usr/pkg/lib/nss -L/usr/pkg/lib/nspr as one mention of the pathS involved):
/bin/sh ../../libtool  --tag=CC   --mode=link gcc -I../../clients -I../../clients -I../../include -I../../include -isystem /usr/pkg/include  -m64 -I/usr/pkg/include -Wno-reserved-identifier -Wno-unknown-warning-option -std=gnu99 -Wno-system-headers -Wall -Wextra -Wsign-compare -pedantic -Werror   -m64 -R/usr/pkg/lib -o nut-scanner nut_scanner-nut-scanner.o libnutscan.la -lpthread

libtool: link: gcc -I../../clients -I../../clients -I../../include -I../../include -isystem /usr/pkg/include -m64 -I/usr/pkg/include -Wno-reserved-identifier -Wno-unknown-warning-option -std=gnu99 -Wno-system-headers -Wall -Wextra -Wsign-compare -pedantic -Werror -m64 -o .libs/nut-scanner nut_scanner-nut-scanner.o  ./.libs/libnutscan.so -L/usr/pkg/lib -L/usr/pkg/lib/nss -L/usr/pkg/lib/nspr /usr/pkg/lib/libltdl.so -lssl3 -lsmime3 -lnss3 -lnssutil3 -lplds4 -lplc4 -lnspr4 -lpthread -Wl,-rpath -Wl,/usr/local/ups/lib -Wl,-rpath -Wl,/usr/pkg/lib
  • upsc:
/bin/sh ../libtool  --tag=CC   --mode=link gcc -I../include -I/usr/pkg/include/nss -I/usr/pkg/include/nspr  -isystem /usr/pkg/include  -m64 -I/usr/pkg/include -Wno-reserved-identifier -Wno-unknown-warning-option -std=gnu99 -Wno-system-headers -Wall -Wextra -Wsign-compare -pedantic -Werror  -m64 -R/usr/pkg/lib -o upsc upsc.o ../common/libcommonclient.la libupsclient.la  -Wl,-R/usr/pkg/lib/nss -L/usr/pkg/lib/nss -lssl3 -lsmime3 -lnss3 -lnssutil3 -L/usr/pkg/lib/nspr -Wl,-R/usr/pkg/lib/nspr -lplds4 -lplc4 -lnspr4  -lpthread

libtool: link: gcc -I../include -I/usr/pkg/include/nss -I/usr/pkg/include/nspr -isystem /usr/pkg/include -m64 -I/usr/pkg/include -Wno-reserved-identifier -Wno-unknown-warning-option -std=gnu99 -Wno-system-headers -Wall -Wextra -Wsign-compare -pedantic -Werror -m64 -o .libs/upsc upsc.o -Wl,-R/usr/pkg/lib/nss -Wl,-R/usr/pkg/lib/nspr  ../common/.libs/libcommonclient.a ./.libs/libupsclient.so -L/usr/pkg/lib/nss -L/usr/pkg/lib/nspr -lssl3 -lsmime3 -lnss3 -lnssutil3 -lplds4 -lplc4 -lnspr4 -lpthread -Wl,-rpath -Wl,/usr/local/ups/lib -Wl,-rpath -Wl,/usr/pkg/lib

but-but-but, the recipe is there to add LIBSSL_LDFLAGS_RPATH ... and LIBSSL_LIBS = -Wl,-R/usr/pkg/lib/nss -L/usr/pkg/lib/nss -lssl3 -lsmime3 -lnss3 -lnssutil3 -L/usr/pkg/lib/nspr -Wl,-R/usr/pkg/lib/nspr -lplds4 -lplc4 -lnspr4 initially

UPDATE: Maybe caught the difference, checking

…BSSL_LDFLAGS_RPATH when linking both libnutscan.la library and nut-scanner program [networkupstools#2870]

Signed-off-by: Jim Klimov <jimklimov+nut@gmail.com>
…E_UNINITIALIZED(_BESIDEFUNC)

Signed-off-by: Jim Klimov <jimklimov+nut@gmail.com>
…A_GCC_DIAGNOSTIC_IGNORED_MAYBE_UNINITIALIZED

Signed-off-by: Jim Klimov <jimklimov+nut@gmail.com>
… setters from BoolInt instances

Signed-off-by: Jim Klimov <jimklimov+nut@gmail.com>
…r too

Signed-off-by: Jim Klimov <jimklimov+nut@gmail.com>
…s, if any - consider not-set

Signed-off-by: Jim Klimov <jimklimov+nut@gmail.com>
@jimklimov jimklimov merged commit c169aed into networkupstools:master Apr 3, 2025
30 checks passed
@jimklimov jimklimov deleted the fix-netbsd branch April 3, 2025 00:36
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

C-str Issues and PRs about C/C++ methods, headers and data types dealing with strings and memory blocks documentation NetBSD portability We want NUT to build and run everywhere possible refactor/fightwarn PR or issue proposal to improve code maintainability without functional changes, or to fix warnings

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants