Skip to content

Use compression libraries#141

Merged
legionus merged 14 commits intomasterfrom
use-compression-libraries
Aug 17, 2025
Merged

Use compression libraries#141
legionus merged 14 commits intomasterfrom
use-compression-libraries

Conversation

@legionus
Copy link
Copy Markdown
Owner

In some configurations, the use of utilities is redundant. Also, popen() requires /bin/sh. However, not all configurations utilize the libz and libbz2 libraries. For example, busybox has its own implementations of gzip and bzip2.

A solution that would suit everyone is to add support for ELF_DLOPEN_METADATA[1]. This will avoid unnecessary dependencies where utilities are available and avoid using utilities where libraries are available.

[1] https://github.com/systemd/systemd/blob/main/docs/ELF_DLOPEN_METADATA.md

Fixes: #138

legionus added 14 commits August 7, 2025 10:23
The sparse utility does not support the code that will be added:

  extern int gzopen(int arg);
  static typeof(gzopen) *sym_gzopen;

  int main(void)
  {
  	int i = sym_gzopen(1);
  	return 0;
  }

For now, we will have to disable this check until support for this
syntax is added.

Signed-off-by: Alexey Gladkov <legion@kernel.org>
Fixed a typo in the ppc64el architecture check that was not working as
intended.

Fixes: 38e46e6 ("tests: disable libkbdfile testcase on ppc64el")
Signed-off-by: Alexey Gladkov <legion@kernel.org>
Added archive suffix check, added subdirectory search check.

Signed-off-by: Alexey Gladkov <legion@kernel.org>
Signed-off-by: Alexey Gladkov <legion@kernel.org>
Use maybe_pipe_open() as a common wrapper to open a file and possibly
uncompress it. The decompressor is now picked only here.

Signed-off-by: Alexey Gladkov <legion@kernel.org>
A more reliable way to determine the archive type is to use the content.
For backward compatibility, archive suffixes are added to filenames, but
this is no longer a criteria for selecting a decompressor.

Signed-off-by: Alexey Gladkov <legion@kernel.org>
Currently, the PIPE flag indicates that the file is compressed, because
an external utility gzip/bzip2 is used to unpack the file.

Signed-off-by: Alexey Gladkov <legion@kernel.org>
In some configurations, the use of utilities is redundant. Also, popen()
requires /bin/sh. However, not all configurations utilize the libz and
libbz2 libraries. For example, busybox has its own implementations of
gzip and bzip2.

A solution that would suit everyone is to add support for ELF_DLOPEN_METADATA[1].
This will avoid unnecessary dependencies where utilities are available
and avoid using utilities where libraries are available.

[1] https://github.com/systemd/systemd/blob/main/docs/ELF_DLOPEN_METADATA.md

Link: #138
Signed-off-by: Alexey Gladkov <legion@kernel.org>
Use dynamic loading of zlib for direct use without direct linking.

Signed-off-by: Alexey Gladkov <legion@kernel.org>
Signed-off-by: Alexey Gladkov <legion@kernel.org>
Use dynamic loading of libbz2 for direct use without direct linking.

Signed-off-by: Alexey Gladkov <legion@kernel.org>
Use dynamic loading of liblzma for direct use without direct linking.

Signed-off-by: Alexey Gladkov <legion@kernel.org>
Use dynamic loading of libzstd for direct use without direct linking.

Signed-off-by: Alexey Gladkov <legion@kernel.org>
Signed-off-by: Alexey Gladkov <legion@kernel.org>
@nikstur
Copy link
Copy Markdown

nikstur commented Aug 12, 2025

I will try building this in Nixpkgs soon

@nikstur
Copy link
Copy Markdown

nikstur commented Aug 15, 2025

I tested this in Nixpkgs: NixOS/nixpkgs#434001

You could build the tests by:

  1. Installing Nix
  2. Checking out the PR
  3. Running nix-build -A kbd.tests

However, this targets staging. So unless you have a very powerful machine I cannot recommend doing that.

Ran the tests and it all looks fine.

I'd love if this landed upstream and if you could cut a new release for it soon.

@legionus legionus merged commit eddc753 into master Aug 17, 2025
8 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Link against compression libs directly instead of shelling out

2 participants