Skip to content

add elf parsing utility function#33628

Merged
haampie merged 1 commit intospack:developfrom
haampie:feature/python-based-elf-parsing-2
Nov 1, 2022
Merged

add elf parsing utility function#33628
haampie merged 1 commit intospack:developfrom
haampie:feature/python-based-elf-parsing-2

Conversation

@haampie
Copy link
Copy Markdown
Member

@haampie haampie commented Oct 31, 2022

This PR introduces some basic ELF parsing logic, which is useful in various
places.

Example uses:

  • Retrieve all ELF executables and shared libs in a prefix
  • Get the offset of the rpath string in a file so it can be modified in-place
  • Get better info about why a file has no rpath (e.g. not an elf file,
    malformed elf, no dynamic section, missing tag in dynamic array, ...), this
    is more fine-grained than in patchelf.

It's very basic, and it's really about parsing, since we don't want to reimplement
the advanced dynamic section growth logic that patchelf offers.

Current API is basically:

import spack.util.elf as elf

try:
    with open("my_binary", "br") as f:
        parsed = elf.parse_elf(f)
except elf.ElfParsingError:
    # handle invalid elf file

# booleans
parsed.has_pt_interp
parsed.has_pt_dynamic
parsed.has_rpath
parsed.has_needed
parsed.has_soname
parsed.is_runpath

# byte strings
parsed.pt_interp_str
parsed.dt_soname_str
parsed.dt_rpath_str
parsed.dt_needed_strs  # list of byte strings

# more low-level offsets into the file, or the strtab associated with the dynamic section (useful for in-place updates)

Another advantage is that we get a ~850x speedup for retrieving rpaths.

In [1]: spack.relocate._elf_rpaths_for("libclang.so.15.0.0") == spack.util.elf.get_rpaths("libclang.so.15.0.0")
Out[1]: True

In [2]: %timeit spack.relocate._elf_rpaths_for("libclang.so.15.0.0")
42.8 ms ± 448 µs per loop (mean ± std. dev. of 7 runs, 10 loops each)

In [3]: %timeit spack.util.elf.get_rpaths("libclang.so.15.0.0")
50.4 µs ± 435 ns per loop (mean ± std. dev. of 7 runs, 10,000 loops each)

@spackbot-app spackbot-app bot added core PR affects Spack core functionality tests General test capability(ies) utilities labels Oct 31, 2022
@haampie haampie force-pushed the feature/python-based-elf-parsing-2 branch from b923698 to d5ab98c Compare October 31, 2022 22:19
@haampie haampie requested a review from trws October 31, 2022 22:19
@haampie haampie force-pushed the feature/python-based-elf-parsing-2 branch 3 times, most recently from bbe79b7 to 7118cc6 Compare October 31, 2022 22:41
@trws
Copy link
Copy Markdown
Contributor

trws commented Nov 1, 2022

I like this, a lot, and the test changes are really welcome. Had a couple of questions but I don't see anything that would make me want to keep this out.

@haampie haampie force-pushed the feature/python-based-elf-parsing-2 branch from 3fed26b to 4b9d459 Compare November 1, 2022 17:40
@haampie haampie enabled auto-merge (squash) November 1, 2022 19:14
@haampie haampie merged commit 230e96f into spack:develop Nov 1, 2022
@haampie haampie deleted the feature/python-based-elf-parsing-2 branch November 1, 2022 19:42
charmoniumQ pushed a commit to charmoniumQ/spack that referenced this pull request Nov 19, 2022
Introduces `spack.util.elf.parse_elf(file_handle)`
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

binary-packages core PR affects Spack core functionality tests General test capability(ies) utilities

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants