Skip to content

build/rgw/lua: remove luarocks support from rhel8#54789

Merged
yuvalif merged 1 commit intoceph:mainfrom
yuvalif:wip-yuval-remove-lua-devel
Dec 10, 2023
Merged

build/rgw/lua: remove luarocks support from rhel8#54789
yuvalif merged 1 commit intoceph:mainfrom
yuvalif:wip-yuval-remove-lua-devel

Conversation

@yuvalif
Copy link
Contributor

@yuvalif yuvalif commented Dec 5, 2023

this is due to the dependency of luarocks with lua-devel which cannot be installed in teuthology

Fixes: https://tracker.ceph.com/issues/63672

Checklist

  • Tracker (select at least one)
    • References tracker ticket
    • Very recent bug; references commit where it was introduced
    • New feature (ticket optional)
    • Doc update (no ticket needed)
    • Code cleanup (no ticket needed)
  • Component impact
    • Affects Dashboard, opened tracker ticket
    • Affects Orchestrator, opened tracker ticket
    • No impact that needs to be tracked
  • Documentation (select at least one)
    • Updates relevant documentation
    • No doc update is appropriate
  • Tests (select at least one)
Show available Jenkins commands
  • jenkins retest this please
  • jenkins test classic perf
  • jenkins test crimson perf
  • jenkins test signed
  • jenkins test make check
  • jenkins test make check arm64
  • jenkins test submodules
  • jenkins test dashboard
  • jenkins test dashboard cephadm
  • jenkins test api
  • jenkins test docs
  • jenkins render docs
  • jenkins test ceph-volume all
  • jenkins test ceph-volume tox
  • jenkins test windows
  • jenkins test rook e2e

@yuvalif yuvalif requested a review from cbodley December 5, 2023 18:48
@batrick batrick added cephfs Ceph File System needs-qa labels Dec 5, 2023
@yuvalif
Copy link
Contributor Author

yuvalif commented Dec 6, 2023

@batrick which teuthology command should i use to verify that the PR fixed the issue?
(i tried to run the fs:verify suite, but it uses only cento9 and ubuntu22 machines)

@yuvalif
Copy link
Contributor Author

yuvalif commented Dec 6, 2023

lua regression (on ubuntu) is passing: https://pulpito.ceph.com/yuvalif-2023-12-06_08:39:15-rgw:lua-wip-yuval-remove-lua-devel-distro-default-smithi/

mchangir
mchangir reviewed Dec 6, 2023
View reviewed changes
@mchangir
Copy link
Contributor

mchangir commented Dec 7, 2023

@batrick which teuthology command should i use to verify that the PR fixed the issue? (i tried to run the fs:verify suite, but it uses only cento9 and ubuntu22 machines)

The fs:verify suite doesn't have a centos/rhel 8 distro available as a choice (its got only ubuntu and centos 9.stream)

try this:

teuthology-suite --ceph wip-yuval-remove-lua-devel --dry-run --limit 10 --suite fs:volumes --filter-all k-stock,rhel_8

@yuvalif
Copy link
Contributor Author

yuvalif commented Dec 7, 2023

rhel8 tests are still failing. i get the following error:

2023-12-07T14:34:41.194 INFO:teuthology.orchestra.run.smithi023.stderr:Error:
2023-12-07T14:34:41.194 INFO:teuthology.orchestra.run.smithi023.stderr: Problem: conflicting requests
2023-12-07T14:34:41.195 INFO:teuthology.orchestra.run.smithi023.stderr:  - nothing provides lua-devel needed by ceph-2:19.0.0-8.g4bd5dd3a.el8.x86_64

@cbodley
Copy link
Contributor

cbodley commented Dec 7, 2023

rhel8 tests are still failing. i get the following error:

2023-12-07T14:34:41.194 INFO:teuthology.orchestra.run.smithi023.stderr:Error:
2023-12-07T14:34:41.194 INFO:teuthology.orchestra.run.smithi023.stderr: Problem: conflicting requests
2023-12-07T14:34:41.195 INFO:teuthology.orchestra.run.smithi023.stderr:  - nothing provides lua-devel needed by ceph-2:19.0.0-8.g4bd5dd3a.el8.x86_64

does ceph.spec.in still enable it elsewhere with %bcond_without lua_packages? i was expecting logic along the lines of

%if 0%{?rhel} <= 8
%bcond_with lua_packages
%else
%bcond_without lua_packages
%endif

@yuvalif
Copy link
Contributor Author

yuvalif commented Dec 7, 2023

it is enabled here:
https://github.com/ceph/ceph/pull/54789/files#diff-34eff6689abb9572612fd8b57b58beefc7fa86fd2bb54efd27dfe628f67bfa62R55

there is also handling of this variable in case of suse.
also, note that the lua regression test on ubuntu passed

@yuvalif
build/rgw/lua: remove lua-devel as a runtime dependency
b0a77a5 
this is due to the fact that lua-devel
cannot be installed in teuthology for rhel8

this would fail the build of some luarocks packages

Fixes: https://tracker.ceph.com/issues/63672

Signed-off-by: Yuval Lifshitz <ylifshit@redhat.com>
@yuvalif yuvalif force-pushed the wip-yuval-remove-lua-devel branch from 4bd5dd3 to b0a77a5 Compare December 10, 2023 11:14
@yuvalif
Copy link
Contributor Author

yuvalif commented Dec 10, 2023

just removed lua-devel from the runtime.

@yuvalif yuvalif removed the needs-qa label Dec 10, 2023
@cbodley
Copy link
Contributor

cbodley commented Dec 10, 2023

The following tests FAILED:
32 - run-rbd-unit-tests-61.sh (Failed)

@cbodley
Copy link
Contributor

cbodley commented Dec 10, 2023

jenkins test make check

@cbodley
Copy link
Contributor

cbodley commented Dec 10, 2023

thanks, it was useful to see how this fails. radosgw shells out to /bin/luarocks which can't find the header files from lua-devel

2023-12-10T10:22:38.147+0000 7f360813e640 20 rgw lua package reloader: Lua CMD: /bin/luarocks install --no-doc --lua-version 5.4 --tree /tmp/rgw_luarocks/client.0/5ik7sc --deps-mode one lua-cjson
▸       mkdir: cannot create directory ‘/var/cache/luarocks’: Permission denied
▸       Installing https://luarocks.org/lua-cjson-2.1.0.10-1.src.rock
▸       Error: Failed finding Lua header files. You may need to install them or configure LUA_INCDIR.
2023-12-10T10:22:39.786+0000 7f360813e640 20 rgw lua package reloader: Lua CMD: /bin/luarocks install --no-doc --lua-version 5.4 --tree /tmp/rgw_luarocks/client.0/5ik7sc --deps-mode one luasocket
▸       mkdir: cannot create directory ‘/var/cache/luarocks’: Permission denied
▸       Installing https://luarocks.org/luasocket-3.1.0-1.src.rock
▸       Error: Failed finding Lua header files. You may need to install them or configure LUA_INCDIR.
2023-12-10T10:22:39.786+0000 7f360813e640  5 rgw lua package reloader: WARNING: failed to install Lua package: lua-cjson from allowlist
2023-12-10T10:22:39.786+0000 7f360813e640  5 rgw lua package reloader: WARNING: failed to install Lua package: luasocket from allowlist

in order for uploaded scripts to run against such packages, admins would have to install lua-devel manually on the nodes running radosgw. there's a desire to support this feature in containerized deployments, where the containers would need lua-devel preinstalled. if we can't use Requires: lua-devel to accomplish that on reef, we'd need some other mechanism

@yuvalif
Copy link
Contributor Author

yuvalif commented Dec 10, 2023

thanks, it was useful to see how this fails. radosgw shells out to /bin/luarocks which can't find the header files from lua-devel

2023-12-10T10:22:38.147+0000 7f360813e640 20 rgw lua package reloader: Lua CMD: /bin/luarocks install --no-doc --lua-version 5.4 --tree /tmp/rgw_luarocks/client.0/5ik7sc --deps-mode one lua-cjson
▸       mkdir: cannot create directory ‘/var/cache/luarocks’: Permission denied
▸       Installing https://luarocks.org/lua-cjson-2.1.0.10-1.src.rock
▸       Error: Failed finding Lua header files. You may need to install them or configure LUA_INCDIR.
2023-12-10T10:22:39.786+0000 7f360813e640 20 rgw lua package reloader: Lua CMD: /bin/luarocks install --no-doc --lua-version 5.4 --tree /tmp/rgw_luarocks/client.0/5ik7sc --deps-mode one luasocket
▸       mkdir: cannot create directory ‘/var/cache/luarocks’: Permission denied
▸       Installing https://luarocks.org/luasocket-3.1.0-1.src.rock
▸       Error: Failed finding Lua header files. You may need to install them or configure LUA_INCDIR.
2023-12-10T10:22:39.786+0000 7f360813e640  5 rgw lua package reloader: WARNING: failed to install Lua package: lua-cjson from allowlist
2023-12-10T10:22:39.786+0000 7f360813e640  5 rgw lua package reloader: WARNING: failed to install Lua package: luasocket from allowlist

in order for uploaded scripts to run against such packages, admins would have to install lua-devel manually on the nodes running radosgw. there's a desire to support this feature in containerized deployments, where the containers would need lua-devel preinstalled. if we can't use Requires: lua-devel to accomplish that on reef, we'd need some other mechanism

note that this is not a rehl8 issue (lua-devel is in the CRB repo of rhel8). its is a rhel8 in the sepia lab issue.
this PR is a temporary workaround until the lab is fixed.

@yuvalif yuvalif merged commit eddde0d into ceph:main Dec 10, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@mchangir mchangir mchangir left review comments

@batrick batrick batrick approved these changes

@cbodley cbodley cbodley approved these changes

Assignees

No one assigned

Labels

build/ops cephfs Ceph File System

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

4 participants

@yuvalif @mchangir @cbodley @batrick