Skip to content

koreader: build from source#178320

Closed
ghost wants to merge 41 commits intomasterfrom
unknown repository
Closed

koreader: build from source#178320
ghost wants to merge 41 commits intomasterfrom
unknown repository

Conversation

@ghost
Copy link
Copy Markdown

@ghost ghost commented Jun 20, 2022

This still needs quite a bit of cleanup. Will undraftify when ready.

Description of changes

Build koreader from source instead of downloading binaries.

Things done
  • Built on platform(s)
    • x86_64-linux
    • aarch64-linux
    • x86_64-darwin
    • aarch64-darwin
  • For non-Linux: Is sandbox = true set in nix.conf? (See Nix manual)
  • Tested, as applicable:
  • Tested compilation of all packages that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review rev HEAD". Note: all changes have to be committed, also see nixpkgs-review usage
  • Tested basic functionality of all binary files (usually in ./result/bin/)
  • 22.11 Release Notes (or backporting 22.05 Release notes)
    • (Package updates) Added a release notes entry if the change is major or breaking
    • (Module updates) Added a release notes entry if the change is significant
    • (Module addition) Added a release notes entry if adding a new NixOS module
    • (Release notes changes) Ran nixos/doc/manual/md-to-db.sh to update generated release notes
  • Fits CONTRIBUTING.md.

This was referenced Jun 20, 2022
Closed
Closed
@github-actions github-actions bot added 6.topic: stdenv Standard environment and removed 6.topic: stdenv Standard environment labels Jun 20, 2022
Frenzie
Frenzie reviewed Jun 20, 2022
View reviewed changes
@AndersonTorres
Copy link
Copy Markdown
Member

AndersonTorres commented Jun 21, 2022

My two cents.
You should keep this file, instead of "editing" it.

First and foremost, correcting what you have said before in koreader/koreader-base#1486, using precompiled packages is not frowned upon in Nixpkgs. Indeed it is sometimes a necessity (for closed source programs), or it is sometimes way easier to rely on a precompiled binary instead of a long and intricated process.

Second, that being said, most of the time compiling from sources is slower and more tedious and error-prone than using a precompiled file. Nonetheless, many people prefer a compilation from sources.

So, I believe the best course of action is to create two derivations: one for tracking the precompiled distribution provided by upstream (call it koreader-bin), and the other for tracking the human-readable source code.

Third, you are thwarting the raison d'étre here: the process to build from sources is completely different from the process to build from a precompiled. It is like to conflate apples with pears. You do not "patch an apple" in order to produce a pear.

@ghost
Copy link
Copy Markdown
Author

ghost commented Jun 21, 2022

one for tracking the precompiled distribution provided by upstream (call it koreader-bin), and the other for tracking the human-readable source code.

Fine with me.

@github-actions github-actions bot added the 6.topic: lua Lua is a powerful, efficient, lightweight, embeddable scripting language. label Jun 21, 2022
@ghost
Copy link
Copy Markdown
Author

ghost commented Jun 21, 2022 •
edited by ghost
Loading

Still WIP but very close now. Outstanding issues:

  1. I've had to disable lua-Spore because it wants luajson, which is not vendored by koreader. I've added a luaPackages.luajson expression, but I can't seem to get koreader's build process to recognize it. Obviously this must be fixed before merging; I will revisit this.
2. Resolved

Apparently a nix sandbox escape (!?!?!) by cmake. I'm getting cmake errors which show the outside-the-sandbox paths (/tmp/nix-build-XXX), which cmake shouldn't even be able to see in the first place. This requires further investigation.

koreader> CMake Error: The source "/build/source/base/thirdparty/lua-rapidjson/build/x86_64-unknown-linux-gnu/lua-rapidjson-prefix/src/lua-rapidjson/CMakeLists.txt" does not match the source "/tmp/nix-build-koreader-2022.05.1.drv-1/ source/base/thirdparty/lua-rapidjson/build/x86_64-unknown-linux-gnu/lua-rapidjson-prefix/src/lua-rapidjson/CMakeLists.txt" used to generate cache. Re-run cmake with a different source directory.

Context:

koreader> [ 62%] No configure step for 'lua-rapidjson'
koreader> [ 75%] Performing build step for 'lua-rapidjson'
koreader> rapidjson 0.7.1-1 depends on lua >= 5.1 (5.2-1 provided by VM)
koreader> CMake Error: The current CMakeCache.txt directory /build/source/base/thirdparty/lua-rapidjson/build/x86_64-unknown-linux-gnu/lua-rapidjson-prefix/src/lua-rapidjson/build.luarocks/CMakeCache.txt is different than the directory /tmp/nix-build-koreader-2022.05.1.drv-1/source/base/thirdparty/lua-rapidjson/build/x86_64-unknown-linux-gnu/lua-rapidjson-prefix/src/lua-rapidjson/build.luarocks where CMakeCache.txt was created. This may result in binaries being created in the wrong place. If you are not sure, reedit the CMakeCache.txt
koreader> CMake Error: The source "/build/source/base/thirdparty/lua-rapidjson/build/x86_64-unknown-linux-gnu/lua-rapidjson-prefix/src/lua-rapidjson/CMakeLists.txt" does not match the source "/tmp/nix-build-koreader-2022.05.1.drv-1/source/base/thirdparty/lua-rapidjson/build/x86_64-unknown-linux-gnu/lua-rapidjson-prefix/src/lua-rapidjson/CMakeLists.txt" used to generate cache.  Re-run cmake with a different source directory.
koreader> Error: Build error: Failed cmake.
koreader> make[4]: *** [CMakeFiles/lua-rapidjson.dir/build.make:86: lua-rapidjson-prefix/src/lua-rapidjson-stamp/lua-rapidjson-build] Error 1
koreader> make[4]: Leaving directory '/build/source/base/thirdparty/lua-rapidjson/build/x86_64-unknown-linux-gnu'
koreader> make[3]: *** [CMakeFiles/Makefile2:83: CMakeFiles/lua-rapidjson.dir/all] Error 2
koreader> make[3]: Leaving directory '/build/source/base/thirdparty/lua-rapidjson/build/x86_64-unknown-linux-gnu'
koreader> make[2]: *** [Makefile:91: all] Error 2
koreader> make[2]: Leaving directory '/build/source/base/thirdparty/lua-rapidjson/build/x86_64-unknown-linux-gnu'
koreader> make[1]: *** [Makefile.third:712: build/x86_64-unknown-linux-gnu/rocks/lib/luarocks/rocks/rapidjson/0.7.1-1/rapidjson-0.7.1-1.rockspec] Error 2
koreader> make[1]: Leaving directory '/build/source/base'
koreader> make: *** [Makefile:128: base/build/x86_64-unknown-linux-gnu/luajit] Error 2

Root cause was the fact that I am very stupid.

@ghost
Copy link
Copy Markdown
Author

ghost commented Jun 21, 2022

@ofborg eval

@ofborg ofborg bot added the 8.has: package (new) This PR adds a new package label Jun 21, 2022
@ofborg ofborg bot requested a review from contrun June 21, 2022 04:46
@ofborg ofborg bot added 10.rebuild-darwin: 1-10 This PR causes between 1 and 10 packages to rebuild on Darwin. 10.rebuild-linux: 1-10 This PR causes between 1 and 10 packages to rebuild on Linux. labels Jun 21, 2022
@ghost
Copy link
Copy Markdown
Author

ghost commented Jun 22, 2022

Fixed sdcv. One down, one to go.

@ghost ghost mentioned this pull request Jun 22, 2022
Closed
6 tasks
@ghost
Copy link
Copy Markdown
Author

ghost commented Jun 22, 2022

Squashed and ready for review here: #178557

@ghost ghost closed this Jun 22, 2022
@ghost ghost deleted the pr/koreader/fromsource branch June 22, 2022 11:14
@Artturin Artturin mentioned this pull request Jan 29, 2024
Open
This pull request was closed.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@contrun contrun Awaiting requested review from contrun

Assignees

No one assigned

Labels

6.topic: lua Lua is a powerful, efficient, lightweight, embeddable scripting language. 8.has: package (new) This PR adds a new package 10.rebuild-darwin: 1-10 This PR causes between 1 and 10 packages to rebuild on Darwin. 10.rebuild-linux: 1-10 This PR causes between 1 and 10 packages to rebuild on Linux.

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

3 participants

@AndersonTorres @Frenzie @Mathnerd314