Skip to content

Split libjulia into runtime and codegen components#39129

Closed
jpsamaroo wants to merge 2 commits intoJuliaLang:masterfrom
jpsamaroo:jps/split-libjulia
Closed

Split libjulia into runtime and codegen components#39129
jpsamaroo wants to merge 2 commits intoJuliaLang:masterfrom
jpsamaroo:jps/split-libjulia

Conversation

@jpsamaroo
Copy link
Copy Markdown
Member

For the purposes of embedding Julia and generating smaller, mostly standalone binaries, we may want to be able to link against only Julia's runtime components to reduce the amount of memory linked into the process. This PR splits out the runtime components from libjulia-internal into libjulia-runtime-internal, and also provides a new libjulia-stub-internal library. The intention is that the user can set an environment variable that will cause libjulia to substitute our codegen implementation (still in libjulia-internal) for a set of codegen stubs in libjulia-stub-internal (from the pre-existing src/anticodegen.c). This implementation reuses the trampoline mechanism from libjulia to allow redirecting codegen-specific symbols to the right library.

This PR does not currently succeed in generating corecompiler.ji for reasons unknown to me; if someone can figure out what I'm doing wrong here, I would appreciate the help!

@JeffBezanson JeffBezanson self-assigned this Jan 22, 2021
@JeffBezanson
Copy link
Copy Markdown
Member

JeffBezanson commented Apr 27, 2021

Thanks for working on this. @vtjnash @Keno @staticfloat and I finally got around to brainstorming how it should work. Our vision is

  • Build a separate libjulia-codegen that contains codegen.cpp and llvm-*.cpp. Ideally only this library should depend on libLLVM.
  • libjulia-internal contains stub codegen entry points (anticodegen.c) as weak symbols.
  • If you're building your own executable via PackageCompiler, libjulia-codegen can be omitted just by not putting it on the link line.
  • Initially julia.exe can just link to libjulia-codegen at build time. But we might want to avoid loading it if --compile=no is passed.
  • The loader_lib.c trampoline tricks are not needed for this, since users of libjulia (embedders) don't need to call codegen entry points directly.

Let me know if you want to keep working on this, or if I should take over.

@vchuravy
Copy link
Copy Markdown
Member

Julian and I looked at this together last week, I think we are happy if you take this over, but we can also throw some cycles at this in May. Something odd was going on with jl_default_cgparams since it was being declared both in anticodegen and codegen.

@jpsamaroo
Copy link
Copy Markdown
Member Author

Do weak symbols work on non-Linux targets? Last time we discussed this, I was told that they didn't work everywhere.

@JeffBezanson
Copy link
Copy Markdown
Member

Ok, we don't need to use weak symbols. We can just use dlsym and plugin-style interface.

JeffBezanson added a commit that referenced this pull request Aug 19, 2021
follow-up to #39129

Co-authored-by: Julian Samaroo <jpsamaroo@jpsamaroo.me>
JeffBezanson added a commit that referenced this pull request Aug 19, 2021
follow-up to #39129

Co-authored-by: Julian Samaroo <jpsamaroo@jpsamaroo.me>
JeffBezanson added a commit that referenced this pull request Aug 19, 2021
follow-up to #39129

Co-authored-by: Julian Samaroo <jpsamaroo@jpsamaroo.me>
JeffBezanson added a commit that referenced this pull request Aug 19, 2021
follow-up to #39129

Co-authored-by: Julian Samaroo <jpsamaroo@jpsamaroo.me>
@JeffBezanson
Copy link
Copy Markdown
Member

Superseded by #41936

JeffBezanson added a commit that referenced this pull request Aug 25, 2021
follow-up to #39129

Co-authored-by: Julian Samaroo <jpsamaroo@jpsamaroo.me>
JeffBezanson added a commit that referenced this pull request Sep 2, 2021
follow-up to #39129

Co-authored-by: Julian Samaroo <jpsamaroo@jpsamaroo.me>
JeffBezanson added a commit that referenced this pull request Sep 2, 2021
follow-up to #39129

Co-authored-by: Julian Samaroo <jpsamaroo@jpsamaroo.me>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants