How best to handle setting build and run environment variables #13926
Replies: 7 comments 3 replies
-
To elaborate further on this, I think there are really only 2 types of environment variables, ones needed to build a package and ones needed to run a package. In my mind, the only point of the def setup_run_environment(self, env):
env.prepend_path('PYTHONPATH', site_packages_dir)in every single Python package. But if we subsume this into the |
Beta Was this translation helpful? Give feedback.
-
|
@alalazo @scheibelp @becker33 @tgamblin I know I opened this issue at an inopportune time (during Thanksgiving break), but I was wondering if any of you have any opinions on this. Basically what I'm proposing is that:
See #14191 and #14198 for another unfortunate victim of this confusion. |
Beta Was this translation helpful? Give feedback.
-
|
@adamjstewart Agreed, I was just refactoring a small package (not in the main spack repo) to add these methods and was struggling to discern the difference between the run environment and the dependent environments. Fortunately it's not too hard to reuse these functions: def setup_run_environment(self, env):
env.set('DATAPATH', self.prefix)
def setup_dependent_build_environment(self, env, dependent_spec):
self.setup_run_environment(env)
setup_dependent_run_environment = setup_dependent_build_environment |
Beta Was this translation helpful? Give feedback.
-
|
Note that this distinction between I would expect Then what's the point of setup_dependent_run_environment except for causing confusion? |
Beta Was this translation helpful? Give feedback.
-
|
Another question is how to handle tests, both at build-time ( |
Beta Was this translation helpful? Give feedback.
Uh oh!
There was an error while loading. Please reload this page.
Uh oh!
There was an error while loading. Please reload this page.
-
While working on #13924, I had some thoughts I wanted to share.
Description
Previously, we used the following API to control environment variables in Spack:
This had a lot of problems, namely that certain files/directories are only present after installation, so we can't run the same function both before and after installation. Now, we use the following API:
This fixes the previously mentioned problems, but also introduces a lot of redundancy that I'll mention below.
Proposal
I'm starting to think that a better API would be something like:
Specifically, I think we should replace
setup_dependent_build_environmentwith an additional call tosetup_run_environment, and I thinksetup_dependent_run_environmentisn't necessary.Rationale
setup_run_environmentandsetup_dependent_build_environmentare almost identical in most packages that use them. This makes sense, as any env var needed to run a package is also needed when that package is being used as a build dependency. So why don't we get rid ofsetup_dependent_build_environmentand just runsetup_run_environmentat build time for packages that depend on a package?setup_dependent_run_environmentshould not actually be necessary. This gets into the discussion we had in Modules: Always autoload run dependencies #8639, but basically, if I declare something to be atype='run'dependency, it should be implied that any env vars needed to run a package are also needed to run a package that depends on this package. We should either autoload run-time dependencies or combine their env vars in the module file.I'm sure there are many use cases I'm ignoring, particularly with things like
PYTHONPATHandPERL5LIBwhere we need to recursively set env vars for language dependencies, but I think we currently overusesetup_dependent_*_environment. I can't think of a case where it should be used aside from anextendable = Truepackage.@alalazo @becker33 @scheibelp might be interested in this discussion.
Beta Was this translation helpful? Give feedback.
All reactions