`sage --package properties` by mkoeppe · Pull Request #37018 · sagemath/sage

mkoeppe · 2024-01-06T20:15:39Z

For example:

$ ./sage -package properties --format shell gap ecl                                                                                                                               git:sage_package_properties
path_ecl='/Users/mkoeppe/s/sage/sage-rebasing/worktree-pristine/build/pkgs/ecl'
version_with_patchlevel_ecl='21.2.1'
type_ecl='standard'
source_ecl='normal'
trees_ecl='SAGE_LOCAL'
path_gap='/Users/mkoeppe/s/sage/sage-rebasing/worktree-pristine/build/pkgs/gap'
version_with_patchlevel_gap='4.12.2'
type_gap='standard'
source_gap='normal'
trees_gap='SAGE_LOCAL'

The output is suitable as input for eval in Bourne shell. (The default output format is more human-friendly.)

Split out from:

Refactor bootstrap-conda, m4/sage_spkg_collect.m4 through sage-package; handle dependencies_build #36740

where this new interface is used to eliminate direct references to build/pkgs/ from various scripts and reduce code duplication and complexity in reading SPKG metadata.

📝 Checklist

The title is concise, informative, and self-explanatory.
The description explains in detail what this PR is about.
I have linked a relevant issue or discussion.
I have created tests covering the changes.
I have updated the documentation accordingly.

⌛ Dependencies

kwankyu · 2024-01-12T02:18:28Z

It would be natural to print like

$ ./sage -package properties gap                                                                                                                              
path='/Users/mkoeppe/s/sage/sage-rebasing/worktree-pristine/build/pkgs/gap'
version_with_patchlevel='4.12.2'
type='standard'
source='normal'
trees='SAGE_LOCAL'

Why accept multiple pac and add _pac?

mkoeppe · 2024-01-12T02:22:00Z

Why accept multiple pac and add _pac?

So that a shell script can request this information in bulk instead of invoking this script many times.
(In #36740, this gives a speedup in running ./configure)

kwankyu · 2024-01-12T02:27:59Z

Would you point where the suffixed string is used?

mkoeppe · 2024-01-12T02:32:18Z

In https://github.com/sagemath/sage/pull/36740/files#diff-b1f0deab6b85906bd925c4fceb337406ebea20bdc199c63996977328d24cf5c5R347, take a look at:

m4/sage_spkg_collect.m4 line 155 SPKG_VERSION=$[version_with_patchlevel_]SPKG_NAME
bootstrap line 58 eval DIR=\$path_$pkgname pkgtype=\$type_$pkgname pkgsource=\$source_$pkgname and similar

kwankyu · 2024-01-12T03:27:22Z

I see. You use the output directly to define variables...

I think the command should accommodate two uses: standalone and as pipeline.

kwankyu · 2024-01-12T03:37:42Z

How about adding an option --output=list and --output=vars with default the former? Thus

$ ./sage -package properties gap
gap:                                                                                                                              
  path='/Users/mkoeppe/s/sage/sage-rebasing/worktree-pristine/build/pkgs/gap'
  version_with_patchlevel='4.12.2'
  type='standard'
  source='normal'
  trees='SAGE_LOCAL'

$ ./sage -package properties --output=vars gap                                                                                                                              
path_gap='/Users/mkoeppe/s/sage/sage-rebasing/worktree-pristine/build/pkgs/gap'
version_with_patchlevel_gap='4.12.2'
type_gap='standard'
source_gap='normal'
trees_gap='SAGE_LOCAL'

mkoeppe · 2024-01-12T03:43:08Z

I'm open to creating output format options when needed (as you may have seen in #36740, sage --package dependencies has an option --format={plain,shell}), but I'm reluctant to add them without a specific use case

kwankyu · 2024-01-12T03:47:24Z

--format is okay.

A use case is just to read the output of ./sage -package properties gap by human eyes. This use is not expected?

mkoeppe · 2024-01-12T04:22:19Z

We already have a tool that formats package info for human inspection, that's sage -info SPKG...

kwankyu · 2024-01-12T04:30:15Z

But ./sage -package properties gap intends, as far as I understand, to provide other info (about installed packages) oriented for developers. That is a sage command, and developers may use them. If the command is only for shells, then why expose it as a sage command?

mkoeppe · 2024-01-12T04:36:45Z

./sage -package just delegates to sage-package, and ./sage -info just delegates to sage-spkg-info; both a provided by the Sage distribution. (But I feel I'm missing your point)

kwankyu · 2024-01-12T04:41:52Z

If the command is internal and not exposed to developers, I won't care the output format. But it is exposed and developers may use it and read the output, and then the output should be in a readable format. Currently the format is only suitable to shells. (I wonder if I misunderstand something...)

mkoeppe · 2024-01-12T04:51:40Z

OK, that makes sense, I'll add an option.

mkoeppe · 2024-01-12T05:04:43Z

How's this?

kwankyu · 2024-01-12T05:07:53Z

Thanks. Looks good to me.

mkoeppe · 2024-01-12T06:28:17Z

Thank you!

…ain'

sagemathgh-37018: `sage --package properties`   For example: ``` $ ./sage -package properties --format shell gap ecl git:sage_package_properties path_ecl='/Users/mkoeppe/s/sage/sage-rebasing/worktree- pristine/build/pkgs/ecl' version_with_patchlevel_ecl='21.2.1' type_ecl='standard' source_ecl='normal' trees_ecl='SAGE_LOCAL' path_gap='/Users/mkoeppe/s/sage/sage-rebasing/worktree- pristine/build/pkgs/gap' version_with_patchlevel_gap='4.12.2' type_gap='standard' source_gap='normal' trees_gap='SAGE_LOCAL' ``` The output is suitable as input for `eval` in Bourne shell. (The default output format is more human-friendly.)  Split out from: - sagemath#36740 where this new interface is used to eliminate direct references to `build/pkgs/` from various scripts and reduce code duplication and complexity in reading SPKG metadata.   ### 📝 Checklist     - [x] The title is concise, informative, and self-explanatory. - [x] The description explains in detail what this PR is about. - [x] I have linked a relevant issue or discussion. - [ ] I have created tests covering the changes. - [ ] I have updated the documentation accordingly. ### ⌛ Dependencies   URL: sagemath#37018 Reported by: Matthias Köppe Reviewer(s): Kwankyu Lee

vbraun · 2024-01-16T15:16:14Z

Do all supported platforms use python3 as the "default" python? I'm getting:

Traceback (most recent call last):
  File "/var/lib/buildbot/slave/sage_git/build/build/bin/sage-package", line 40, in <module>
    from sage_bootstrap.cmdline import run
  File "/var/lib/buildbot/slave/sage_git/build/build/bin/../sage_bootstrap/cmdline.py", line 31, in <module>
    from sage_bootstrap.app import Application
  File "/var/lib/buildbot/slave/sage_git/build/build/bin/../sage_bootstrap/app.py", line 80
    def properties(self, *package_classes, props=['path', 'version_with_patchlevel', 'type', 'source', 'trees'],
                                               ^
SyntaxError: invalid syntax
************************************************************************
Error downloading tarball of ecl
************************************************************************

mkoeppe · 2024-01-16T17:05:34Z

Sorry about this. No, we still support python 2.7 as sage-bootstrap-python; see #35858.

sagemathgh-37018: `sage --package properties`   For example: ``` $ ./sage -package properties --format shell gap ecl git:sage_package_properties path_ecl='/Users/mkoeppe/s/sage/sage-rebasing/worktree- pristine/build/pkgs/ecl' version_with_patchlevel_ecl='21.2.1' type_ecl='standard' source_ecl='normal' trees_ecl='SAGE_LOCAL' path_gap='/Users/mkoeppe/s/sage/sage-rebasing/worktree- pristine/build/pkgs/gap' version_with_patchlevel_gap='4.12.2' type_gap='standard' source_gap='normal' trees_gap='SAGE_LOCAL' ``` The output is suitable as input for `eval` in Bourne shell. (The default output format is more human-friendly.)  Split out from: - sagemath#36740 where this new interface is used to eliminate direct references to `build/pkgs/` from various scripts and reduce code duplication and complexity in reading SPKG metadata.   ### 📝 Checklist     - [x] The title is concise, informative, and self-explanatory. - [x] The description explains in detail what this PR is about. - [x] I have linked a relevant issue or discussion. - [ ] I have created tests covering the changes. - [ ] I have updated the documentation accordingly. ### ⌛ Dependencies   URL: sagemath#37018 Reported by: Matthias Köppe Reviewer(s): Kwankyu Lee

github-actions · 2024-01-16T17:48:31Z

Documentation preview for this PR (built with commit b3be864; changes) is ready! 🎉

sagemathgh-37018: `sage --package properties`   For example: ``` $ ./sage -package properties --format shell gap ecl git:sage_package_properties path_ecl='/Users/mkoeppe/s/sage/sage-rebasing/worktree- pristine/build/pkgs/ecl' version_with_patchlevel_ecl='21.2.1' type_ecl='standard' source_ecl='normal' trees_ecl='SAGE_LOCAL' path_gap='/Users/mkoeppe/s/sage/sage-rebasing/worktree- pristine/build/pkgs/gap' version_with_patchlevel_gap='4.12.2' type_gap='standard' source_gap='normal' trees_gap='SAGE_LOCAL' ``` The output is suitable as input for `eval` in Bourne shell. (The default output format is more human-friendly.)  Split out from: - sagemath#36740 where this new interface is used to eliminate direct references to `build/pkgs/` from various scripts and reduce code duplication and complexity in reading SPKG metadata.   ### 📝 Checklist     - [x] The title is concise, informative, and self-explanatory. - [x] The description explains in detail what this PR is about. - [x] I have linked a relevant issue or discussion. - [ ] I have created tests covering the changes. - [ ] I have updated the documentation accordingly. ### ⌛ Dependencies   URL: sagemath#37018 Reported by: Matthias Köppe Reviewer(s): Kwankyu Lee

sagemathgh-37350: `sage -package dependencies`   We add a new command `sage --package dependencies`. Like the command `sage --package properties` introduced in sagemath#37018, it can be used to eliminate direct references to `build/pkgs/` from various scripts and reduce code duplication in reading SPKG metadata. Here we only make such changes in `sage-spkg-info`. The only user-visible change is that the listed dependencies are now sorted alphabetically; this hides [subtle nuances that may be expressed in the sort order of some `dependencies` files](https://github.com/sagem ath/sage/pull/36878#issuecomment-1876426258). - Closes sagemath#33860 Split out from: - sagemath#36740    ### 📝 Checklist     - [x] The title is concise, informative, and self-explanatory. - [x] The description explains in detail what this PR is about. - [x] I have linked a relevant issue or discussion. - [ ] I have created tests covering the changes. - [ ] I have updated the documentation accordingly. ### ⌛ Dependencies   URL: sagemath#37350 Reported by: Matthias Köppe Reviewer(s): Kwankyu Lee, Matthias Köppe

sagemathgh-37430: Refactor `bootstrap`, `bootstrap-conda`, `m4/sage_spkg_collect.m4`, `sage-spkg-info` through `sage --package properties`, `sage-get-system-packages`   Using the command `sage --package properties` added in sagemath#37018, we eliminate some direct references to `build/pkgs/` from various scripts and reduce code duplication in reading SPKG metadata. Using other `sage --package` commands, we also simplify the code for `bootstrap -s` and `bootstrap -D`. Review instructions: Note that there are changes to the `bootstrap`-generated file `m4/sage_spkg_configures.m4`: - The macro `SAGE_SPKG_COLLECT_INIT` is now called explicitly, with an argument (the full list of packages). - Lines such as `m4_define([SPKG_INSTALL_REQUIRES_exceptiongroup], [['exceptiongroup; python_version<\"3.11\"',]])dnl` are added for use by the macro `SAGE_PYTHON_PACKAGE_CHECK` (replacing direct access to `version_requirements.txt` files)   This is: - Split out from sagemath#36740  ### 📝 Checklist     - [x] The title is concise, informative, and self-explanatory. - [x] The description explains in detail what this PR is about. - [x] I have linked a relevant issue or discussion. - [ ] I have created tests covering the changes. - [ ] I have updated the documentation accordingly. ### ⌛ Dependencies   URL: sagemath#37430 Reported by: Matthias Köppe Reviewer(s): Julian Rüth, Kwankyu Lee, Matthias Köppe, Tobias Diez

kwankyu approved these changes Jan 12, 2024

View reviewed changes

kwankyu added s: positive review and removed s: needs review labels Jan 12, 2024

Matthias Koeppe added 6 commits January 14, 2024 13:58

sage --package properties: New

f5f4b18

sage --package properties: Add path, source, trees

02dfaa9

Fix handling of SPKGs with patchlevel

55241d7

build/sage_bootstrap/package.py: Add doc

eadb3d5

sage --package properties: Add option --format, change default to 'pl…

3e5c66b

…ain'

sage --package properties: Update help

8ff9e85

vbraun added the s: needs info label Jan 16, 2024

sage_bootstrap.app: Restore Python 2 support

b3be864

vbraun removed the s: needs info label Jan 16, 2024

vbraun merged commit a5b015c into sagemath:develop Jan 22, 2024

github-actions bot removed the s: positive review label Jan 22, 2024

Uh oh!

Conversation

mkoeppe commented Jan 6, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

📝 Checklist

⌛ Dependencies

Uh oh!

kwankyu commented Jan 12, 2024

Uh oh!

mkoeppe commented Jan 12, 2024

Uh oh!

kwankyu commented Jan 12, 2024

Uh oh!

mkoeppe commented Jan 12, 2024

Uh oh!

kwankyu commented Jan 12, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

kwankyu commented Jan 12, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

mkoeppe commented Jan 12, 2024

Uh oh!

kwankyu commented Jan 12, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

mkoeppe commented Jan 12, 2024

Uh oh!

kwankyu commented Jan 12, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

mkoeppe commented Jan 12, 2024

Uh oh!

kwankyu commented Jan 12, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

mkoeppe commented Jan 12, 2024

Uh oh!

mkoeppe commented Jan 12, 2024

Uh oh!

kwankyu commented Jan 12, 2024

Uh oh!

mkoeppe commented Jan 12, 2024

Uh oh!

vbraun commented Jan 16, 2024

Uh oh!

mkoeppe commented Jan 16, 2024

Uh oh!

github-actions bot commented Jan 16, 2024

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

mkoeppe commented Jan 6, 2024 •

edited

Loading

kwankyu commented Jan 12, 2024 •

edited

Loading

kwankyu commented Jan 12, 2024 •

edited

Loading

kwankyu commented Jan 12, 2024 •

edited

Loading

kwankyu commented Jan 12, 2024 •

edited

Loading

kwankyu commented Jan 12, 2024 •

edited

Loading