This will need some input from Gap / downstream packaging experts @dimpase @orlitzky @tornaria @antonio-rojas; I have not followed the discussion on this closely.

The alternative to what is done here in the PR would be to keep the feature definitions for these packages and to change the tags from # optional - gap_packages_polycyclic to # needs gap_packages_polycyclic etc.

I don't care what you want to call it but the tests shouldn't fail when the system gap is missing one of these packages. The only GAP packages that you can assume to be present are gapdoc, primgrp, smallgrp, and transgrp.

Could you use something like GapPackage("polycyclic", spkg="gap", type='standard') This is what you really want to say about this package: it is standard (for sage-the-distro) and it comes in spkg gap (for sage-the-distro). But you still need to test the presence of this package since system gap might not have it.

I think it's OK for sage to assume some GAP packages are available if the funtionality they provide is deemed to be non-optional, the same way it assumes all standard dependencies are installed (and it's the packagers responsibility to make sure sage pulls them as dependencies). But in that case the spkg-configure script should be adapted to only accept system GAP if all required packages are available.

I don't care what you want to call it but the tests shouldn't fail when the system gap is missing one of these packages. The only GAP packages that you can assume to be present are gapdoc, primgrp, smallgrp, and transgrp.

Do we have CI runs that cover such cases? If yes can you give an example?

Could you use something like GapPackage("polycyclic", spkg="gap", type='standard') This is what you really want to say about this package: it is standard (for sage-the-distro) and it comes in spkg gap (for sage-the-distro). But you still need to test the presence of this package since system gap might not have it.

Currently GapPackage("polycyclic", spkg="gap", type='standard') causes UserWarning: Feature gap_package_polycyclic is declared standard, but it is provided by gap_package... (see #36741 (comment)). At least we should suppress this warning than. What about to introduce a new value for type for instance semi standard or indirect standard indicating a standard downstream package of a standard upstream package.

How about simply moving all such GAP packages from gap_packages spkg into gap spkg?

I don't care what you want to call it but the tests shouldn't fail when the system gap is missing one of these packages. The only GAP packages that you can assume to be present are gapdoc, primgrp, smallgrp, and transgrp.

Do we have CI runs that cover such cases? If yes can you give an example?

Probably not, but those packages are what's tested in spkg-configure.m4 and in the new meson (build system for sagelib) pull request. So they're guaranteed to be there but nothing else is. I run without the others.

How about simply moving all such GAP packages from gap_packages spkg into gap spkg?

Yes, that would be a solution. Or alternatively make it a separate gap_packages_standard package.

upstream GAP's default is in PackagesToLoad

   PackagesToLoad 
        A  list  of names of packages which should be loaded during startup. 
        For backwards compatibility, the default lists most of
        packages that were autoloaded in GAP 4.4 (add or remove packages as you like).
  
        Default:  [  "autpgrp",  "alnuth",  "crisp",  "ctbllib",  "factint",  "fga", "irredsol", "laguna", 
                            "polenta", "polycyclic", "resclasses", "sophus", "tomlib" ].  

of these, ctbllib is 73Mb, tomlib 53Mb, irredsol 27Mb . The rest don't have data, only code, and are much smaller.

OK to move them all?

OK to move them all?

At least it would be consequent! But I don't know if the additional volume of datasize is justified.

Could you use something like GapPackage("polycyclic", spkg="gap", type='standard') This is what you really want to say about this package: it is standard (for sage-the-distro) and it comes in spkg gap (for sage-the-distro). But you still need to test the presence of this package since system gap might not have it.

Currently GapPackage("polycyclic", spkg="gap", type='standard') causes UserWarning: Feature gap_package_polycyclic is declared standard, but it is provided by gap_package... (see #36741 (comment)). At least we should suppress this warning than. What about to introduce a new value for type for instance semi standard or indirect standard indicating a standard downstream package of a standard upstream package.

I'm confused. I thought the warning was caused by GapPackage("polycyclic", spkg="gap_packages", type='standard'). Do you get the same warning with spkg="gap"?

Since polycyclic is indeed shipped as part of spkg gap (and not gap_packages), the feature GapPackage("polycyclic", spkg="gap", type='standard') seems to accurately match the current state of things.

I'm confused. I thought the warning was caused by GapPackage("polycyclic", spkg="gap_packages", type='standard'). Do you get the same warning with spkg="gap"?

Sorry! I missed that detail. You are right, that should not lead to the warning!

Documentation preview for this PR (built with commit b392d5b; changes) is ready! 🎉

How about simply moving all such GAP packages from gap_packages spkg into gap spkg?

@dimpase who can do that? Shall we have a follow-up issue / PR for that? Then I would just delete my branch!

I don't think we ever got around to this, so please complete this one, and we'll merge it

Ah, OK, sorry, I was not reading well.

It is fine to move all such GAP packages to be installed by gap spkg.

In case you don't see how to do this, open an issue, otherwise just another PR.

One problem you are going to run into is that the packages in gap_packages are outdated and e.g. don't build with gcc-15. Moving them into the gap SPKG keeps them tied to the gap tarball & release schedule.

It would be much better to split them out into separate SPKGs that can be version-bumped and feature-tested independently. Most have their own upstreams and release schedules.

In any case, I don't really care what you do in sage-the-distro, so long as these tests still pass when the GAP packages aren't installed. The cohomolo and datastructures packages, for example, still can't be built on ~arch Gentoo so it's pretty convenient that they're optional.

I would rather have GAP's PackageManager take care of this.

Move to gcc-15 is a one time event, which can be addressed by e.g. a custom tarball