Needs work because this is not compatible with 2.15. So either it needs to be merged with the pari upgrade or be made to work with older pari too (no idea how).

Running tests now.

Many test failures.

If the changes here eventually allow us to use a system Pari 2.17, then we should undo the change in #38772.

With this MR and the cypari fixes sagemath/cypari2#165 and sagemath/cypari2#166 all tests are passing with pari 2.17.

The changes are not compatible with 2.15 though, making them compatible requires more work. Also, some pari opeations (such as the number field prime ideals above a given prime) give random output with 2.17, which makes it harder to test. To solve both issues (and make tests more future proof), we should gradually move away from testing the exact output to just testing that the output is correct.

Can reproduce it with only cypari:

In [1]: from cypari2 import Pari; pari = Pari()
In [2]: pari.read("/path/to/sage/src/sage/ext_data/pari/simon/ell.gp")
In [3]: pari.read("/path/to/sage/src/sage/ext_data/pari/simon/ellQ.gp")
In [4]: pari.setrand(1209536342279735568550346040011605782246702833689772877304100620911108570480780740384089905163231047607123731593582021687093129210645
   ...: 71674101506923221278148133793749583051043738859713730664573754392713601641751215724996543526664136197570382761930402228161378224511221436475043527
   ...: 81803440171954204516264406242258671111841440239456733647009014780493455274411161042711294346821526681893547946879139422756103853116248201592191237
   ...: 54296653651859353932015052017461312213048974428044416332457630173182056086440363409573858424366003614351392377579254220075523320605177394401598268
   ...: 19361734611515460948647432289906959793397383374073405140584680024763495157214415530031347044685621228043724290750521240469636604916727213544754631
   ...: 20304490996168334299705327969279854371179695048343426915403153805829051714387865370587161864376503770006912668191437501843065810203695377965875769
   ...: 39226621893095504530313134720900961094544924546869115554636603265293282491217782912986492685821953373693052224323165383071579088684260761929910101
   ...: 86525473510603426942413568168229990800710467673687993174453940635391234342246655437670515924492873895022209977348503127900195737248363816492641149
   ...: 272679677763898075462633737145624001967479977851573175371291641060763413884398681394550114502492552)
In [5]: pari("bnfellrank")(pari.bnfinit("y^2 - y - 232"), pari.ellinit("[2 - y, 18 + 3*y, 209 + 9*y, 2581 + 175*y, 852 - 55*y]"))
---------------------------------------------------------------------------
PariError                                 Traceback (most recent call last)
Cell In[5], line 1
----> 1 pari("bnfellrank")(pari.bnfinit("y^2 - y - 232"), pari.ellinit("[2 - y, 18 + 3*y, 209 + 9*y, 2581 + 175*y, 852 - 55*y]"))
File /usr/lib/python3.13/site-packages/cypari2/gen.pyx:4112, in cypari2.gen.Gen.__call__()
File /usr/lib/python3.13/site-packages/cypari2/handle_error.pyx:211, in cypari2.handle_error._pari_err_handle()
PariError: inconsistent dimensions in matrix col assignment

I didn't manage to reproduce it in gp. Bisected to https://pari.math.u-bordeaux.fr/cgi-bin/sgitweb.cgi?p=pari.git;a=commitdiff;h=784f1ae504e929a3873fdf25ed04f664326b5607, which I can't make much out of.

For the record, the macos conda build is failing with

In file included from sage/graphs/base/boost_graph.cpp:1270:
sage/graphs/base/boost_interface.cpp:119:34: error: non-constant-expression cannot be narrowed from type 'value_type' (aka 'unsigned long') to 'int' in initializer list [-Wc++11-narrowing-const-reference]
119 | to_return.push_back({index[boost::source(*ei, graph)],
| ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
sage/graphs/base/boost_graph.cpp:16899:29: note: in instantiation of member function 'BoostGraph<boost::vecS, boost::vecS, boost::undirectedS, boost::vecS, boost::property<boost::edge_weight_t, double>>::edge_list' requested here
16899 | __pyx_v_edges = __pyx_v_g.edge_list();
| ^
In file included from sage/graphs/base/boost_graph.cpp:1270:
sage/graphs/base/boost_interface.cpp:119:34: error: non-constant-expression cannot be narrowed from type 'value_type' (aka 'unsigned long') to 'int' in initializer list [-Wc++11-narrowing-const-reference]
119 | to_return.push_back({index[boost::source(*ei, graph)],
| ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
sage/graphs/base/boost_graph.cpp:17495:29: note: in instantiation of member function 'BoostGraph<boost::vecS, boost::vecS, boost::directedS, boost::vecS, boost::property<boost::edge_weight_t, double>>::edge_list' requested here
17495 | __pyx_v_edges = __pyx_v_g.edge_list();
| ^
2 errors generated.

Which should be fixed with #39526.

@JohnCremona - can you have a look at the report by @collares ? There is something to feed into bnfellrank of Pari 2.17 and see if it works right.

@dimpase Sorry, I've been away. Do you still need some input from me?

@JohnCremona - an expert's view of what's going on here is appreciated.

Is this a Pari bug? Or a cypari2/sagemath bug?

What is missing here? This is IMO critical (if not a blocker) for 10.6.

I don't think the bug found above is a reason to not upgrade pari. It's not even clear to me that this is a new bug. Note that bfninit() is non-deterministic, so the same construction for a given field may give different K_pari. This seems to be more noticeable on pari 2.17 because the new lll implementation (flatter) might be more numerically unstable than the previous one (fplll).

Yes, please let get this in. I've wasted enough of my time already rebasing this multiple times.

Can reproduce it with only cypari:

Here is a minimal reproducer of the underlying issue without using Simon's scripts:

In [1]: from cypari2 import Pari; pari = Pari()
In [2]: pari.setrand(1209536342279735568550346040011605782246702833689772877304100620911108570480780740384089905163231047607123731593582021687093129210645
      ⋮ 71674101506923221278148133793749583051043738859713730664573754392713601641751215724996543526664136197570382761930402228161378224511221436475043527
      ⋮ 81803440171954204516264406242258671111841440239456733647009014780493455274411161042711294346821526681893547946879139422756103853116248201592191237
      ⋮ 54296653651859353932015052017461312213048974428044416332457630173182056086440363409573858424366003614351392377579254220075523320605177394401598268
      ⋮ 19361734611515460948647432289906959793397383374073405140584680024763495157214415530031347044685621228043724290750521240469636604916727213544754631
      ⋮ 20304490996168334299705327969279854371179695048343426915403153805829051714387865370587161864376503770006912668191437501843065810203695377965875769
      ⋮ 39226621893095504530313134720900961094544924546869115554636603265293282491217782912986492685821953373693052224323165383071579088684260761929910101
      ⋮ 86525473510603426942413568168229990800710467673687993174453940635391234342246655437670515924492873895022209977348503127900195737248363816492641149
      ⋮ 272679677763898075462633737145624001967479977851573175371291641060763413884398681394550114502492552)
In [3]: pari.bnfinit("y^2 - y - 232")[7]
Out[3]: [[1, [], []], 25.8147691776365, 1, [2, -1], [;]]
In [4]: pari.bnfinit("y^2 - y - 232")[7]
Out[4]: [[1, [], []], 25.8147691776365, 1, [2, -1], [-5335854130*y - 78649159403]]

Note that the fundamental unit is missing in the first bnfinit invocation after setrand. Again, I can't reproduce it in gp.

for me ; instead of an algebraic number is pointing at a possible cypari2 bug.
@videlec ?

for me ; instead of an algebraic number is pointing at a possible cypari2 bug. @videlec ?

[;] stands for an empty matrix/vector in pari

Can reproduce it with only cypari:

Here is a minimal reproducer of the underlying issue without using Simon's scripts:

In [1]: from cypari2 import Pari; pari = Pari()
In [2]: pari.setrand(1209536342279735568550346040011605782246702833689772877304100620911108570480780740384089905163231047607123731593582021687093129210645
      ⋮ 71674101506923221278148133793749583051043738859713730664573754392713601641751215724996543526664136197570382761930402228161378224511221436475043527
      ⋮ 81803440171954204516264406242258671111841440239456733647009014780493455274411161042711294346821526681893547946879139422756103853116248201592191237
      ⋮ 54296653651859353932015052017461312213048974428044416332457630173182056086440363409573858424366003614351392377579254220075523320605177394401598268
      ⋮ 19361734611515460948647432289906959793397383374073405140584680024763495157214415530031347044685621228043724290750521240469636604916727213544754631
      ⋮ 20304490996168334299705327969279854371179695048343426915403153805829051714387865370587161864376503770006912668191437501843065810203695377965875769
      ⋮ 39226621893095504530313134720900961094544924546869115554636603265293282491217782912986492685821953373693052224323165383071579088684260761929910101
      ⋮ 86525473510603426942413568168229990800710467673687993174453940635391234342246655437670515924492873895022209977348503127900195737248363816492641149
      ⋮ 272679677763898075462633737145624001967479977851573175371291641060763413884398681394550114502492552)
In [3]: pari.bnfinit("y^2 - y - 232")[7]
Out[3]: [[1, [], []], 25.8147691776365, 1, [2, -1], [;]]
In [4]: pari.bnfinit("y^2 - y - 232")[7]
Out[4]: [[1, [], []], 25.8147691776365, 1, [2, -1], [-5335854130*y - 78649159403]]

Note that the fundamental unit is missing in the first bnfinit invocation after setrand. Again, I can't reproduce it in gp.

Looks like the difference between the cypari and gp behavior is due to the default precision. Can now reproduce in gp:

? default(realbitprecision, 53)
? setrand(120953634227973556855034604001160578224670283368977287730410062091110857048078074038408990516323104760712373159358202168709312921064571674101506923221278148133793749583051043738859713730664573754392713601641751215724996543526664136197570382761930402228161378224511221436475043527818034401719542045162644062422586711118414402394567336470090147804934552744111610427112943468215266818935479468791394227561038531162482015921912375429665365185935393201505201746131221304897442804441633245763017318205608644036340957385842436600361435139237757925422007552332060517739440159826819361734611515460948647432289906959793397383374073405140584680024763495157214415530031347044685621228043724290750521240469636604916727213544754631203044909961683342997053279692798543711796950483434269154031538058290517143878653705871618643765037700069126681914375018430658102036953779658757693922662189309550453031313472090096109454492454686911555463660326529328249121778291298649268582195337369305222432316538307157908868426076192991010186525473510603426942413568168229990800710467673687993174453940635391234342246655437670515924492873895022209977348503127900195737248363816492641149272679677763898075462633737145624001967479977851573175371291641060763413884398681394550114502492552)
? bnfinit(y^2-y-232)[8]
%3 = [[1, [], []], 25.8147691776365, 1, [2, -1], [;]]
? bnfinit(y^2-y-232)[8]
%4 = [[1, [], []], 25.8147691776365, 1, [2, -1], [-5335854130*y - 78649159403]]

Will report upstream. From quick testing, this seems to happen between 1-2% of the times for different seeds.

@antonio-rojas Great!

Could someone please also have a look at #39526 so that both PRs are merged at the same time to not have a red conda test on macos (see #38749 (comment)). Thanks!

Will report upstream. From quick testing, this seems to happen between 1-2% of the times for different seeds.

OK, the issue should be fixed now. The point is that pari's bnfinit needs to be called with flag=1 to guarantee that all data is correctly populated.

Could someone please also have a look at #39526 so that both PRs are merged at the same time to not have a red conda test on macos (see #38749 (comment)). Thanks!

How is that related to this PR?

Could someone please also have a look at #39526 so that both PRs are merged at the same time to not have a red conda test on macos (see #38749 (comment)). Thanks!

How is that related to this PR?

it's about getting the CI on macOS green.

Maybe the title should be edited to add "numpy 2.2" or something similar? the conda lock files are listing numpy 2.

Also what's the plan for numpy 1.26 support now? Are they still officially supported? (if they're not supported how are they tested? test-long still use numpy 1.26 right?)

Maybe the title should be edited to add "numpy 2.2" or something similar? the conda lock files are listing numpy 2.

Also what's the plan for numpy 1.26 support now? Are they still officially supported? (if they're not supported how are they tested? test-long still use numpy 1.26 right?)

Numpy 1.26 is EOL on 17 Sep 2025 so let's not overdo it please.

Maybe the title should be edited to add "numpy 2.2" or something similar? the conda lock files are listing numpy 2.

Also what's the plan for numpy 1.26 support now? Are they still officially supported? (if they're not supported how are they tested? test-long still use numpy 1.26 right?)

The PR to update numpy+scipy #39655
Please review

Somehow the build is falling on my machine.

#39662

Investigation needed.