Finally got this to pass the tests - the problem with not having done anything with erfa directly!

This PR works for me; I will put an updated liberfa with that patch to our experimental release to play with astropy 4rc.
The only issue is that eraGetLeapSeconds() and eraSetLeapSeconds() don't have prototypes, which causes a warning when building astropy.

They are defined in src/erfaextra.h, which is included in astropy/_erfa/ufunc.c.templ - should it be added somewhere else?

p.s Great to hear that tests pass, though!!!!

They are defined in src/erfaextra.h

Ah, I somehow missed that file in my patch. Sorry for confusion.
It is great that it works!

Thanks for this, @mhvk! (and @olebole for the testing)

I agree we need a release with this, but I'm a bit concerned about the prospect of including this in erfa before we've had a chance to have users exposed to it. What I would like to do is have a release that's clearly flagged as experimental which we might roll back if there's serious issues.

@olebole - is there any practical path that would allow us to include a "temporary fork" in debian? Or do the version number requirements mean we are stuck with just following the standard release process?

If it's going to be a substantial extra pain to do this, it's not worth it, but just wanted to ask how we might do it before committing to a "normal" release with this.

@eteq - your comment makes me realize we really need to document the non-sofa extras... As a start, shall I use this PR to add a much longer comment in erfaextra.h? That might then include the fact that the leap seconds routines are completely experimental and liable to be changed (but that we welcome feedback on how to make this work independently of astropy).

@eteq while this is technically possible (I could just create a patches version including the changes, as I already did on experimental, I am a bit afraid that this just moves the risk to Debian:
If it turns out later that the API is not accepted generically unchanged, then I would have the problem to support both astropy 4.0 and the updated API. That would be quite a bit unpleasant for me.
For the API, I would really like to have it in the released version, or at least a confirmation that it will go there without change.

@olebole gotcha, I see your point. With that in mind I very much like @mhvk's idea of using erfaextra as something of a "here there be dragons" zone. That makes the prospect of "supporting" it (even on sort of an as-is basis if we go a different direction) quite a bit easier to swallow.

(not all of erfaextras since we have some of the version stuff there, but just that it's a clear wall around where the user should look carefully for experimental things)

Hmm, it still has the danger that I have to maintain an erfadataextra.[ch] it specifically for astropy, or did I misunderstand you?
Couldn't the part that is currently in erfadataextra be integrated in astropy's _erfa/ufunc.c, so that this patch is limited to the public exposure of the eraLEAPSECOND structure? Then the required change here were minimal.

@olebole - the extra routines in erfadatextra.c are to exchange the leap second list used internally in dat.c. I don't think we can move those to astropy.

But I think @eteq's suggestion is in fact to bundle those extra routines in erfa, just have a big warning in the also-bundled erfaextra.h that the leap second and version routines mentioned there are experimental. In that sense, nothing would change from how we do it now.

I do hope that eventually we can provide a formal routine with erfa that allows updating the leap seconds - with perhaps some small helpers that for a given OS/distribution update using the standard system leap second table.

@mhvk OK, I misunderstood the structure here. However, it is not clear to me how I should handle changes in erfaextra.h: In Debian, this structure then would be fixed by the API that is required by Astropy 4.0. So, if it comes out that the API needs a change, I rely on that Astropy also makes the according changes in their stable version 4.0 (which is not obvious). I can't update erfa unless that happened, which is a dependency that I want to avoid.

@olebole - I think we are indeed on the hook at being ultra-careful with changing the infrastructure here, and also ensuring that we get a bit better following SOFA so that packages that rely on sofa/erfa have up-to-date leap seconds even without our astropy work-arounds.

@mhvk, I have no problem if there is a confirmation that the API will not change. Since Astropy 4.0 is going to be an LTS release, I need to maintain its work with Erfa for quite some time, so I think this interface must be somehow stable. That's why I am worried about the statement that these functions are "experimental".

@olebole - indeed, we need to maintain an erfa consistent with astropy 4.0, and I think we can promise that it will stay backward compatible at least for as long as astropy 4.0 is supported (plus, of course, always fully compatible with SOFA). I think our main point is that if we find that the interface of 4.0 is not really suitable, we don't want to be stuck with supporting it beyond the 4.0 LTS cycle.

Of course, it probably is good to make this explicit! I'll do so.

OK, I pushed what I hope will be clearer descriptions. @eteq, @olebole - does this look OK?

@mhvk Thank you very much for your patience and the confirmation! I am happy with this solution. It is always great to see when doubts are taken seriously, even when they may look a bit insisting. That is a real pleasure to cooperate!

Hello! In checking things that are holding up astropy 4.0 (astropy/astropy#9654), I ended up here. What is the timeline to get this merged and a new release? Thanks!

I think this is ready to go in - @eteq, would you like a last look?

Yes, indeed, tests would be good... In principle, if you're willing to test from python using numpy, you could grab the relevant parts of astropy/astropy#9329. Though likely a quick C routine is easier, knowing that fairly stringest tests are done in astropy... Eventually, it may still be easiest to move astropy._erfa over here, as a python module.

Anyway, let me know if I can help.

@mhvk - I just pushed up some commits adding a couple of testing functions. However, the last one (t_leap_seconds) does not work, yielding t_leap_seconds failed - after adding one change, leap second table has 1584767136 entries instead of 43. I'm probably mis-using eraSetLeapSeconds (i.e., that looks like a pointer error of some sort...), but am out of time right now, so maybe you have an easy fix?

Alright, this is looking good now. I'm going to merge this and crank through an ERFA release shortly. Thanks @mhvk

@pllim @olebole @mhvk - release completed with this PR in!