This is an optimistic-lock; it expects that multiple threads to be calling non-mutating functions most of the time, allowing them to run concurrently. It behaves like a turnstile, allowing threads that arrived first to pass, and then blocks if another thread calls extend, until it finishes updating the polynomial coefficients.

While this solution may yield dirty-reads for threads that execute before extend, it allows for better concurrency by not serializing each call that depends on the coefficients, which would slowdown the overall execution time when multiple threads are calling instructions different from extend.

An alternative would be to signal in the docs that the API is not thread-safe with respect to extend and let the user handle the concurrency explicitly when a mixture of extend and other calls happen

Not sure I understand this comment, and most certainly a comment is needed not only on GH but also in code.

Also, if it's something to do with the extend method, can the lock move to extend? If not, why it's only in __call__ but not in other functions (.derivative, .integrate etc)

This one still confuses me, I've to admit. Let me try rephrasing my confusion:

1. This with-stanza acquires and releases the lock, so that if two threads arrive simultaneously, one arrives to this line and the other is doing something else, that other one is stopped and waits until the lock is released -- -is this correct? If it is, why is it needed, why is it useful and how does this help anything?

2. The very need for this is because of existence of .extend method which mutates the class state, is this correct? If correct,cannot one lock the .extend internals instead?

3. What is this pattern, is it something known/standard or is it something you conjured out of thin air?

4. Can you actually show a scenario where this is needed? And explain how it works in this scenario?
   Previous discussion seems to suggest something rather esoteric, with threaded use of reads and .extend --- a concrete scenario would really help.

As I mentioned in the first comment, this lock acts like a "turnstile with a block"; imagine n persons entering an amusement park through a turnstile, once they are in, they are independent of each other, despite entering in a serialized fashion. Now, imagine there's a service mode that the turnstile can enter in which it blocks while it goes maintenance, during this time no people can enter the park, until the service mode gets released.

This is the same scenario posed here, what's behind the wall is the code that can be run by each thread independent of each other, assuming that the polynomial coefficients haven't changed at all. What the so-called "service mode" in the analogy does is changing the coefficients while stopping everyone else from accessing them.

When the polynomial undergoes modifications (via extend), any thread that has already got in, will get a dirty read if they are trying to access the polynomial coefficients, that's why is called optimistic in principle, you'd expect that the number of calls to extend be way much lower than the calls to the actual interpolation function. In contrast, a pessimistic approach would be to block and serialize any call to the interpolation function, given that at any time a call to extend would lead to dirty reads for any thread.

While the pessimistic approach guarantees clean reads and consistency, it comes at the expense of performance, since the calls performed by n threads would be effectively serialized. The optimistic approach tries to balance performance at the expense of potential dirty reads that are expected to be low, since the number of calls to extend is expected to occur in a non-concurrent fashion or in few occurrences.

This is more of a thought experiment that can be an exotic scenario in the practice, that's why I posed the question about signaling in the docs that calls to extend should not be interleaved with concurrent calls to any of the interpolation subclasses that inherit from _PPolyBase, since it is not thread-safe.

Okay, the turnstile analogy I get, thank you. That the implementation looks bizarre can be attributed to me not being used to this, so can discount that :-).

What I'm not sure about is the status of this:

• this is a best-effort trick, to remedy some of free-threading usages, correct?
• (if yes) is there a way to estimate how exotic should a user load be for this trick to not work?

Let's consider an example: https://docs.scipy.org/doc/scipy/tutorial/interpolate/extrapolation_examples.html#cubicspline-extend-the-boundary-conditions

• if we keep land this PR as is, how would the example need changing?
• if we land the PR without this turnstile, how would the example need changing?

This mutation goes against the guidelines declared on https://github.com/Quansight-Labs/free-threaded-compatibility?tab=readme-ov-file#suggested-plan-of-attack, also SciPy should not anticipate exotic use cases like the one that initially motivated this whole "optimistic lock" approach.

The resolution in this case was to document the extend function and mark it as not thread-safe when used with interleaved interpolation methods.

Not sure I understand this comment, and most certainly a comment is needed not only on GH but also in code.

Also, if it's something to do with the extend method, can the lock move to extend? If not, why it's only in __call__ but not in other functions (.derivative, .integrate etc)

This one still confuses me, I've to admit. Let me try rephrasing my confusion:

1. This with-stanza acquires and releases the lock, so that if two threads arrive simultaneously, one arrives to this line and the other is doing something else, that other one is stopped and waits until the lock is released -- -is this correct? If it is, why is it needed, why is it useful and how does this help anything?

2. The very need for this is because of existence of .extend method which mutates the class state, is this correct? If correct,cannot one lock the .extend internals instead?

3. What is this pattern, is it something known/standard or is it something you conjured out of thin air?

4. Can you actually show a scenario where this is needed? And explain how it works in this scenario?
   Previous discussion seems to suggest something rather esoteric, with threaded use of reads and .extend --- a concrete scenario would really help.