@the-moisrex The problem might be algorithmic.

The extractor follows this algorithm:

1. Visit a key in the JSON. Compare it against all target keys until one is found.
2. Move to the next key, go back to 1.

In some instances, this is fine.

However, look how the reference ondemand code is like in the two instances that I have identified. We do...

1. 'find_field("first key")' this moves you through the JSON up until you have found the key. In JSON files, the keys are often in predictible order and so we might be lucky and hit it first.
2. we do 'find_field("second key")'... this starts right after 'first key'.

And so on.

When the code is written to match the order of the keys in the JSON... this second approach is going to be more efficient than the extractor code.

So, possibly, we could use a hint parameter to choose the algorithm you want to use. If you think you know the order of the keys, you could use one algorithm, otherwise you could use another... ???

However, look how the reference ondemand code is like in the two instances that I have identified. We do...

• 'find_field("first key")' this moves you through the JSON up until you have found the key. In JSON files, the keys are often in predictible order and so we might be lucky and hit it first.
  we do 'find_field("second key")'... this starts right after 'first key'.

And so on.

@lemire that was the idea from which I optimized find_all as well; but it wasn't as clean as I wanted it; let me add a feature to the iterator itself that finds the next key instead of searching for an specific key and leave the equality checking to us.

So, possibly, we could use a hint parameter to choose the algorithm you want to use. If you think you know the order of the keys, you could use one algorithm, otherwise you could use another... ???

I don't think a hint is a good idea. Is there a scenario that the current algorithm would perform better?

I'm gonna merge this.

@lemire See simdjson#2247 (comment)

@the-moisrex Conceptually, there might be different algorithms that are best. But I agree that your latest design is quite good and likely difficult to improve upon.