CI seemed to fail on lots of things here, despite working fine on my build machine. Lots of timeouts...

Well at least it passed on Firefox (the run_webworker and reentrant_error tests regularly fail on main). It must be crashing chrome. I don't have any good way of understanding what is going on when code works on one browser and totally breaks the other though.

One recent problem I hit is that chrome is more delicate about pointer alignment than firefox. The following code crashes chrome consistently (though it is also noticeably slow on firefox):

EM_JS(void, my_func, (), {
	  let addr = _malloc(8);
	  let byte_ptr = ptr1 + 3;
	  let ptr_ptr = addr + 4;
	  HEAP8[byte_ptr] = 7;
	  HEAPU32[ptr_ptr/4] = byte_ptr;
	  _print_byte_ptrptr(ptr_ptr);
});

void print_byte_ptrptr(char **p){
	printf("byte is %x\n", **p);
}

Oops didn't mean to push here, sorry.

I synced with main after patches were merged in #1708. There were still a few failures even with the latest CI setup that look genuine to me though.

Well one problem is that libffi needs to be updated. I think we can just remove one of the compile flags from libffi since my port uses #ifdef to handle different fpcast options.

@rth On circleci I keep getting:

make[1]: *** [Makefile:2020: Modules/unicodedata.o] Killed

each time in the same place, at around 11 minutes and 20 seconds +/- a bit. Sometimes it doesn't happen. Related to #2013?

@rth Also, is there a way to manually flush all ccaches when merging this? I changed the compiler, so all caches need to be invalidated, but I think the cache invalidation mechanism only looks for changes in the source, not for changes in the compiler.

Oh I think the problem might be that it uses up too much ram loading the AST for unicodedata.o. The AST json dump unicodedata.c.ast is a gigabyte! Maybe I should check whether the ast is super big and if so skip the patching step...

Another option would be to use a compiled language to parse the ast or something. Sounds annoying though.

unicodedata.c has 8000 lines of static data. The ast really blows that up by recording a bunch of location information for each 0.

Also, is there a way to manually flush all ccaches when merging this?

Normally it hashes Makefile.env for the cache key, so you should be OK. Otherwise you can bump the date in

The AST json dump unicodedata.c.ast is a gigabyte! Maybe I should check whether the ast is super big and if so skip the patching step...

Any way to process it with a generator without loading it all in memory?

Any way to process it with a generator without loading it all in memory?

Ironically processing the human-readable AST made this a lot easier. But the best approach is probably to use jq.

https://github.com/stedolan/jq/wiki/FAQ#streaming-json-parser

I think this is working now, but am having trouble with build timeouts.
But I just thought of a different approach to this which might be easier. Basically, we should just patch call trampolines into the Python interpreter in a few places rather than trying to fix the package source code.

Closed in favor of #2019.