@sunfishcode, @juj, thoughts?

Sounds very reasonable to me, although I'm probably not the expert to comment on the details here. It sounds like we want to pull out call to opt altogether before compilation for speed benefits? Does that look feasible on the long run?

Yes, that's the target - once we no longer call opt, llc and instead just call llc, we'll avoid serializing and deserializing the entire LLVM module. This should be easy after this pull. (Previous step was removing llvm-link, we used to have llvm-link, opt, llc.)

This sounds reasonable to me.

Module serialization/deserialization can change the use-list order. There are command-line parameters to opt which request that it preserve the use-list order, however it's still preferable to fix any bugs due to depending on use-list order anyway, because there is no guarantee that uses will be in any particular order in any case.

Aside from use-list order changes, I noticed that ExpandVarArgs creates a bitcast on each function (that has varargs), as it creates legalized versions of them. But if the function is not used, it still has a use, in that dangling bitcast. Apparently serialization/deserialization fixes that, since the bitcast "goes away". I guess that's a bug in ExpandVarArgs? (I avoided it by making it ignore functions with no uses.)