Parallelize superpmi-diffs pipeline per collection #106759
Conversation
Submit helix items for each collection to speed up the superpmi-diffs runs.
|
cc @dotnet/jit-contrib Example run with some test JIT changes (+ limiting the collections to a smaller subset to speed up the dev loop): https://dev.azure.com/dnceng-public/public/_build/results?buildId=784894&view=results |
| self.host_os = coreclr_args.host_os | ||
| self.target_os = coreclr_args.target_os | ||
| self.arch_name = coreclr_args.arch | ||
| self.host_os = "windows" if platform.system() == "Windows" else "linux" |
There was a problem hiding this comment.
superpmi-diffs only supports Windows host (asmdiffs targetting all OS/archs, tpdiffs with MSVC) and Linux host (tpdiffs with Clang)
|
Another issue is that the mch downloading is too slow. Most of the time is spent here. |
That's not true. We don't spend more than a couple of minutes downloading the collections. |
This PR changes superpmi-diffs such that it submits a Helix work item for each collection that we will be collecting asm diffs or tp diffs for.
superpmi_diffs_setup.pythat creates a .json file for every (target OS, target arch, collection) tuple--summary_as_jsonCLI arg tosuperpmi.py asmdiffs/tpdiff. When passed,superpmi.pywill output the information necessary to generate a .md summary as JSON instead. These JSON files are now the results of the Helix work items.superpmi.py summarizecommand. This command takes a collection of JSON summary files and generates a consolidated .md summary in the same format as before.superpmi_diffs_summarize.pyto consolidate the JSON summaries created for each (target OS, target arch, collection) tuple into a consolidated .md for each (target OS, target arch), before doing the final consolidation to the single overall (host OS, host arg) .md file that is shown on AzDO.With this PR superpmi-diffs is significantly faster. Actual time spent doing useful work (building and running superpmi) is only around 30 minutes total. The remaining CI time is spent waiting for AzDO runners to pick up the pipeline jobs, and for the Helix queues to pick up the Helix work items.
An end-to-end superpmi-diffs run takes a little over 1 hour now, in comparison to up to 3 hours before.