Convert JIT/Methodical/Array/range Pri0 tests and hook up mobile testing#62752
Conversation
…ting console-based runner functioning.
…Build to support the same mechanisms as the libraries tests for local mobile testing.
…ly and class names.
…cal xharness build
…d on WASM startup.
…runner. Copy the rest ofthe referenced/used assemblies to the output directory. Fix assembly names to match the IL names.
…ts-merged-payload-mobile
|
Tagging subscribers to this area: @dotnet/runtime-infrastructure Issue DetailsConvert a set of il-based Pri0 tests to use the new testing framework and update the runtime test tree's "send to Helix" infra to correctly support merged test payloads for mobile platforms.
|
|
/azp run runtime-manual |
|
Azure Pipelines successfully started running 1 pipeline(s). |
There was a problem hiding this comment.
Looks awesome to me, thanks Jeremy! I'd love to hear whether there are any general patterns we might be able to bulk transform to streamline subsequent refactorings, .assembly and .module seem obvious, my ILTransform also already supports quite a few tricks regarding class name / namespace deduplication or other details. Moving forward, I would also love to give another thought to the _d / _do / _r / _ro suffixes and some way to remove them by somehow encoding the desire to build & test the different codegen options in the metadata but I'm not yet fully there.
…es it (since the target is conditional but the check wasnt)
…nsky/runtime into src-tests-merged-payload-mobile
|
Azure Pipelines successfully started running 1 pipeline(s). |
|
/azp run runtime-manual |
|
Azure Pipelines successfully started running 1 pipeline(s). |
|
Marking this as ready for review as it seems the tests I've migrated over are correctly running in the mobile legs. |
…use in our build.
…ts-merged-payload-mobile
|
/azp run runtime-manual |
|
Azure Pipelines successfully started running 1 pipeline(s). |
|
/azp run runtime-manual |
|
Azure Pipelines successfully started running 1 pipeline(s). |
|
I've validated that all of the new merged runners are running correctly on mobile as well as desktop platforms. This PR is ready for another round of review. |
trylek
left a comment
There was a problem hiding this comment.
Looks great to me, thank you! I admit I'm still not an expert on device testing so that side of your change certainly merits Nathan's and / or Fan's review.
|
/azp run runtime-manual |
|
Azure Pipelines successfully started running 1 pipeline(s). |
|
Failing legs aren't runtime test legs, so I'm going to merge this in. |
|
It seems that with the conversion, the msbuild task to create the mobile apps failed to be triggered. For example, this task https://github.com/dotnet/runtime/blob/main/src/tests/build.proj#L189 |
|
@fanyang-mono it looks like that target is correctly triggered for the "legacy" payloads still: https://dev.azure.com/dnceng/public/_build/results?buildId=1539343&view=logs&j=c1e461de-48bf-5e60-0a97-1d3931972820&t=ca8eb547-1fee-51dd-a4ab-6571ac77788b The merged payloads don't use that target and instead use the same mechanism that the libraries tests use to build and run android test assemblies (iOS and WASM are similar). |
|
@jkoritzinsky so what would be the command to build and run these converted tests locally? I saw that the merged work items ran on Android x64 successfully on helix. Thanks for making it work! I just confirm the IOSSimulator run on runtime-staging (rolling build) that the merged work items ran successfully too. |
|
I believe you can run the |
Convert a set of il-based Pri0 tests to use the new testing framework and update the runtime test tree's "send to Helix" infra to correctly support merged test payloads for mobile platforms.