Skip to content

add model test for Android#74793

Closed
linbinyu wants to merge 6 commits intomasterfrom
linbin/android
Closed

add model test for Android#74793
linbinyu wants to merge 6 commits intomasterfrom
linbin/android

Conversation

@linbinyu
Copy link
Copy Markdown
Contributor

@linbinyu linbinyu commented Mar 26, 2022

This pr

moved some model generation scripts to a central place (mobile net v2, android test model)
updated scripts so these models can run on Android (Java doesn't support certain scalar types as return value)
updated model generation script to take arguments (generate models for android/ios, checked-in model or on-the-fly model)
add Android instrumentation tests for these new models
After this change, the Android instrumentation test will run 35 models which covered 91% of production ops. The coverage information can be found in this file: https://github.com/pytorch/pytorch/blob/master/test/mobile/model_test/coverage.yaml

Note that these models are checked-in for back compatibility check (to ensure they can run with newer pytorch versions).

The script generates models for mobile test. For each model we have a "checked-in" version
and an "on-the-fly" version. The "on-the-fly" version will be generated during test, and
should not be committed to the repo. The "checked-in" version is used for back compatibility check.

Note that Android only support checked-in model right now. iOS can test both (in another pr).

use 'gen_test_model.py android-test' to generate on the fly models for android
use 'gen_test_model.py ios-test' to generate on the fly models for ios
use 'python gen_test_model.py android' to generate checked-in models for android
use 'python gen_test_model.py ios' to generate on-the-fly models for ios
use 'gen_test_model.py <model_name_no_suffix>' to update the given checked-in model

@facebook-github-bot
Copy link
Copy Markdown
Contributor

facebook-github-bot commented Mar 26, 2022

🔗 Helpful links

💊 CI failures summary and remediations

As of commit cb79ff3 (more details on the Dr. CI page):


💚 💚 Looks good so far! There are no failures yet. 💚 💚


This comment was automatically generated by Dr. CI (expand for details).

Please report bugs/suggestions to the (internal) Dr. CI Users group.

Click here to manually regenerate this comment.

@linbinyu
Copy link
Copy Markdown
Contributor Author

@pytorchmergebot merge this

@pytorchmergebot
Copy link
Copy Markdown
Collaborator

Merge failed due to PR 74793 does not match merge rules
Raised by https://github.com/pytorch/pytorch/actions/runs/2055342073

@linbinyu
Copy link
Copy Markdown
Contributor Author

@pytorchmergebot merge this

@github-actions
Copy link
Copy Markdown
Contributor

Hey @linbinyu.
You've committed this PR, but it does not have both a 'release notes: ...' and 'topics: ...' label. Please add one of each to the PR. The 'release notes: ...' label should represent the part of PyTorch that this PR changes (fx, autograd, distributed, etc) and the 'topics: ...' label should represent the kind of PR it is (not user facing, new feature, bug fix, perf improvement, etc). The list of valid labels can be found here for the 'release notes: ...' and here for the 'topics: ...'.
For changes that are 'topic: not user facing' there is no need for a release notes label.

@seemethere
Copy link
Copy Markdown
Member

@pytorchbot revert this, broke internal formatters

pytorchmergebot added a commit that referenced this pull request Mar 29, 2022
pytorchmergebot pushed a commit that referenced this pull request Mar 30, 2022
See #74793 for more details
Redo everything in one pr. This diff added test model generation script and updated android and ios tests.
Pull Request resolved: #74947
Approved by: https://github.com/kit1980
facebook-github-bot pushed a commit that referenced this pull request Mar 31, 2022
Summary:
See #74793 for more details
Redo everything in one pr. This diff added test model generation script and updated android and ios tests.

Pull Request resolved: #74947
Approved by: https://github.com/kit1980

allow-large-files

Test Plan: contbuild & OSS CI, see https://hud.pytorch.org/commit/pytorch/pytorch/28a4b4759add0b3eb0f4c34a1e042be4f93e976e

Reviewed By: atalman

Differential Revision: D35284140

Pulled By: linbinyu

fbshipit-source-id: 50c73e006554b8e4301275599433b69950ff9e33
@github-actions github-actions Bot deleted the linbin/android branch February 15, 2024 01:56
laurentdupin pushed a commit to laurentdupin/pytorch that referenced this pull request Apr 25, 2026
This pr

moved some model generation scripts to a central place (mobile net v2, android test model)
updated scripts so these models can run on Android (Java doesn't support certain scalar types as return value)
updated model generation script to take arguments (generate models for android/ios, checked-in model or on-the-fly model)
add Android instrumentation tests for these new models
After this change, the Android instrumentation test will run 35 models which covered 91% of production ops. The coverage information can be found in this file: https://github.com/pytorch/pytorch/blob/master/test/mobile/model_test/coverage.yaml

Note that these models are checked-in for back compatibility check (to ensure they can run with newer pytorch versions).

The script generates models for mobile test. For each model we have a "checked-in" version
and an "on-the-fly" version. The "on-the-fly" version will be generated during test, and
should not be committed to the repo. The "checked-in" version is used for back compatibility check.

Note that Android only support checked-in model right now. iOS can test both (in another pr).

use 'gen_test_model.py android-test' to generate on the fly models for android
use 'gen_test_model.py ios-test' to generate on the fly models for ios
use 'python gen_test_model.py android' to generate checked-in models for android
use 'python gen_test_model.py ios' to generate on-the-fly models for ios
use 'gen_test_model.py <model_name_no_suffix>' to update the given checked-in model

Pull Request resolved: pytorch#74793
Approved by: https://github.com/kit1980
laurentdupin pushed a commit to laurentdupin/pytorch that referenced this pull request Apr 25, 2026
laurentdupin pushed a commit to laurentdupin/pytorch that referenced this pull request Apr 25, 2026
See pytorch#74793 for more details
Redo everything in one pr. This diff added test model generation script and updated android and ios tests.
Pull Request resolved: pytorch#74947
Approved by: https://github.com/kit1980
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants