Conversation
🔗 Helpful Links🧪 See artifacts and rendered test results at hud.pytorch.org/pr/141087
Note: Links to docs will display an error until the docs builds have been completed. ❗ 1 Active SEVsThere are 1 currently active SEVs. If your PR is affected, please view them below: ✅ No FailuresAs of commit 16a4c61 with merge base 4f2543c ( This comment was automatically generated by Dr. CI and updates every 15 minutes. |
This is the format used by ClickHouse, so it's useful to support this too in addition to a valid JSON list of benchmark records. ### Testing * Adding a test case https://github.com/pytorch/test-infra/actions/runs/11927946564/job/33244134505#step:5:124 * I'm also testing this in pytorch/pytorch#141087
|
@pytorchbot merge -f "Lint is green and then job is periodic anyway" |
Merge startedYour change will be merged immediately since you used the force (-f) flag, bypassing any CI checks (ETA: 1-5 minutes). Please use Learn more about merging in the wiki. Questions? Feedback? Please reach out to the PyTorch DevX Team |
A small fix I just realize after #141087. Pull Request resolved: #141145 Approved by: https://github.com/malfet
Summary: This uploads the MPS benchmark results to benchmark database. The data can then be queried, for example: ``` select benchmark, model, metric from oss_ci_benchmark_v3 where head_sha = '99a133116fee15aa1467165f2b209b37da53f189' and metric.name in ['eager_peak_mem', 'dynamo_peak_mem', 'speedup'] and model.name = 'BERT_pytorch' ``` I'm documenting the JSON format at https://github.com/pytorch/pytorch/wiki/How-to-integrate-with-PyTorch-OSS-benchmark-database ### Testing Locally, ``` PYTHONPATH=/Users/huydo/Storage/mine/benchmark python benchmarks/dynamo/torchbench.py --performance --only resnet152 --backend eager --training --devices mps --output test/test-reports/torchbench_training.csv ``` Workflow dispatch https://github.com/pytorch/pytorch/actions/runs/11927990520 X-link: pytorch/pytorch#141087 Approved by: https://github.com/malfet Reviewed By: izaitsevfb Differential Revision: D66262910 Pulled By: huydhn fbshipit-source-id: 318fbb0ded0e85742aa441e023f5abf906398997
Summary: A small fix I just realize after pytorch/pytorch#141087. X-link: pytorch/pytorch#141145 Approved by: https://github.com/malfet Reviewed By: izaitsevfb Differential Revision: D66262943 Pulled By: huydhn fbshipit-source-id: 4ccec00d23bac753fbba3ad1a39de1569efd11f8
) Similar to #141087, this uploads the micro benchmark results to benchmark database with its new schema v3. The data can then be queried. ~I'm testing with `inductor-micro-benchmark-x86` which should be sufficient because `inductor-micro-benchmark` is broken atm. The CSV output stays for now until the dashboard is migrated to schema v3.~ #141747 has been resolved, so inductor-micro-benchmark should work now Pull Request resolved: #141745 Approved by: https://github.com/yanboliang
This uploads the MPS benchmark results to benchmark database. The data can then be queried, for example: ``` select benchmark, model, metric from oss_ci_benchmark_v3 where head_sha = '99a133116fee15aa1467165f2b209b37da53f189' and metric.name in ['eager_peak_mem', 'dynamo_peak_mem', 'speedup'] and model.name = 'BERT_pytorch' ``` I'm documenting the JSON format at https://github.com/pytorch/pytorch/wiki/How-to-integrate-with-PyTorch-OSS-benchmark-database ### Testing Locally, ``` PYTHONPATH=/Users/huydo/Storage/mine/benchmark python benchmarks/dynamo/torchbench.py --performance --only resnet152 --backend eager --training --devices mps --output test/test-reports/torchbench_training.csv ``` Workflow dispatch https://github.com/pytorch/pytorch/actions/runs/11927990520 Pull Request resolved: pytorch#141087 Approved by: https://github.com/malfet
…#141145) A small fix I just realize after pytorch#141087. Pull Request resolved: pytorch#141145 Approved by: https://github.com/malfet
…rch#141745) Similar to pytorch#141087, this uploads the micro benchmark results to benchmark database with its new schema v3. The data can then be queried. ~I'm testing with `inductor-micro-benchmark-x86` which should be sufficient because `inductor-micro-benchmark` is broken atm. The CSV output stays for now until the dashboard is migrated to schema v3.~ pytorch#141747 has been resolved, so inductor-micro-benchmark should work now Pull Request resolved: pytorch#141745 Approved by: https://github.com/yanboliang
This uploads the MPS benchmark results to benchmark database. The data can then be queried, for example:
I'm documenting the JSON format at https://github.com/pytorch/pytorch/wiki/How-to-integrate-with-PyTorch-OSS-benchmark-database
Testing
Locally,
Workflow dispatch https://github.com/pytorch/pytorch/actions/runs/11927990520
cc @voznesenskym @penguinwu @EikanWang @jgong5 @Guobing-Chen @XiaobingSuper @zhuhaozhe @blzheng @wenzhe-nrv @jiayisunx @chenyang78 @kadeng @chauhang @amjames