Add flag _metrics_log_runtime to disable runtime metric logging by default#153506
Add flag _metrics_log_runtime to disable runtime metric logging by default#153506exclamaforte wants to merge 11 commits intomainfrom
flag _metrics_log_runtime to disable runtime metric logging by default#153506Conversation
🔗 Helpful Links🧪 See artifacts and rendered test results at hud.pytorch.org/pr/153506
Note: Links to docs will display an error until the docs builds have been completed. ✅ No FailuresAs of commit e3f7169 with merge base 72a3c8d ( This comment was automatically generated by Dr. CI and updates every 15 minutes. |
flag _metrics_log_runtime to disable runtime metric logging by default
| metrics.num_bytes_accessed += num_bytes | ||
| metrics.node_runtimes += node_runtimes | ||
| metrics.nodes_num_elem += nodes_num_elem | ||
| if config._metrics_log_runtime: |
There was a problem hiding this comment.
Should we just move this to TORCH_LOGS="inductor_metrics" ? it's a small change to add the logs - see https://github.com/pytorch/pytorch/pull/147248/files
2739678 to
7a15fa1
Compare
de5d03a to
f5b3f92
Compare
| "inductor backend is not available", | ||
| ) | ||
| def test_save_and_load_inductor(self): | ||
| torch._logging.set_logs(inductor_metrics=True) |
|
@pytorchbot merge |
Merge startedYour change will be merged once all checks pass (ETA 0-4 Hours). Learn more about merging in the wiki. Questions? Feedback? Please reach out to the PyTorch DevX Team |
Merge failedReason: 1 mandatory check(s) failed. The first few are: Dig deeper by viewing the failures on hud |
|
@pytorchbot merge |
Merge startedYour change will be merged once all checks pass (ETA 0-4 Hours). Learn more about merging in the wiki. Questions? Feedback? Please reach out to the PyTorch DevX Team |
Merge failedReason: 1 mandatory check(s) failed. The first few are: Dig deeper by viewing the failures on hud |
cfc623b to
e3f7169
Compare
|
@pytorchbot merge |
Merge startedYour change will be merged once all checks pass (ETA 0-4 Hours). Learn more about merging in the wiki. Questions? Feedback? Please reach out to the PyTorch DevX Team |
#152708 expanded support of
get_estimated_runtimeto many more types ofSchedulerNodes. This caused an increase in compile time because we're always callingget_estimated_runtimeto populate the metrics table. This PR adds a flag for this logging, which reduces the instruction count by 8%. Long term, we should probably merge metrics.py with TORCH_LOGS/tlparse (suggestion from @xmfan).Update: added support for TORCH_LOGS for the metrics logging.
Test Plan:
mm_loop.py and many existing tests cover.
cc @voznesenskym @penguinwu @EikanWang @jgong5 @Guobing-Chen @XiaobingSuper @zhuhaozhe @blzheng @wenzhe-nrv @jiayisunx @ipiszy @chenyang78 @kadeng @muchulee8 @amjames @chauhang @aakhundov