[nativert] Move TensorMeta to pytorch core by yiming0416 · Pull Request #152475 · pytorch/pytorch

yiming0416 · 2025-04-29T21:52:25Z

Summary:
Torch Native Runtime RFC: pytorch/rfcs#72

This diff moves TensorMeta.cpp and TensorMeta.h to PyTorch core under torch/nativert/graph/

Existing torch::_export::TensorMeta in torch/csrc/utils/generated_serialization_types.h is auto-generated from the export serde schema and therefore only containing the most basic serializable types. We need the newly added TensorMeta.cpp to deserialize the metadata into a in-memory class with c10 types so that it can be consumed by the runtime later.

Test Plan:

Added test under test/cpp/nativert/test_tensor_meta.cpp

Differential Revision: D73820548

pytorch-bot · 2025-04-29T21:52:29Z

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/152475

📄 Preview Python docs built from this PR
📄 Preview C++ docs built from this PR
❓ Need help or want to give feedback on the CI? Visit the bot commands wiki or our office hours

Note: Links to docs will display an error until the docs builds have been completed.

❗ 1 Active SEVs

There are 1 currently active SEVs. If your PR is affected, please view them below:

CI workflows being skipped on PR

✅ You can merge normally! (1 Unrelated Failure)

As of commit 004d31a with merge base 99dac70 ():

BROKEN TRUNK - The following job failed but were present on the merge base:

👉 Rebase onto the `viable/strict` branch to avoid these failures

trunk / macos-py3-arm64 / test (default, 1, 3, macos-m1-stable) (gh) (trunk failure)
'test/dynamo/test_unittest.py::CPythonTest_Assertions::testAssertNotRegex'

This comment was automatically generated by Dr. CI and updates every 15 minutes.

facebook-github-bot · 2025-04-29T21:52:48Z

This pull request was exported from Phabricator. Differential Revision: D73820548

facebook-github-bot · 2025-04-29T22:10:17Z

This pull request was exported from Phabricator. Differential Revision: D73820548

facebook-github-bot · 2025-04-29T22:46:13Z

This pull request was exported from Phabricator. Differential Revision: D73820548

facebook-github-bot · 2025-04-29T23:39:46Z

This pull request was exported from Phabricator. Differential Revision: D73820548

facebook-github-bot · 2025-04-30T17:46:35Z

This pull request was exported from Phabricator. Differential Revision: D73820548

Summary: Pull Request resolved: pytorch#152475 Torch Native Runtime RFC: pytorch/rfcs#72 Thi diff moves `TensorMeta.cpp` and `TensorMeta.h` to PyTorch core. Test Plan: ``` buck2 run mode/dev-nosan caffe2/test/cpp/nativert:nativert ``` Differential Revision: D73820548

albanD · 2025-04-30T17:57:52Z

Can you add some details in the description on how this is different from export::TensorMeta and why we need this separate one?

albanD · 2025-04-30T18:01:42Z

test/cpp/nativert/CMakeLists.txt

+)
+
+# We also build with UBSAN flag in build_asan.h
+if(USE_ASAN)


We actually run this under asan?
Or is this a CMake from somewhere else?

I copied this from test/cpp/jit/CMakeLists.txt. I guess we can remove the USE_ASAN branch if not needed?

Yes let's remove everything we don't need from this cmake

@albanD sure! removed in the latest version.

albanD · 2025-04-30T18:03:39Z

build_variables.bzl

 )

-libtorch_cmake_sources = libtorch_core_sources + libtorch_core_jit_sources
+libtorch_cmake_sources = libtorch_core_sources + libtorch_core_jit_sources + libtorch_nativert_sources


Should we make the runtime a separate .so like other subsets of libtorch to improve dependency management?
In particular, do you expect you users to only link against your function (and transitively get the rest of libtorch). Or you expect them to use whole of libtorch?

cc @malfet

I think we expect the users to use whole of libtorch cc @zhxchen17

I think we expect users to use the whole of libtorch.

yiming0416 · 2025-05-01T15:46:11Z

Can you add some details in the description on how this is different from export::TensorMeta and why we need this separate one?

@albanD Added details in the PR summary

facebook-github-bot · 2025-05-01T17:05:33Z

This pull request was exported from Phabricator. Differential Revision: D73820548

facebook-github-bot · 2025-05-02T17:00:38Z

This pull request was exported from Phabricator. Differential Revision: D73820548

Summary: Pull Request resolved: pytorch#152475 Torch Native Runtime RFC: pytorch/rfcs#72 Thi diff moves `TensorMeta.cpp` and `TensorMeta.h` to PyTorch core. Test Plan: Internal CI. GitHub CI with newly added test under `test/cpp/nativert/test_tensor_meta.cpp` Differential Revision: D73820548

facebook-github-bot · 2025-05-02T18:30:58Z

This pull request was exported from Phabricator. Differential Revision: D73820548

Summary: Pull Request resolved: pytorch#152475 Torch Native Runtime RFC: pytorch/rfcs#72 Thi diff moves `TensorMeta.cpp` and `TensorMeta.h` to PyTorch core. Test Plan: Internal CI. GitHub CI with newly added test under `test/cpp/nativert/test_tensor_meta.cpp` Differential Revision: D73820548

facebook-github-bot · 2025-05-03T01:02:20Z

This pull request was exported from Phabricator. Differential Revision: D73820548

Summary: Pull Request resolved: pytorch#152475 Torch Native Runtime RFC: pytorch/rfcs#72 Thi diff moves `TensorMeta.cpp` and `TensorMeta.h` to PyTorch core. Test Plan: Internal CI. GitHub CI with newly added test under `test/cpp/nativert/test_tensor_meta.cpp` Differential Revision: D73820548

yiming0416 · 2025-05-05T17:12:33Z

@albanD Can you pls review again? I have made APIs private and added the corresponding source file to the test cmake file. If there is any other suggesting changes before merging, I'd be happy to make those. Thanks!

albanD

Sounds ok!

albanD · 2025-05-05T18:35:42Z

torch/nativert/graph/TensorMeta.cpp

+c10::ScalarType convertJsonScalarType(
+    const torch::_export::ScalarType& scalarType) {
+  switch (scalarType) {
+    case torch::_export::ScalarType::UNKNOWN:
+      TORCH_CHECK(false, "scalar type is not properly set");
+    case torch::_export::ScalarType::BYTE:
+      return c10::ScalarType::Byte;
+    case torch::_export::ScalarType::CHAR:
+      return c10::ScalarType::Char;
+    case torch::_export::ScalarType::SHORT:
+      return c10::ScalarType::Short;
+    case torch::_export::ScalarType::INT:
+      return c10::ScalarType::Int;
+    case torch::_export::ScalarType::LONG:
+      return c10::ScalarType::Long;
+    case torch::_export::ScalarType::HALF:
+      return c10::ScalarType::Half;
+    case torch::_export::ScalarType::FLOAT:
+      return c10::ScalarType::Float;
+    case torch::_export::ScalarType::DOUBLE:
+      return c10::ScalarType::Double;
+    case torch::_export::ScalarType::COMPLEXHALF:
+      return c10::ScalarType::ComplexHalf;
+    case torch::_export::ScalarType::COMPLEXFLOAT:
+      return c10::ScalarType::ComplexFloat;
+    case torch::_export::ScalarType::COMPLEXDOUBLE:
+      return c10::ScalarType::ComplexDouble;
+    case torch::_export::ScalarType::BOOL:
+      return c10::ScalarType::Bool;
+    case torch::_export::ScalarType::BFLOAT16:
+      return c10::ScalarType::BFloat16;
+    case torch::_export::ScalarType::UINT16:
+      return c10::ScalarType::UInt16;
+    case torch::_export::ScalarType::FLOAT8E4M3FN:
+      return c10::ScalarType::Float8_e4m3fn;
+    case torch::_export::ScalarType::FLOAT8E5M2:
+      return c10::ScalarType::Float8_e5m2;
+    default:
+      TORCH_CHECK(false, "unknown scalar type", static_cast<int>(scalarType));
+  }
+}
+
+c10::MemoryFormat convertJsonMemoryFormat(
+    const torch::_export::MemoryFormat& memoryFormat) {
+  switch (memoryFormat) {
+    case torch::_export::MemoryFormat::Unknown:
+      TORCH_CHECK(false, "got unknown scalar type");
+    case torch::_export::MemoryFormat::ContiguousFormat:
+      return c10::MemoryFormat::Contiguous;
+    case torch::_export::MemoryFormat::ChannelsLast:
+      return c10::MemoryFormat::ChannelsLast;
+    case torch::_export::MemoryFormat::ChannelsLast3d:
+      return c10::MemoryFormat::ChannelsLast3d;
+    case torch::_export::MemoryFormat::PreserveFormat:
+      return c10::MemoryFormat::Preserve;
+    default:
+      TORCH_CHECK(
+          false, "unknown memory format", static_cast<int>(memoryFormat));
+  }
+}
+
+c10::Layout convertJsonLayout(const torch::_export::Layout& layout) {
+  switch (layout) {
+    case torch::_export::Layout::Unknown:
+      TORCH_CHECK(false, "got unknown layout");
+    case torch::_export::Layout::SparseCoo:
+      // TODO is this the right translation
+      return c10::Layout::Sparse;
+    case torch::_export::Layout::SparseCsr:
+      return c10::Layout::SparseCsr;
+    case torch::_export::Layout::SparseCsc:
+      return c10::Layout::SparseCsc;
+    case torch::_export::Layout::SparseBsr:
+      return c10::Layout::SparseBsr;
+    case torch::_export::Layout::SparseBsc:
+      return c10::Layout::SparseBsc;
+    case torch::_export::Layout::_mkldnn:
+      return c10::Layout::Mkldnn;
+    case torch::_export::Layout::Strided:
+      return c10::Layout::Strided;
+    default:
+      TORCH_CHECK(false, "unknown layout", static_cast<int>(layout));
+  }
+}
+
+c10::Device convertJsonDevice(const torch::_export::Device& device) {
+  c10::Device d(device.get_type());
+  if (auto index = device.get_index()) {
+    d.set_index(*index);
+  }
+  return d;
+}


Ok, as long as we don't make them public from here, that sounds like a good tradeoff for now

facebook-github-bot · 2025-05-05T18:45:05Z

This pull request was exported from Phabricator. Differential Revision: D73820548

facebook-github-bot · 2025-05-05T19:48:54Z

This pull request was exported from Phabricator. Differential Revision: D73820548

Summary: Pull Request resolved: pytorch#152475 Torch Native Runtime RFC: pytorch/rfcs#72 Thi diff moves `TensorMeta.cpp` and `TensorMeta.h` to PyTorch core. Test Plan: Internal CI. GitHub CI with newly added test under `test/cpp/nativert/test_tensor_meta.cpp` Reviewed By: zhxchen17 Differential Revision: D73820548

facebook-github-bot · 2025-05-05T20:07:34Z

This pull request was exported from Phabricator. Differential Revision: D73820548

facebook-github-bot · 2025-05-06T01:43:19Z

@pytorchbot merge

(Initiating merge automatically since Phabricator Diff has merged)

pytorchmergebot · 2025-05-06T01:45:07Z

Merge started

Your change will be merged once all checks pass (ETA 0-4 Hours).

Learn more about merging in the wiki.

Questions? Feedback? Please reach out to the PyTorch DevX Team

Advanced Debugging

Check the merge workflow status
here

facebook-github-bot added the fb-exported label Apr 29, 2025

yiming0416 marked this pull request as draft April 29, 2025 21:53

yiming0416 added the topic: not user facing topic category label Apr 29, 2025

yiming0416 force-pushed the export-D73820548 branch from c5fee75 to c815ce9 Compare April 29, 2025 22:10

yiming0416 force-pushed the export-D73820548 branch from c815ce9 to c7b3295 Compare April 29, 2025 22:46

yiming0416 force-pushed the export-D73820548 branch from c7b3295 to fd8fa87 Compare April 29, 2025 23:39

yiming0416 marked this pull request as ready for review April 30, 2025 16:49

yiming0416 requested review from albanD, malfet, seemethere and zhxchen17 April 30, 2025 16:50

yiming0416 added the ciflow/trunk Trigger trunk jobs on your pull request label Apr 30, 2025

yiming0416 force-pushed the export-D73820548 branch from fd8fa87 to a715f62 Compare April 30, 2025 17:46

albanD reviewed Apr 30, 2025

View reviewed changes

yiming0416 requested a review from albanD May 1, 2025 15:45

yiming0416 force-pushed the export-D73820548 branch from a715f62 to d1ec045 Compare May 1, 2025 17:05

yiming0416 force-pushed the export-D73820548 branch from a5a12ef to d72e177 Compare May 2, 2025 17:00

yiming0416 force-pushed the export-D73820548 branch from d72e177 to a794bb9 Compare May 2, 2025 18:31

yiming0416 force-pushed the export-D73820548 branch from a794bb9 to 5f4e9ea Compare May 2, 2025 19:00

yiming0416 requested a review from albanD May 2, 2025 20:00

yiming0416 force-pushed the export-D73820548 branch from 5f4e9ea to ff54b0b Compare May 3, 2025 01:02

albanD approved these changes May 5, 2025

View reviewed changes

yiming0416 force-pushed the export-D73820548 branch from ff54b0b to 3a7c541 Compare May 5, 2025 18:45

yiming0416 force-pushed the export-D73820548 branch from 3a7c541 to a66d52d Compare May 5, 2025 19:48

yiming0416 force-pushed the export-D73820548 branch from a66d52d to 004d31a Compare May 5, 2025 20:07

pytorchmergebot added the merging label May 6, 2025

pytorchmergebot closed this in 1d77280 May 6, 2025

pytorchmergebot added Merged and removed merging labels May 6, 2025

Conversation

yiming0416 commented Apr 29, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

pytorch-bot bot commented Apr 29, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/152475

❗ 1 Active SEVs

✅ You can merge normally! (1 Unrelated Failure)

Uh oh!

facebook-github-bot commented Apr 29, 2025

Uh oh!

facebook-github-bot commented Apr 29, 2025

Uh oh!

facebook-github-bot commented Apr 29, 2025

Uh oh!

facebook-github-bot commented Apr 29, 2025

Uh oh!

facebook-github-bot commented Apr 30, 2025

Uh oh!

albanD commented Apr 30, 2025

Uh oh!

albanD Apr 30, 2025

Choose a reason for hiding this comment

Uh oh!

yiming0416 Apr 30, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

albanD May 1, 2025

Choose a reason for hiding this comment

Uh oh!

yiming0416 May 1, 2025

Choose a reason for hiding this comment

Uh oh!

albanD Apr 30, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

yiming0416 Apr 30, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

zhxchen17 May 1, 2025

Choose a reason for hiding this comment

Uh oh!

yiming0416 commented May 1, 2025

Uh oh!

facebook-github-bot commented May 1, 2025

Uh oh!

facebook-github-bot commented May 2, 2025

Uh oh!

facebook-github-bot commented May 2, 2025

Uh oh!

facebook-github-bot commented May 3, 2025

Uh oh!

yiming0416 commented May 5, 2025

Uh oh!

albanD left a comment

Choose a reason for hiding this comment

Uh oh!

albanD May 5, 2025

Choose a reason for hiding this comment

Uh oh!

facebook-github-bot commented May 5, 2025

Uh oh!

facebook-github-bot commented May 5, 2025

Uh oh!

facebook-github-bot commented May 5, 2025

Uh oh!

facebook-github-bot commented May 6, 2025

Uh oh!

pytorchmergebot commented May 6, 2025

Merge started

Uh oh!

Reviewers

Assignees

Labels

Projects

yiming0416 commented Apr 29, 2025 •

edited

Loading

pytorch-bot bot commented Apr 29, 2025 •

edited

Loading

yiming0416 Apr 30, 2025 •

edited

Loading

albanD Apr 30, 2025 •

edited

Loading

yiming0416 Apr 30, 2025 •

edited

Loading