Skip to content

[package] implement get_resource_reader API#51674

Closed
suo wants to merge 7 commits intogh/suo/391/basefrom
gh/suo/391/head
Closed

[package] implement get_resource_reader API#51674
suo wants to merge 7 commits intogh/suo/391/basefrom
gh/suo/391/head

Conversation

@suo
Copy link
Copy Markdown
Member

@suo suo commented Feb 3, 2021

Stack from ghstack:

See
https://docs.python.org/3/library/importlib.html#importlib.abc.ResourceReader

Differential Revision: D26237034

@facebook-github-bot facebook-github-bot added cla signed oncall: jit Add this issue/PR to JIT oncall triage queue labels Feb 3, 2021
@facebook-github-bot
Copy link
Copy Markdown
Contributor

facebook-github-bot commented Feb 3, 2021

💊 CI failures summary and remediations

As of commit 111ae91 (more details on the Dr. CI page):


  • 7/7 failures possibly* introduced in this PR
    • 2/7 non-scanned failure(s)

🕵️ 4 new failures recognized by patterns

The following CI failures do not appear to be due to upstream breakages:

See CircleCI build pytorch_linux_xenial_py3_6_gcc5_4_test (1/4)

Step: "Run tests" (full log | diagnosis details | 🔁 rerun)

Mar 10 18:40:06 [E request_callback_no_python.cpp:656] Received error while processing request type 256: The following operation failed in the TorchScript interpreter.
Mar 10 18:40:06 [E request_callback_no_python.cpp:656] Received error while processing request type 256: The following operation failed in the TorchScript interpreter.
Mar 10 18:40:06 Traceback of TorchScript (most recent call last):
Mar 10 18:40:06   File "/opt/conda/lib/python3.6/site-packages/torch/testing/_internal/distributed/rpc/jit/rpc_test.py", line 825, in run_ref_script_module
Mar 10 18:40:06     ref_script_module: RRef[MyModuleInterface], t: Tensor
Mar 10 18:40:06 ) -> Tensor:
Mar 10 18:40:06     module = ref_script_module.to_here()
Mar 10 18:40:06              ~~~~~~~~~~~~~~~~~~~~~~~~~ <--- HERE
Mar 10 18:40:06     return module.forward() + t
Mar 10 18:40:06 RuntimeError: UserRRef(rref_id=GloballyUniqueId(created_on=1, local_id=0), fork_id=GloballyUniqueId(created_on=1, local_id=3)) is an RRef to a ScriptModule. It can't be sent through RPC from owner, WorkerInfo(id=2, name=worker2), to user, WorkerInfo(id=1, name=worker1).
Mar 10 18:40:06 
Mar 10 18:40:06 [E request_callback_no_python.cpp:656] Received error while processing request type 256: The following operation failed in the TorchScript interpreter.
Mar 10 18:40:06 Traceback of TorchScript (most recent call last):
Mar 10 18:40:06   File "/opt/conda/lib/python3.6/site-packages/torch/testing/_internal/distributed/rpc/jit/rpc_test.py", line 825, in run_ref_script_module
Mar 10 18:40:06     ref_script_module: RRef[MyModuleInterface], t: Tensor
Mar 10 18:40:06 ) -> Tensor:
Mar 10 18:40:06     module = ref_script_module.to_here()
Mar 10 18:40:06              ~~~~~~~~~~~~~~~~~~~~~~~~~ <--- HERE
Mar 10 18:40:06     return module.forward() + t
Mar 10 18:40:06 RuntimeError: UserRRef(rref_id=GloballyUniqueId(created_on=3, local_id=0), fork_id=GloballyUniqueId(created_on=3, local_id=3)) is an RRef to a ScriptModule. It can't be sent through RPC from owner, WorkerInfo(id=0, name=worker0), to user, WorkerInfo(id=3, name=worker3).
Mar 10 18:40:06 
Mar 10 18:40:06 ok (1.940s)

See CircleCI build pytorch_linux_xenial_cuda10_2_cudnn7_py3_gcc7_test1 (2/4)

Step: "Run tests" (full log | diagnosis details | 🔁 rerun)

Mar 10 19:00:48 [E request_callback_no_python.cpp:656] Received error while processing request type 256: The following operation failed in the TorchScript interpreter.
Mar 10 19:00:48 [E request_callback_no_python.cpp:656] Received error while processing request type 256: The following operation failed in the TorchScript interpreter.
Mar 10 19:00:48 Traceback of TorchScript (most recent call last):
Mar 10 19:00:48   File "/opt/conda/lib/python3.6/site-packages/torch/testing/_internal/distributed/rpc/jit/rpc_test.py", line 825, in run_ref_script_module
Mar 10 19:00:48     ref_script_module: RRef[MyModuleInterface], t: Tensor
Mar 10 19:00:48 ) -> Tensor:
Mar 10 19:00:48     module = ref_script_module.to_here()
Mar 10 19:00:48              ~~~~~~~~~~~~~~~~~~~~~~~~~ <--- HERE
Mar 10 19:00:48     return module.forward() + t
Mar 10 19:00:48 RuntimeError: UserRRef(rref_id=GloballyUniqueId(created_on=0, local_id=0), fork_id=GloballyUniqueId(created_on=0, local_id=3)) is an RRef to a ScriptModule. It can't be sent through RPC from owner, WorkerInfo(id=1, name=worker1), to user, WorkerInfo(id=0, name=worker0).
Mar 10 19:00:48 
Mar 10 19:00:48 [E request_callback_no_python.cpp:656] Received error while processing request type 256: The following operation failed in the TorchScript interpreter.
Mar 10 19:00:48 Traceback of TorchScript (most recent call last):
Mar 10 19:00:48   File "/opt/conda/lib/python3.6/site-packages/torch/testing/_internal/distributed/rpc/jit/rpc_test.py", line 825, in run_ref_script_module
Mar 10 19:00:48     ref_script_module: RRef[MyModuleInterface], t: Tensor
Mar 10 19:00:48 ) -> Tensor:
Mar 10 19:00:48     module = ref_script_module.to_here()
Mar 10 19:00:48              ~~~~~~~~~~~~~~~~~~~~~~~~~ <--- HERE
Mar 10 19:00:48     return module.forward() + t
Mar 10 19:00:48 RuntimeError: UserRRef(rref_id=GloballyUniqueId(created_on=3, local_id=0), fork_id=GloballyUniqueId(created_on=3, local_id=3)) is an RRef to a ScriptModule. It can't be sent through RPC from owner, WorkerInfo(id=0, name=worker0), to user, WorkerInfo(id=3, name=worker3).
Mar 10 19:00:48 
Mar 10 19:00:48 ok (1.230s)

See CircleCI build pytorch_windows_vs2019_py36_cuda10.1_test2 (3/4)

Step: "Test" (full log | diagnosis details | 🔁 rerun)

RuntimeError: test_package failed!
Generating XML reports...
Generated XML report: test-reports\python-unittest\test_package\TEST-ManglingTest-20210310191728.xml
Generated XML report: test-reports\python-unittest\test_package\TEST-PackagingTest-20210310191728.xml
Generated XML report: test-reports\python-unittest\test_package\TEST-TestImporter-20210310191728.xml
Generated XML report: test-reports\python-unittest\test_package\TEST-TestPackageResources-20210310191728.xml
Traceback (most recent call last):
  File "run_test.py", line 1025, in <module>
    main()
  File "run_test.py", line 1004, in main
    raise RuntimeError(err_message)
RuntimeError: test_package failed!

(base) C:\Users\circleci\project\test>if ERRORLEVEL 1 exit /b 1 
+ cleanup
+ retcode=1
+ set +x


Exited with code exit status 1

See CircleCI build pytorch_linux_bionic_py3_6_clang9_test (4/4)

Step: "Run tests" (full log | diagnosis details | 🔁 rerun)

Mar 10 18:07:20 [E request_callback_no_python.cpp:656] Received error while processing request type 256: The following operation failed in the TorchScript interpreter.
Mar 10 18:07:20 [E request_callback_no_python.cpp:656] Received error while processing request type 256: The following operation failed in the TorchScript interpreter.
Mar 10 18:07:20 Traceback of TorchScript (most recent call last):
Mar 10 18:07:20   File "/opt/conda/lib/python3.6/site-packages/torch/testing/_internal/distributed/rpc/jit/rpc_test.py", line 825, in run_ref_script_module
Mar 10 18:07:20     ref_script_module: RRef[MyModuleInterface], t: Tensor
Mar 10 18:07:20 ) -> Tensor:
Mar 10 18:07:20     module = ref_script_module.to_here()
Mar 10 18:07:20              ~~~~~~~~~~~~~~~~~~~~~~~~~ <--- HERE
Mar 10 18:07:20     return module.forward() + t
Mar 10 18:07:20 RuntimeError: UserRRef(rref_id=GloballyUniqueId(created_on=0, local_id=0), fork_id=GloballyUniqueId(created_on=0, local_id=3)) is an RRef to a ScriptModule. It can't be sent through RPC from owner, WorkerInfo(id=1, name=worker1), to user, WorkerInfo(id=0, name=worker0).
Mar 10 18:07:20 
Mar 10 18:07:20 [E request_callback_no_python.cpp:656] Received error while processing request type 256: The following operation failed in the TorchScript interpreter.
Mar 10 18:07:20 Traceback of TorchScript (most recent call last):
Mar 10 18:07:20   File "/opt/conda/lib/python3.6/site-packages/torch/testing/_internal/distributed/rpc/jit/rpc_test.py", line 825, in run_ref_script_module
Mar 10 18:07:20     ref_script_module: RRef[MyModuleInterface], t: Tensor
Mar 10 18:07:20 ) -> Tensor:
Mar 10 18:07:20     module = ref_script_module.to_here()
Mar 10 18:07:20              ~~~~~~~~~~~~~~~~~~~~~~~~~ <--- HERE
Mar 10 18:07:20     return module.forward() + t
Mar 10 18:07:20 RuntimeError: UserRRef(rref_id=GloballyUniqueId(created_on=1, local_id=0), fork_id=GloballyUniqueId(created_on=1, local_id=3)) is an RRef to a ScriptModule. It can't be sent through RPC from owner, WorkerInfo(id=2, name=worker2), to user, WorkerInfo(id=1, name=worker1).
Mar 10 18:07:20 
Mar 10 18:07:21 ok (1.844s)

1 failure not recognized by patterns:

Job Step Action
CircleCI pytorch_linux_xenial_py3_clang5_asan_test2 Run tests 🔁 rerun

ci.pytorch.org: 1 failed


This comment was automatically generated by Dr. CI (expand for details).Follow this link to opt-out of these comments for your Pull Requests.

Please report bugs/suggestions to the (internal) Dr. CI Users group.

@suo suo requested a review from zdevito February 3, 2021 22:39
Comment thread torch/package/importer.py Outdated
Comment thread torch/package/importer.py Outdated
Comment thread torch/package/importer.py Outdated
@suo suo requested a review from zdevito February 24, 2021 00:57
Copy link
Copy Markdown
Contributor

@zdevito zdevito left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

looks good!

@facebook-github-bot
Copy link
Copy Markdown
Contributor

@suo merged this pull request in b4d8f4a.

@facebook-github-bot facebook-github-bot deleted the gh/suo/391/head branch March 14, 2021 14:14
xsacha pushed a commit to xsacha/pytorch that referenced this pull request Mar 31, 2021
Summary:
Pull Request resolved: pytorch#51674

See
https://docs.python.org/3/library/importlib.html#importlib.abc.ResourceReader

Test Plan: Imported from OSS

Reviewed By: zdevito

Differential Revision: D26237034

Pulled By: suo

fbshipit-source-id: 4c19f6172d16b710737528d3de48372873b9368d
laurentdupin pushed a commit to laurentdupin/pytorch that referenced this pull request Apr 24, 2026
Summary:
Pull Request resolved: pytorch#51674

See
https://docs.python.org/3/library/importlib.html#importlib.abc.ResourceReader

Test Plan: Imported from OSS

Reviewed By: zdevito

Differential Revision: D26237034

Pulled By: suo

fbshipit-source-id: 4c19f6172d16b710737528d3de48372873b9368d
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

cla signed Merged oncall: jit Add this issue/PR to JIT oncall triage queue

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants