OP-6382 - Thumbnail Integration Problem by tokejepsen · Pull Request #6156 · ynput/OpenPype

tokejepsen · 2024-01-23T17:59:14Z

Changelog Description

This ticket alerted to 3 different cases of integration issues;

Using the Tray Publisher with the same image format (extension) for representation and review representation.
Clash on publish file path from output definitions in ExtractOIIOTranscode.
Clash on publish file from thumbnail in ExtractThumbnail

There might be an issue with this fix, if a studio does not use the {output} token in their render anatomy template. But thinking if they have customized it, they will be responsible to maintain these edge cases.

Testing notes:

Use Tray Publisher to publish Image with two different images of same extension.
Publish should succeed.
Setup ayon+settings://core/publish/ExtractOIIOTranscode/profiles with settings below.
Setup single frame render from Maya and publish to Deadline.
Publish on Deadline should succeed.
Setup ayon+settings://core/publish/ExtractOIIOTranscode/profiles with settings below (just adding review tag from above settings).
Setup single frame render from Maya and publish to Deadline.
Publish on Deadline should succeed.

ynbot · 2024-01-23T17:59:17Z

Task linked: OP-6382 Thumbnail integration problem

LiborBatek

Did test the first case using Tray Publisher

and getting this error on Integrate Asset

DEBUG: Establishing staging directory @ C:\Users\lbate\AppData\Local\Temp\traypublisher_70zblecs
DEBUG: Looking for matching profile for: hosts: "traypublisher" | families: "image" | task_names: "art" | task_types: "Art"
DEBUG: "image" not found in "families": ['review', 'render', 'prerender']
DEBUG: "traypublisher" not found in "hosts": ['standalonepublisher']
DEBUG: "traypublisher" not found in "hosts": ['maya']
DEBUG: "image" not found in "families": ['online']
DEBUG: "traypublisher" not found in "hosts": ['max']
DEBUG: Profile selected: {'hosts': [], 'task_types': [], 'task_names': [], 'template_name': 'publish', 'families': []}
DEBUG: Subset: imageReference
DEBUG: http://localhost:5000 "POST /graphql HTTP/1.1" 200 105
DEBUG: Response <RestApiResponse [200]>
INFO: Subset 'imageReference' not found, creating ...
DEBUG: Prepared subset: imageReference
DEBUG: http://localhost:5000 "POST /graphql HTTP/1.1" 200 105
DEBUG: Response <RestApiResponse [200]>
DEBUG: Source: C:\Projects\Temp_Data\volcanoFiction.png
DEBUG: Creating new version ...
DEBUG: Prepared version: v001
DEBUG: http://localhost:5000 "POST /graphql HTTP/1.1" 200 112
DEBUG: Response <RestApiResponse [200]>
DEBUG: Anatomy template name: publish
DEBUG: Anatomy template name: publish
Traceback (most recent call last):
  File "C:\Users\lbate\AppData\Local\Ynput\AYON\addons\openpype_3.18.5-nightly.2\openpype\plugins\publish\integrate.py", line 183, in process
    self.register(instance, file_transactions, filtered_repres)
  File "C:\Users\lbate\AppData\Local\Ynput\AYON\addons\openpype_3.18.5-nightly.2\openpype\plugins\publish\integrate.py", line 283, in register
    file_transactions.add(src, dst)
  File "C:\Users\lbate\AppData\Local\Ynput\AYON\addons\openpype_3.18.5-nightly.2\openpype\lib\file_transaction.py", line 98, in add
    raise DuplicateDestinationError(
openpype.lib.file_transaction.DuplicateDestinationError: Transfer to destination is already in queue: C:\Projects\Temp_Data\volcanoFiction.png -> C:\projects\Ayon_Tests\2d_comps\artwork_A\publish\image\imageReference\v001\ATS_artwork_A_imageReference_v001.png. It's not allowed to be replaced by a new transfer from C:\Projects\Temp_Data\volcanoFiction_prev.png

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "C:\Users\lbate\AppData\Local\Ynput\AYON\dependency_packages\ayon_2401161802_windows.zip\dependencies\pyblish\plugin.py", line 527, in __explicit_process
    runner(*args)
  File "C:\Users\lbate\AppData\Local\Ynput\AYON\addons\openpype_3.18.5-nightly.2\openpype\plugins\publish\integrate.py", line 188, in process
    six.reraise(KnownPublishError,
  File "six.py", line 718, in reraise
  File "C:\Users\lbate\AppData\Local\Ynput\AYON\addons\openpype_3.18.5-nightly.2\openpype\plugins\publish\integrate.py", line 183, in process
    self.register(instance, file_transactions, filtered_repres)
  File "C:\Users\lbate\AppData\Local\Ynput\AYON\addons\openpype_3.18.5-nightly.2\openpype\plugins\publish\integrate.py", line 283, in register
    file_transactions.add(src, dst)
  File "C:\Users\lbate\AppData\Local\Ynput\AYON\addons\openpype_3.18.5-nightly.2\openpype\lib\file_transaction.py", line 98, in add
    raise DuplicateDestinationError(
openpype.pipeline.publish.publish_plugins.KnownPublishError: Transfer to destination is already in queue: C:\Projects\Temp_Data\volcanoFiction.png -> C:\projects\Ayon_Tests\2d_comps\artwork_A\publish\image\imageReference\v001\ATS_artwork_A_imageReference_v001.png. It's not allowed to be replaced by a new transfer from C:\Projects\Temp_Data\volcanoFiction_prev.png

I was using two PNG images heres my Publisher settings

moonyuet · 2024-01-24T10:32:30Z

Guess the error above is related to this code:

OpenPype/openpype/plugins/publish/integrate.py

Lines 179 to 181 in c8e00d4

    
           file_transactions = FileTransaction(log=self.log, 
        
                                               # Enforce unique transfers 
        
                                               allow_queue_replacements=False)

when allow_queue_requirements set to False

LiborBatek · 2024-01-24T10:37:23Z

Did anotther test step (05) and Im getting error on DL for publish job:

2024-01-24 11:33:40:  0: STDOUT: DEBUG:pyblish.IntegrateAsset:Anatomy template name: publish_render
2024-01-24 11:33:40:  0: STDOUT: ERROR:pyblish.plugin:Traceback (most recent call last):
2024-01-24 11:33:40:  0: STDOUT:   File "C:\Users\lbate\AppData\Local\Ynput\AYON\addons\openpype_3.18.5-nightly.2\openpype\plugins\publish\integrate.py", line 183, in process
2024-01-24 11:33:40:  0: STDOUT:   File "C:\Users\lbate\AppData\Local\Ynput\AYON\addons\openpype_3.18.5-nightly.2\openpype\plugins\publish\integrate.py", line 283, in register
2024-01-24 11:33:40:  0: STDOUT:   File "C:\Users\lbate\AppData\Local\Ynput\AYON\addons\openpype_3.18.5-nightly.2\openpype\lib\file_transaction.py", line 98, in add
2024-01-24 11:33:40:  0: STDOUT:     raise DuplicateDestinationError(
2024-01-24 11:33:40:  0: STDOUT: openpype.lib.file_transaction.DuplicateDestinationError: Transfer to destination is already in queue: C:\Users\lbate\AppData\Local\Temp\op_transcoding_cax4llrd\Main.1001.jpg -> C:\projects\Ayon_Tests\maya\props\crate\publish\render\renderGenericMain_beauty\v001\ATS_crate_renderGenericMain_beauty_v001.jpg. It's not allowed to be replaced by a new transfer from C:\Users\lbate\AppData\Local\Temp\pyblish_tmp_9hhlfndf\Main.1001_thumb.jpg
2024-01-24 11:33:40:  0: STDOUT: During handling of the above exception, another exception occurred:
2024-01-24 11:33:40:  0: STDOUT: Traceback (most recent call last):
2024-01-24 11:33:40:  0: STDOUT:   File "C:\Users\lbate\AppData\Local\Ynput\AYON\dependency_packages\ayon_2401161802_windows.zip\dependencies\pyblish\plugin.py", line 527, in __explicit_process
2024-01-24 11:33:40:  0: STDOUT:     runner(*args)
2024-01-24 11:33:40:  0: STDOUT:   File "C:\Users\lbate\AppData\Local\Ynput\AYON\addons\openpype_3.18.5-nightly.2\openpype\plugins\publish\integrate.py", line 188, in process
2024-01-24 11:33:40:  0: STDOUT:   File "six.py", line 718, in reraise
2024-01-24 11:33:40:  0: STDOUT:   File "C:\Users\lbate\AppData\Local\Ynput\AYON\addons\openpype_3.18.5-nightly.2\openpype\plugins\publish\integrate.py", line 183, in process
2024-01-24 11:33:40:  0: STDOUT:   File "C:\Users\lbate\AppData\Local\Ynput\AYON\addons\openpype_3.18.5-nightly.2\openpype\plugins\publish\integrate.py", line 283, in register
2024-01-24 11:33:40:  0: STDOUT:   File "C:\Users\lbate\AppData\Local\Ynput\AYON\addons\openpype_3.18.5-nightly.2\openpype\lib\file_transaction.py", line 98, in add
2024-01-24 11:33:40:  0: STDOUT:     raise DuplicateDestinationError(
2024-01-24 11:33:40:  0: STDOUT: openpype.pipeline.publish.publish_plugins.KnownPublishError: Transfer to destination is already in queue: C:\Users\lbate\AppData\Local\Temp\op_transcoding_cax4llrd\Main.1001.jpg -> C:\projects\Ayon_Tests\maya\props\crate\publish\render\renderGenericMain_beauty\v001\ATS_crate_renderGenericMain_beauty_v001.jpg. It's not allowed to be replaced by a new transfer from C:\Users\lbate\AppData\Local\Temp\pyblish_tmp_9hhlfndf\Main.1001_thumb.jpg
2024-01-24 11:33:40:  0: STDOUT: Traceback (most recent call last):
2024-01-24 11:33:40:  0: STDOUT:   File "<string>", line 183, in process
2024-01-24 11:33:40:  0: STDOUT:   File "<string>", line 283, in register
2024-01-24 11:33:40:  0: STDOUT:   File "C:\Users\lbate\AppData\Local\Ynput\AYON\addons\openpype_3.18.5-nightly.2\openpype\lib\file_transaction.py", line 98, in add
2024-01-24 11:33:40:  0: STDOUT:     raise DuplicateDestinationError(
2024-01-24 11:33:40:  0: STDOUT: openpype.lib.file_transaction.DuplicateDestinationError: Transfer to destination is already in queue: C:\Users\lbate\AppData\Local\Temp\op_transcoding_cax4llrd\Main.1001.jpg -> C:\projects\Ayon_Tests\maya\props\crate\publish\render\renderGenericMain_beauty\v001\ATS_crate_renderGenericMain_beauty_v001.jpg. It's not allowed to be replaced by a new transfer from C:\Users\lbate\AppData\Local\Temp\pyblish_tmp_9hhlfndf\Main.1001_thumb.jpg
2024-01-24 11:33:40:  0: STDOUT: During handling of the above exception, another exception occurred:
2024-01-24 11:33:40:  0: STDOUT: Traceback (most recent call last):
2024-01-24 11:33:40:  0: STDOUT:   File "C:\Users\lbate\AppData\Local\Ynput\AYON\dependency_packages\ayon_2401161802_windows.zip\dependencies\pyblish\plugin.py", line 527, in __explicit_process
2024-01-24 11:33:40:  0: STDOUT:     runner(*args)
2024-01-24 11:33:40:  0: STDOUT:   File "<string>", line 188, in process
2024-01-24 11:33:40:  0: STDOUT:   File "six.py", line 718, in reraise
2024-01-24 11:33:40:  0: STDOUT:   File "<string>", line 183, in process
2024-01-24 11:33:40:  0: STDOUT:   File "<string>", line 283, in register
2024-01-24 11:33:40:  0: STDOUT:   File "C:\Users\lbate\AppData\Local\Ynput\AYON\addons\openpype_3.18.5-nightly.2\openpype\lib\file_transaction.py", line 98, in add
2024-01-24 11:33:40:  0: STDOUT:     raise DuplicateDestinationError(
2024-01-24 11:33:40:  0: STDOUT: openpype.pipeline.publish.publish_plugins.KnownPublishError: Transfer to destination is already in queue: C:\Users\lbate\AppData\Local\Temp\op_transcoding_cax4llrd\Main.1001.jpg -> C:\projects\Ayon_Tests\maya\props\crate\publish\render\renderGenericMain_beauty\v001\ATS_crate_renderGenericMain_beauty_v001.jpg. It's not allowed to be replaced by a new transfer from C:\Users\lbate\AppData\Local\Temp\pyblish_tmp_9hhlfndf\Main.1001_thumb.jpg
2024-01-24 11:33:40:  0: STDOUT: !!! ERR: 2024-01-24 11:33:40,083 >>> { CLI-publish }: [ Failed IntegrateAsset: Transfer to destination is already in queue: C:\Users\lbate\AppData\Local\Temp\op_transcoding_cax4llrd\Main.1001.jpg -> C:\projects\Ayon_Tests\maya\props\crate\publish\render\renderGenericMain_beauty\v001\ATS_crate_renderGenericMain_beauty_v001.jpg. It's not allowed to be replaced by a new transfer from C:\Users\lbate\AppData\Local\Temp\pyblish_tmp_9hhlfndf\Main.1001_thumb.jpg -- ('C:\\Users\\lbate\\AppData\\Local\\Ynput\\AYON\\addons\\openpype_3.18.5-nightly.2\\openpype\\plugins\\publish\\integrate.py', 98, 'add', 'raise DuplicateDestinationError(') ] 
2024-01-24 11:33:40:  0: STDOUT: --- your system is set to use custom CA certificate bundle.

LiborBatek · 2024-01-24T11:16:25Z

In the last testing step with review tag added, it ends pretty much the same with failure of the DL publish job with the exact same error msg. so check that above too.

tokejepsen · 2024-01-24T12:41:33Z

Did test the first case using Tray Publisher

@LiborBatek I've replicated here and it works. I can see what you are using Production bundle, could you let me know how you build and are setup, so I can replicate here?

LiborBatek · 2024-01-24T15:36:48Z

Did test the first case using Tray Publisher

@LiborBatek I've replicated here and it works. I can see what you are using Production bundle, could you let me know how you build and are setup, so I can replicate here?

You can build from within your OP Repo by

OpenPype\server_addon> python .\create_ayon_addons.py

then just locate the folder \server_addon\packages\ and upload those needed packages on Ayon server and put those into new Addon Bundle, copy settings from previously used Addon bundle and set the new one as Production so it will be used when run Ayon Launcher in normal mode (aka production).

I rather do it this way...to be sure it works really. Not much using developer mode...

antirotor

Tested all combinations - same image, two different images, jpgs, pngs, representation and little thumbnail and all works! Thanks.

Note

Tested using dev-mode

LiborBatek · 2024-01-25T10:36:53Z

@tokejepsen ok I re did the test again and double checked my addons being correctly set and in production and finally also works for me. sorry for confusion (probably didnt deleted local ayon data folder content before!)

Went smoothly in Tray Publisher and with thumbnails integrated on Ayon too

and on Ayon:

openpype/plugins/publish/extract_color_transcode.py

Co-authored-by: Jakub Trllo <43494761+iLLiCiTiT@users.noreply.github.com>

this is adding #6156 to 3.17

tokejepsen added 3 commits January 23, 2024 17:36

Fix simple instances for Tray Publisher.

26409d4

Fix edge case for ExtractOIIOTranscode

f59e403

Fix thumbnail extraction

741eab7

tokejepsen requested review from antirotor and iLLiCiTiT January 23, 2024 17:59

tokejepsen assigned LiborBatek Jan 23, 2024

tokejepsen changed the title ~~OP-6382 Thumbnail Integration Problem~~ OP-6382 - Thumbnail Integration Problem Jan 23, 2024

ynbot added type: bug Something isn't working size/XS Denotes a PR changes 0-99 lines, ignoring general files host: TrayPublisher labels Jan 23, 2024

LiborBatek self-requested a review January 24, 2024 09:53

LiborBatek suggested changes Jan 24, 2024

View reviewed changes

antirotor approved these changes Jan 25, 2024

View reviewed changes

LiborBatek self-requested a review January 25, 2024 10:27

LiborBatek approved these changes Jan 25, 2024

View reviewed changes

tokejepsen assigned antirotor Jan 25, 2024

tokejepsen unassigned LiborBatek Jan 25, 2024

iLLiCiTiT reviewed Jan 25, 2024

View reviewed changes

openpype/plugins/publish/extract_color_transcode.py Outdated Show resolved Hide resolved

Update openpype/plugins/publish/extract_color_transcode.py

cb9802e

Co-authored-by: Jakub Trllo <43494761+iLLiCiTiT@users.noreply.github.com>

tokejepsen requested a review from iLLiCiTiT January 25, 2024 11:05

antirotor merged commit d5643d2 into ynput:develop Jan 25, 2024

ynbot added this to the next-patch milestone Jan 25, 2024

tokejepsen deleted the bugfix/OP-6382_thumbnail_integration_problem branch January 25, 2024 11:28

antirotor added a commit that referenced this pull request Jan 25, 2024

♻️ backport of OP-6382

b207de4

this is adding #6156 to 3.17

antirotor mentioned this pull request Jan 25, 2024

OP-6382 - Thumbnail Integration Problem - backport 3.17 #6167

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

OP-6382 - Thumbnail Integration Problem#6156

OP-6382 - Thumbnail Integration Problem#6156
antirotor merged 4 commits intoynput:developfrom
tokejepsen:bugfix/OP-6382_thumbnail_integration_problem

tokejepsen commented Jan 23, 2024 •

edited

Loading

Uh oh!

ynbot commented Jan 23, 2024

Uh oh!

LiborBatek left a comment

Uh oh!

moonyuet commented Jan 24, 2024 •

edited

Loading

Uh oh!

LiborBatek commented Jan 24, 2024

Uh oh!

LiborBatek commented Jan 24, 2024

Uh oh!

tokejepsen commented Jan 24, 2024

Uh oh!

LiborBatek commented Jan 24, 2024 •

edited

Loading

Uh oh!

antirotor left a comment •

edited

Loading

Uh oh!

LiborBatek commented Jan 25, 2024

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

6 participants

Conversation

tokejepsen commented Jan 23, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Changelog Description

Testing notes:

Uh oh!

ynbot commented Jan 23, 2024

Uh oh!

LiborBatek left a comment

Choose a reason for hiding this comment

Uh oh!

moonyuet commented Jan 24, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

LiborBatek commented Jan 24, 2024

Uh oh!

LiborBatek commented Jan 24, 2024

Uh oh!

tokejepsen commented Jan 24, 2024

Uh oh!

LiborBatek commented Jan 24, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

antirotor left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

LiborBatek commented Jan 25, 2024

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

6 participants

tokejepsen commented Jan 23, 2024 •

edited

Loading

moonyuet commented Jan 24, 2024 •

edited

Loading

LiborBatek commented Jan 24, 2024 •

edited

Loading

antirotor left a comment •

edited

Loading