DYN-9388 Make PythonNet3 default python engine as built-in package #16548

ivaylo-matov · 2025-09-30T09:06:06Z

Purpose

PR in response to DYN-9388.

for testing UX scenarios
pythonnet3 loaded as built-in package
for now cpython is still present but does not load
to be discussed what to do with python tests. Note that test will fail now as they still rely on built-in cpython logic.
when users open old graph with cpython nodes, they get updated to PythonNet3 and toast is displayed. On save another text box appears to explain the changes. Should reflect latest Figma design

Declarations

Check these if you believe they are true

Is documented according to the standards
The level of testing this PR includes is appropriate
Changes to the API follow Semantic Versioning and are documented in the API Changes document.

Release Notes

Make PythonNet3 default python engine as built-in package

Reviewers

@zeusongit
@DynamoDS/eidos

FYIs

@dnenov
@achintyabhat

This reverts commit 0389294.

…kage-does-not-load

cpython still present but does not load

…wip)

…com/ivaylo-matov/Dynamo into DYN-9388-Net3-package-does-not-load

github-actions

See the ticket for this pull request: https://jira.autodesk.com/browse/DYN-9388

Copilot

Pull Request Overview

This PR makes PythonNet3 the default Python engine as a built-in package, replacing CPython3. It provides automatic migration of existing CPython nodes to PythonNet3 with user notifications and adds preferences to control these notifications.

Key changes:

Switches default Python engine from CPython3 to PythonNet3 across the codebase
Implements automatic node migration with toast notifications and dialog boxes
Adds user preference controls for hiding migration notifications

Reviewed Changes

Copilot reviewed 66 out of 77 changed files in this pull request and generated no comments.

Show a summary per file

File	Description
src/PythonMigrationViewExtension/PythonMigrationViewExtension.cs	Adds CPython to PythonNet3 migration logic with notifications and automatic node upgrading
src/NodeServices/PythonServices.cs	Updates default engine references and removes automatic CPython loading
src/Libraries/PythonNodeModels/PythonNode.cs	Changes default engine from CPython3 to PythonNet3 and adds ShowAutoUpgradedBar property
src/DynamoCoreWpf/Views/Core/DynamoView.xaml.cs	Implements UI handlers for Python engine change notifications and dialogs
src/DynamoCore/Configuration/PreferenceSettings.cs	Adds HideCPython3Notifications preference setting
extern/PythonNet3Engine/	Adds PythonNet3 engine as built-in package with all necessary configuration files

Files not reviewed (3)

src/DynamoCoreWpf/Properties/Resources.Designer.cs: Language not supported
src/Libraries/PythonNodeModels/Properties/Resources.Designer.cs: Language not supported
src/PythonMigrationViewExtension/Properties/Resources.Designer.cs: Language not supported

Comments suppressed due to low confidence (5)

ivaylo-matov · 2025-09-30T09:42:19Z

What should we do with the new DSPytnotNet3 versions?

zeusongit · 2025-09-30T14:49:39Z

src/DynamoCoreWpf/Properties/Resources.it-IT.resx

-  <data name="ResetCPythonButtonText" xml:space="preserve">
-    <value>Reimposta CPython</value>
+  <data name="ResetPythonNet3ButtonText" xml:space="preserve">
+    <value>Reimposta PythonNet3</value>


You don't need to update the strings for all locale(s) they are updated by some automation we have setup afterwards. We just need to update them in Resources.Designer.cs and Resources.resx files

reverted the changes now

zeusongit · 2025-09-30T14:52:18Z

What should we do with the new DSPytnotNet3 versions?

We can add them to the excluded files list here: https://github.com/DynamoDS/Dynamo/blob/master/.github/scripts/check_file_version.ps1#L15

Copilot

Pull Request Overview

Copilot reviewed 67 out of 78 changed files in this pull request and generated no new comments.

Files not reviewed (3)

src/DynamoCoreWpf/Properties/Resources.Designer.cs: Language not supported
src/Libraries/PythonNodeModels/Properties/Resources.Designer.cs: Language not supported
src/PythonMigrationViewExtension/Properties/Resources.Designer.cs: Language not supported

Comments suppressed due to low confidence (5)

…com/ivaylo-matov/Dynamo into DYN-9388-Net3-package-does-not-load

zeusongit · 2025-10-13T23:02:24Z

@twastvedt @mjkkirschner I think this is ready for another look, the changes from other PR as well as the Nuget is merged in this

zeusongit · 2025-10-13T23:03:52Z

@avidit @QilongTang The self serve job fails repeatedly with instance being killed, cannot figure out if tests are failing/causing it or something else

twastvedt · 2025-10-14T19:57:42Z

src/DynamoCore/DynamoCore.csproj

+            <DllExtra Include="$(LibDir)DSPythonNet3Extension_ExtensionDefinition.xml" Condition="Exists('$(LibDir)DSPythonNet3Extension_ExtensionDefinition.xml')" />            
+        </ItemGroup>
+        <Copy SourceFiles="@(DllExtra)" DestinationFolder="$(DestExtra)" SkipUnchangedFiles="True" />  
+    </Target>    


What's the reasoning for needing this to know so much about the exact files it's copying, and where they go? Why can't the package created by the PythonNet3 engine repo come formatted correctly, with all the files in a single known folder ready to copy into Built-in packages? That way DynamoCore wouldn't need to know about the inner workings of the Python package, and thus need to be updated if the package structure changes.

yeah pretty much agree, the less code here the better...

it appears you are just copying everything from bin, so just copy the contents of lib, are you excluding something?

is there not a net10 version of pythonNet? Then you could do Lib\$(TargetFramework)

mjkkirschner · 2025-10-14T22:06:37Z

src/DynamoPackages/PackageManagerExtension.cs

-                catch(Exception ex)
+
+                // Set PythonNet3 as the default engine if it is isntalled and no other valid engine is set as default
+                var eng = e.NewItems[0] as PythonServices.PythonEngine;


is it a problem that this might get fired at some time in the future when a user installs a package that includes a python engine?

You are right, I postponed this for after all engines are loaded.

mjkkirschner · 2025-10-14T22:09:14Z

src/PythonMigrationViewExtension/MigrationAssistant/ScriptMigrator.cs

        /// <returns></returns>
        internal static string MigrateCode(string code)
        {
-            DSCPython.CPythonEvaluator.InstallPython();


how does this work with the new package - where / how does the runtime get installed?

PythonNet3 bootstraps the runtime itself and wires the engine up when the package loads.
On startup PackageLoader loads built-in packages. After a package is loaded it calls LoadPythonEngine() to discover and register engine classes from those assemblies.

mjkkirschner · 2025-10-14T22:09:21Z

src/PythonMigrationViewExtension/MigrationAssistant/ScriptMigrator.cs

-            {
-                PythonEngine.ReleaseLock(gs);
-            }
+            finally { }


not really useful...

Removed now.
Switched to using (Py.GIL()) because pythonnet3 removed AcquireLock/ReleaseLock and we must explicitly hold the GIL. Also PythonEngine.BeginAllowThreads() because that’s for releasing the GIL during long .NET-only work.

mjkkirschner · 2025-10-14T22:23:36Z

@ivaylo-matov a few comments and one potentially bigger change:
WRT to the IronPython package that is embedded for testing, could we reference a build somewhere instead of copying the files in? Even just embedding a zip (if it's small enough) might be preferrable to the loose files in the git repo - just easier to update and reference a specific build - which makes it easier for the next dev IMO.

Also I ran into a problem running IronPython3 package under net10 - did you see the same issue with IronPython2?

ivaylo-matov · 2025-10-15T14:02:37Z

@ivaylo-matov a few comments and one potentially bigger change: WRT to the IronPython package that is embedded for testing, could we reference a build somewhere instead of copying the files in? Even just embedding a zip (if it's small enough) might be preferrable to the loose files in the git repo - just easier to update and reference a specific build - which makes it easier for the next dev IMO.

Also I ran into a problem running IronPython3 package under net10 - did you see the same issue with IronPython2?

Done - zipped the loose package files and added an MSBuild target in DynamoPythonTests.csproj to unzip during test builds. Also added a local .gitignore in to whitelist PythonEnginePackage.zip. Hope that's okay.

mjkkirschner · 2025-10-15T15:53:27Z

10:30:51  C:\Jenkins\workspace\Dynamo\DynamoSelfServe\pullRequestValidation>pwsh.exe -ExecutionPolicy ByPass -File .\CICDTools\scripts\RunTests.ps1 
10:30:51  Using dotnet test
10:30:51  The test type to execute is all tests
10:30:51  Execute Serial tests
10:30:51  Starting run 1
11:08:43  Cannot contact CDA-VS22-DT_7668: java.lang.InterruptedException
11:17:04  Agent CDA-VS22-DT_7668 was deleted; cancelling node body
11:17:04  Could not connect to CDA-VS22-DT_7668 to send interrupt signal to process

maybe you can try running the same test script locally and see if the test runner crashes?

zeusongit

@ivaylo-matov thank you for the updates, I have been testing the PR, and it seems to be in a good state, I will merge this now so that QAs can push testing further.
@mjkkirschner @twastvedt thank you for the review, if their are some outstanding comments, they will be addressed in separate PRs, however I think most of them have already been addressed, free to take another look.

…ynamoDS#16548) (cherry picked from commit 38f5741)

ivaylo-matov added 14 commits September 3, 2025 09:42

new DSPrythonNet3 project and PythonServices modified

0389294

Revert "new DSPrythonNet3 project and PythonServices modified"

b70f95d

This reverts commit 0389294.

Merge remote-tracking branch 'upstream/master' into DYN-9388-Net3-pac…

acb9b38

…kage-does-not-load

pythonnet3 loads as default

ee79b74

cpython still present but does not load

resources

62af16c

notifications transfer

cdabfa3

resources correction

2dfc883

cancel button removed

c0f5720

resources updates to match new design

250ad77

python update bar relocated

5c32588

request to update banner removed and new design for engine dropdown (…

68741ed

…wip)

python update bar relocated & request to update bar removed

95c1660

Merge branch 'DYN-9388-Net3-package-does-not-load' of https://github.…

a076326

…com/ivaylo-matov/Dynamo into DYN-9388-Net3-package-does-not-load

for pr

0058c33

Copilot AI review requested due to automatic review settings September 30, 2025 09:06

github-actions bot reviewed Sep 30, 2025

View reviewed changes

Copilot AI reviewed Sep 30, 2025

View reviewed changes

public api register

e5f81c1

ivaylo-matov mentioned this pull request Sep 30, 2025

DYN-9388 Make PythonNet3 default python engine in Dynamo 4.0 #16516

Closed

3 tasks

zeusongit reviewed Sep 30, 2025

View reviewed changes

zeusongit requested review from a team and twastvedt September 30, 2025 14:54

ivaylo-matov force-pushed the DYN-9388-Net3-package-does-not-load branch from 68456d0 to e5f81c1 Compare September 30, 2025 17:45

ivaylo-matov added 2 commits September 30, 2025 18:49

Update check_file_version.ps1

6434f92

Update check_file_version.ps1

7319898

zeusongit requested a review from Copilot September 30, 2025 19:27

Copilot AI reviewed Sep 30, 2025

View reviewed changes

Merge branch 'master' into DYN-9388-Net3-package-does-not-load

4b9a297

ivaylo-matov and others added 11 commits October 10, 2025 18:31

net3package from \extern removed

94ef70f

nuget in place

eb5f6a7

migrator working and \extern removed

7a39d15

unit tests

76c30e9

comments in DynamoDS#16582 addressed

c676f4b

nuget updated to 1.4.2

468f488

ShowCPythonNotifications correction

c10eddc

tag resources

77b6621

Merge branch 'master' into DYN-9388-Net3-package-does-not-load

2a56e43

Update check_file_version.ps1

d28afd9

Merge branch 'DYN-9388-Net3-package-does-not-load' of https://github.…

6f8b452

…com/ivaylo-matov/Dynamo into DYN-9388-Net3-package-does-not-load

twastvedt reviewed Oct 14, 2025

View reviewed changes

mjkkirschner reviewed Oct 14, 2025

View reviewed changes

ivaylo-matov added 4 commits October 15, 2025 11:29

Update DynamoCore.csproj

78b92be

Update ScriptMigrator.cs

892e408

postpone setting default python engine

13c6f62

dependency test package zipped

2272b1b

zeusongit approved these changes Oct 15, 2025

View reviewed changes

zeusongit merged commit 38f5741 into DynamoDS:master Oct 15, 2025
26 of 27 checks passed

zeusongit mentioned this pull request Oct 16, 2025

Revert "DYN-9388 Make PythonNet3 default python engine as built-in package" #16602

Merged

reddyashish pushed a commit to reddyashish/Dynamo that referenced this pull request Oct 16, 2025

DYN-9388 Make PythonNet3 default python engine as built-in package (D…

0ccff99

…ynamoDS#16548) (cherry picked from commit 38f5741)

reddyashish mentioned this pull request Oct 16, 2025

PythonNet3 as built-in package #16604

Closed

3 tasks

ivaylo-matov mentioned this pull request Oct 24, 2025

DYN-9388 Make PythonNet3 default python engine as built-in package #16644

Merged

3 tasks

DYN-9388 Make PythonNet3 default python engine as built-in package #16548

DYN-9388 Make PythonNet3 default python engine as built-in package #16548

Uh oh!

Conversation

ivaylo-matov commented Sep 30, 2025

Purpose

Declarations

Release Notes

Reviewers

FYIs

Uh oh!

github-actions bot left a comment

Choose a reason for hiding this comment

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull Request Overview

Reviewed Changes

Uh oh!

ivaylo-matov commented Sep 30, 2025

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

zeusongit commented Sep 30, 2025

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull Request Overview

Uh oh!

zeusongit commented Oct 13, 2025

Uh oh!

zeusongit commented Oct 13, 2025

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

ivaylo-matov Oct 15, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

mjkkirschner commented Oct 14, 2025

Uh oh!

ivaylo-matov commented Oct 15, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

mjkkirschner commented Oct 15, 2025

Uh oh!

zeusongit left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

ivaylo-matov Oct 15, 2025 •

edited

Loading

ivaylo-matov commented Oct 15, 2025 •

edited

Loading