Skip to content

Conversation

@QilongTang
Copy link
Contributor

@QilongTang QilongTang commented Apr 13, 2020

Please Note:

  1. Before submitting the PR, please review How to Contribute to Dynamo
  2. Dynamo Team will meet 1x a month to review PRs found on Github (Issues will be handled separately)
  3. PRs will be reviewed from oldest to newest
  4. If a reviewed PR requires changes by the owner, the owner of the PR has 30 days to respond. If the PR has seen no activity by the next session, it will be either closed by the team or depending on its utility will be taken over by someone on the team
  5. PRs should use either Dynamo's default PR template or one of these other template options in order to be considered for review.
  6. PRs that do not have one of the Dynamo PR templates completely filled out with all declarations satisfied will not be reviewed by the Dynamo team.
  7. PRs made to the DynamoRevit repo will need to be cherry-picked into all the DynamoRevit Release branches that Dynamo supports. Contributors will be responsible for cherry-picking their reviewed commits to the other branches after a LGTM label is added to the PR.

Purpose

Team agreed that we are not going to introduce any setting in Debug Menu. Instead, simply use context menu options under TestMode and to hook up with the work @reddyashish and @Dewb are working on.

image

With this setting, Python nodes can switch Engine anytime in debug mode.
The example below distinguish the case that print statement got changed in Python3 so that same statement in left (python3 node) will throw a warning instead of execute.
image

Declarations

Check these if you believe they are true

  • The codebase is in a better state after this PR
  • Is documented according to the standards
  • The level of testing this PR includes is appropriate
  • User facing strings, if any, are extracted into *.resx files
  • All tests pass using the self-service CI.
  • Snapshot of UI changes, if any.
  • Changes to the API follow Semantic Versioning and are documented in the API Changes document.

Reviewers

@DynamoDS/dynamo

FYIs

@QilongTang QilongTang changed the base branch from master to DSCPython3 April 21, 2020 01:39
@QilongTang QilongTang changed the title [ WIP ] Python Engine Switcher for internal testing Python Engine Switcher for internal testing Apr 21, 2020
@QilongTang
Copy link
Contributor Author

Self serve is a pass, the failing test is a sporadic failure

/// </summary>
private void UpdateToPython2Engine()
{
pythonNodeModel.Engine = PythonNodeBase.DefaultPythonEngine;
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think it's a good idea to rely on PythonNodeBase.DefaultPythonEngine pointing to version 2, as we should eventually make version 3 the default, and then this code would break. Is there some other constant/enum that we can use that's explicitly about Python 2.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@mmisol If we change the value for PythonNodeBase.DefaultPythonEngine, we should update the code here. Let me add a TODO, this is the only new flag we added, there is no other indication. Also I think this alpha test code will be removed later.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I agree with @mmisol, this should check for 2 directly.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@Dewb @mmisol Do you mean checking the actual python2 string value instead of this property? If that's the case, how about we rename it to not call it default? From the API standpoint, referencing the default still do not give me a clue which engine version it is

Copy link
Collaborator

@mmisol mmisol Apr 23, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I just checked out the branch to take a look at the code. Renaming the DefaultPythonEngine constant to PythonNet2Engine (or something with a 2 in it :) ) would be good to me.
Edit: Copying relevant code for reference

        public static readonly string DefaultPythonEngine = "IronPython2";
        public static readonly string PythonNet3Engine = "CPython3";

        private string engine = DefaultPythonEngine;

        public string Engine
        {
            get { return engine; }
            set
            {
                if (engine != value)
                {
                    engine = value;
                    RaisePropertyChanged("Engine");
                }
            }
        }

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Order should not be a problem if they were serialized as strings, but I agree that you lose the flexibility of maybe having other python engines loaded dynamically.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I agree enums would be superior here. What we really would like is a type-checked enum that can be dynamically populated by what assemblies you have loaded (so the enum only contains CPython3 if DSCPython.dll is loaded and DSIronPython.dll is not.)

This sort of fancy, module-spanning dynamic enum has been built in C++ for other products. I'm not sure there's an easy technical solution in C#, but it seems possible. I think this is a good area to investigate further in the tasks to make the engines optional; it doesn't need to gate merging this feature.

It might be a good idea to make the property non-public so we preserve the ability to change the API before the Python 3 features are finalized. The property will have to be serialized to a string in .dyn files regardless, so a move to enums won't have to break graphs.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah, I see the source of the confusion now. The DefaultPythonEngine static property was not intended to be "the name of Python 2 engine" it was intended to mean "which of the possible values of this string-faked enumeration should be assigned to nodes that do not have a value of the Engine property at deserialization time." The idea was it would be set once here, and referenced wherever we needed to, and in the future when we make the default 3, we can change it in one place.

Ideally, the NodeModel code wouldn't know about the details of either of these engines. We could create a RegisteredPythonEngines data structure and each engine DLL could call a RegisterPythonEngine method during initialization, providing a serialization name and an entry point.

We should also separate the concept of "the default value of this property on new nodes" from "the value we attach for graphs from 2.6 and earlier." We will want to change the default engine from 2 to 3 in the future, but pre-2.6 graphs should always be set to 2.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

these are important point's Mike - thanks for clarifying them.

Copy link
Contributor Author

@QilongTang QilongTang Apr 24, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

hmmm I think contractor's work especially the UI part and editor switch will need the whole series of the values somewhere. Do you have a preference between Enums with display name or set of strings? I think we need them available anyway. We can implement the dynamic list later but probably need a static list to solve the immediate needs.

@QilongTang
Copy link
Contributor Author

Hi @DynamoDS/dynamo and @Dewb I have added Enums with display value instead of static string from our discussion to remove the whole confusion around version checking. Currently it is as

    /// <summary>
    /// Enum of possible values of python engine versions
    /// </summary>
    public enum PythonEngineVersions
    {
        [Display(Name = "IronPython2")]
        IronPython2 = 0,

        [Display(Name = "CPython3")]
        CPython3 = 1,
    }

Maybe for the enum values we can move to use the actual version number so they are still identical plus we reserve more information? What do you think. I am supportive of getting version info loading time as a future improvement as well. Notice after the change, the graph schema is still consistent with before.

@Dewb
Copy link
Contributor

Dewb commented Apr 27, 2020

I don't think we need to maintain both a string and an enum version of the property for compatibility. The enum alone should be sufficient.

On the other hand, it might be a good idea to go ahead and build the engine registration data structure now. Something like:

public class PythonEngine
{
   public string Name;
   public string Version;
   public Func<string, IList, IList, object> Evaluator; 
   public bool IsDeprecated;
}

Dictionary<String, PythonEngine> RegisteredPythonEngines;
RegisterPythonEngine(PythonEngine engine)
{
   RegisteredPythonEngines.Add(engine.Name, engine);
}

and then call RegisterPythonEngine inside a static constructor in each engine assembly.

@QilongTang
Copy link
Contributor Author

QilongTang commented Apr 28, 2020

Addressed some comments, please let me know. @Dewb The string property is there so that serialization can happen without a dedicated converter, the default serialization logic for enum is its number value.

Due to the impact of this work to what @SHKnudsen is doing, I would like to wrap up and sync with the UI work potentially building on top of it. Plus this should give @SHKnudsen an easier way to test the engine switch. I would like to address the good points raised by @Dewb here in the follow up Jira task.

@mmisol
Copy link
Collaborator

mmisol commented Apr 28, 2020

@QilongTang How about having a single property called Engine, that is an enum, but serializes as a string? I think that should be the simplest approach and it should be possible if we use the [JsonConverter(typeof(StringEnumConverter))] attribute on top of the property.
https://stackoverflow.com/questions/2441290/javascriptserializer-json-serialization-of-enum-as-string

Also, is there a reason to keep the Display attributes now?

@QilongTang
Copy link
Contributor Author

@mmisol Thank you for pointing out. Addressed your comments and cleaned up changes

Copy link
Collaborator

@mmisol mmisol left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good to me but please address that last comment

@QilongTang
Copy link
Contributor Author

@mmisol Removed. Merging

@QilongTang QilongTang merged commit 9d7b630 into DSCPython3 Apr 28, 2020
@QilongTang QilongTang deleted the FlipPythonEngine branch April 28, 2020 16:41
@QilongTang QilongTang mentioned this pull request Apr 28, 2020
7 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

6 participants