Skip to content

Conversation

@SHKnudsen
Copy link
Contributor

@SHKnudsen SHKnudsen commented Apr 13, 2021

Purpose

Still a few things to do here, but opening this as a draft for now so its easier to discuss.

This PR does:

  • Adds new LinterManager to DynamoCore.
    • The LinterManager keeps track of the currently active and all available linter extensions using LinterExtensionDescriptors.
    • The LinterManager is in charge of subscribing to the nessercary node/graph events and requestion evaluation of the Active linters rules.
    • The LinterManager gets notified when a rule has been evaluated by the LinterRules static event RuleEvaluated, if the result is failed it gets stored in the linter manager and if it passes it will be removed.
  • Adds Linter rule classes. Linter rules will all inherit from the LinterRule base class, currently there is an implementation for Graph and Node rules, which extensions can use to make their own rules.
  • Adds LinterExtensionBase base class which new Linter extensions needs to inherit from. The base class handles evaluation of Rules when they are requested by the manager

Declarations

Check these if you believe they are true

  • The codebase is in a better state after this PR
  • Is documented according to the standards
  • The level of testing this PR includes is appropriate
  • User facing strings, if any, are extracted into *.resx files
  • All tests pass using the self-service CI.
  • Snapshot of UI changes, if any.
  • Changes to the API follow Semantic Versioning and are documented in the API Changes document.
  • This PR modifies some build requirements and the readme is updated

Reviewers

@saintentropy @nate-peters @BogdanZavu

FYIs

  • The TestLinterExtension is only here for testing purposes and will be removed before merging. The test extension should go into the DynamoSamples project at some point
  • The most pressing concern right now is probably how we decide which events each rule reacts to, the current implementation evaluates all rules no matter what event triggers the request, which is not great. Also still looking at a good way to make it possible to request a rule evaluation outside of the events specified in the manager
  • In the process of adding tests

@mjkkirschner
Copy link
Member

mjkkirschner commented Apr 14, 2021

@SHKnudsen @BogdanZavu @nate-peters @saintentropy @pinzart90 @QilongTang @sm6srw @aparajit-pratap
here are some high level thoughts/a rough proposal regarding where this works lives, how it's introduced as an API, and using it as a jumping off point for other improvements in our process / API organization.

DynamoCore / DynmoCoreWPF is quite large - to write an extension currently you must reference DynamoCore nuget packages.

A couple issues here:

  • the boundary between extensions and the rest of DynamoCore quickly become vague. Clients get access to things like the entire DynamoView or cast interfaces to concrete implementations - this means that we don't get feedback that the API should be expanded and the extensions become coupled to those fuzzy boundaries.
  • the nuget package supports many use cases, IMO it would be nice to grab a nuget based on the use case so it was more obvious which ones to reference and what assemblies to look through for useful methods. Especially since we don't have much in the way of written tutorials or guides on these integration subjects.
  • Dynamo core will continue to grow and grow - making deliberate definition of the APIs more difficult.
    (adskers see links below)

Okay, why am I pointing this out now?

This work appears to me to be geared almost exclusively to extension authors - which is great, an expansion of our extension API! 🎉

IMO it could serve as a nice starting point for the creation of a Dynamo.Extension (Dynamo.Extension.WPF might one day exist as well) nuget package.

My rough proposal is as follows:

  • this work (linter manager) can be merged to core, everything that is public should be made internal - the interfaces might stay public - but maybe they are moved to the proposed Dynamo.Extension.dll binary or perhaps DynamoServices.dll which contains many interfaces and has no dependency on DynamoCore.dll
  • A new .csproject and nuget/nuspec package are created DynamoVisualProgramming.Extensions - we only release this with -beta flag at first to experiment with it.
  • the nuget is published with dynamo's build.
  • the new binary is given access to the internal members of DynamoCore via InternalVisibleTo attribute.
  • The new binary exposes the public apis of the Linter extension.
  • We slowly do the same for other extension apis, obsoleting them, migrating them to internal, and wrapping them deliberately in the new binary.
  • Extensions that need to use the linter will reference the new binary, they will likely reference DynamoCore as well for a time, maybe a long time - but I think it will be instructive to see which use cases can be fully migrated to the new package.

thoughts?

for some more (and possibly outdated info) please see:
https://wiki.autodesk.com/pages/viewpage.action?pageId=692803375
https://wiki.autodesk.com/pages/viewpage.action?pageId=554340892

@BogdanZavu
Copy link
Contributor

@mjkkirschner let's have a meeting tomorrow about this since making this architectural change in the way Dynamo handles extensions will increase the scope of our work a little bit.
We plan to have very little public API and the linter manager and the rules will not make use of anything from viewmodel. We will provide a base extension with almost all functionality already implemented and an API extension dev that will subclass our default extension will only be allowed to add these linting rules and not much more.


namespace Dynamo.Linting
{
public class LinterExtensionDescriptor
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copy link
Member

@mjkkirschner mjkkirschner Apr 29, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@BogdanZavu @SHKnudsen

  1. does this class need to exist? See the very similarExtensionData class, could this just be a tuple instead? Do we expect this to grow?
  2. Does this class really need to be public?

Copy link
Contributor

@BogdanZavu BogdanZavu Apr 29, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So ExtensionData has a few more things in it and I feel that using it instead of LinterDescriptor will make things more confusing. I guess the most valuable thing of LinterDescriptor is the actual name. I would expect that there are many similar or almost similar simple structs across dynamo. I think we can have a tuple here but I would prefer the named structure instead. Most likely it will not grow or if it will grow it will grow very little.
I think it can be internal.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@BogdanZavu - sounds good to me if thats possible.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I forgot about the view, we need it there (in the other PR ) where the active linter is being set from the UI.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ah

@SHKnudsen SHKnudsen mentioned this pull request Apr 20, 2021
8 tasks
@BogdanZavu
Copy link
Contributor

@SHKnudsen overall I think this is looking really good! Let's clarify the serialization aspect tomorrow and I think we can merge it.

@SHKnudsen SHKnudsen marked this pull request as ready for review April 28, 2021 08:11
@SHKnudsen
Copy link
Contributor Author

Hmm, i cant access the SelfServe details, so not sure what this issue is..

@BogdanZavu
Copy link
Contributor

BogdanZavu commented Apr 28, 2021

linter-tests.txt

@BogdanZavu
Copy link
Contributor

Only 1 test remaining :

DynamoCoreWpfTests.SerializationTests.JSONisSameBeforeAndAfterSaveWithDummyNodes

MESSAGE:
Expected: True
But was: False

+++++++++++++++++++
STACK TRACE:
at NUnit.Framework.Assert.That(Object actual, IResolveConstraint expression, String message, Object[] args)
at DynamoCoreWpfTests.SerializationTests.JSONisSameBeforeAndAfterSaveWithDummyNodes() in c:\WorkspaceDynamo\test\DynamoCoreWpfTests\SerializationTests.cs:line 444

/// <param name="verboseLogging">Indicates if detailed descriptions should be logged</param>
/// <param name="isTestMode">Indicates if current code is running in tests</param>
/// <param name="fileName">Name of file where the workspace is saved</param>
[Obsolete("please use the version with linterManager parameter.")]
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

some questions on this:

  1. have you verified that all these constructors can be replaced by your new constructor? - What I mean is, do they cover any other use cases?
  2. If a file is deserialized with no linterManager - should null be passed?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

  • i would say I've tested this as much as possible, also all different constructors have gotten a new equivalent just with the linterManager param. Before the changes there were 3 ctors, I've add 3 new ones that replicate the old ones but with that new param.
  • Essentially the only thing that is serialized from the linter manager is the active linter (name and id) and issue counts (the counts are only for things like GD and player). If there is no active linter when a graph is saved we just serialize Linting : [], when deserializing all we are doing is setting the active linter to whatever was serialized (if it's available on the manager). The linter manager is created on the DynamoModel, so if a workspace is deserialized without an active linter value we just don't set any active linter on the LinterManager.

@BogdanZavu BogdanZavu merged commit 2ac55d6 into DynamoDS:AGD-2060-Linting-API Apr 30, 2021
@SHKnudsen SHKnudsen mentioned this pull request May 7, 2021
8 tasks
BogdanZavu pushed a commit that referenced this pull request May 7, 2021
* Initial commit

* Move event subscription to extension base class + add property changed filter to rules

* comment updates

* Update LinterExtensionBase.cs

* Update InputNodesNotAllowedRule.cs

* Add Deactivate method to linterExtensionBase + LinterManager tests

* comment updates

* Dispose LinterManager

* comment updates

* Serialize linter manager to dyn file

* clear ruleEvaluationResults on workspace change

* Remove test extension

* Handle GraphRuleEvaluationResults nodeIds changes

* Update SerializationConverters.cs

* Fix failing serialization test

* comment updates

* revert changes to test file
BogdanZavu pushed a commit that referenced this pull request May 11, 2021
* Linter manager (#11606)

* Initial commit

* Move event subscription to extension base class + add property changed filter to rules

* comment updates

* Update LinterExtensionBase.cs

* Update InputNodesNotAllowedRule.cs

* Add Deactivate method to linterExtensionBase + LinterManager tests

* comment updates

* Dispose LinterManager

* comment updates

* Serialize linter manager to dyn file

* clear ruleEvaluationResults on workspace change

* Remove test extension

* Handle GraphRuleEvaluationResults nodeIds changes

* Update SerializationConverters.cs

* Fix failing serialization test

* comment updates

* revert changes to test file

* Linter ViewExtension (#11634)

* Initial commit

* Move event subscription to extension base class + add property changed filter to rules

* comment updates

* Update LinterExtensionBase.cs

* Update InputNodesNotAllowedRule.cs

* Add Deactivate method to linterExtensionBase + LinterManager tests

* comment updates

* Initial commit

* Add issues count to WorkspaceSaving

* add scrollviewer

* Dispose LinterManager

* Make concrete issues internal

* summaries on IRuleIssue

* comment updates

* Update severity code icon

* Serialize linter manager to dyn file

* clear ruleEvaluationResults on workspace change

* Remove test extension

* Show names instead of GUIDs

* add help doc

* clean up

* Handle GraphRuleEvaluationResults nodeIds changes

* Update LintingViewExtension.csproj

* Update LintingViewExtension.csproj

* Update SerializationConverters.cs

* Fix failing serialization test

* comment updates

* revert changes to test file

* fix available linters binding

* add rules back in test ext

* comment updates

* remove output path

* Update LinterManagerTests.cs

* comment updates

* hide LinterViewExtension and LinterExtensionBase

* Update LinterViewModel.cs

* if statements

* remove Prism

* clean up
BogdanZavu added a commit that referenced this pull request May 11, 2021
* Linter manager (#11606)

* Initial commit

* Move event subscription to extension base class + add property changed filter to rules

* comment updates

* Update LinterExtensionBase.cs

* Update InputNodesNotAllowedRule.cs

* Add Deactivate method to linterExtensionBase + LinterManager tests

* comment updates

* Dispose LinterManager

* comment updates

* Serialize linter manager to dyn file

* clear ruleEvaluationResults on workspace change

* Remove test extension

* Handle GraphRuleEvaluationResults nodeIds changes

* Update SerializationConverters.cs

* Fix failing serialization test

* comment updates

* revert changes to test file

* Linter ViewExtension (#11634)

* Initial commit

* Move event subscription to extension base class + add property changed filter to rules

* comment updates

* Update LinterExtensionBase.cs

* Update InputNodesNotAllowedRule.cs

* Add Deactivate method to linterExtensionBase + LinterManager tests

* comment updates

* Initial commit

* Add issues count to WorkspaceSaving

* add scrollviewer

* Dispose LinterManager

* Make concrete issues internal

* summaries on IRuleIssue

* comment updates

* Update severity code icon

* Serialize linter manager to dyn file

* clear ruleEvaluationResults on workspace change

* Remove test extension

* Show names instead of GUIDs

* add help doc

* clean up

* Handle GraphRuleEvaluationResults nodeIds changes

* Update LintingViewExtension.csproj

* Update LintingViewExtension.csproj

* Update SerializationConverters.cs

* Fix failing serialization test

* comment updates

* revert changes to test file

* fix available linters binding

* add rules back in test ext

* comment updates

* remove output path

* Update LinterManagerTests.cs

* comment updates

* hide LinterViewExtension and LinterExtensionBase

* Update LinterViewModel.cs

* remove unused test files

* Fix pr comments (#11677)

* Linter manager (#11606)

* Initial commit

* Move event subscription to extension base class + add property changed filter to rules

* comment updates

* Update LinterExtensionBase.cs

* Update InputNodesNotAllowedRule.cs

* Add Deactivate method to linterExtensionBase + LinterManager tests

* comment updates

* Dispose LinterManager

* comment updates

* Serialize linter manager to dyn file

* clear ruleEvaluationResults on workspace change

* Remove test extension

* Handle GraphRuleEvaluationResults nodeIds changes

* Update SerializationConverters.cs

* Fix failing serialization test

* comment updates

* revert changes to test file

* Linter ViewExtension (#11634)

* Initial commit

* Move event subscription to extension base class + add property changed filter to rules

* comment updates

* Update LinterExtensionBase.cs

* Update InputNodesNotAllowedRule.cs

* Add Deactivate method to linterExtensionBase + LinterManager tests

* comment updates

* Initial commit

* Add issues count to WorkspaceSaving

* add scrollviewer

* Dispose LinterManager

* Make concrete issues internal

* summaries on IRuleIssue

* comment updates

* Update severity code icon

* Serialize linter manager to dyn file

* clear ruleEvaluationResults on workspace change

* Remove test extension

* Show names instead of GUIDs

* add help doc

* clean up

* Handle GraphRuleEvaluationResults nodeIds changes

* Update LintingViewExtension.csproj

* Update LintingViewExtension.csproj

* Update SerializationConverters.cs

* Fix failing serialization test

* comment updates

* revert changes to test file

* fix available linters binding

* add rules back in test ext

* comment updates

* remove output path

* Update LinterManagerTests.cs

* comment updates

* hide LinterViewExtension and LinterExtensionBase

* Update LinterViewModel.cs

* if statements

* remove Prism

* clean up

Co-authored-by: BogdanZavu <zavu_bogdan@yahoo.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants