Skip to content

Simplify how we do fix all tests.#26826

Merged
heejaechang merged 2 commits intodotnet:masterfrom
CyrusNajmabadi:simplifyFixAllTests
May 15, 2018
Merged

Simplify how we do fix all tests.#26826
heejaechang merged 2 commits intodotnet:masterfrom
CyrusNajmabadi:simplifyFixAllTests

Conversation

@CyrusNajmabadi
Copy link
Copy Markdown
Contributor

No description provided.

@CyrusNajmabadi CyrusNajmabadi requested a review from a team as a code owner May 13, 2018 01:31
@CyrusNajmabadi CyrusNajmabadi changed the title WIP: Simplify how we do fix all tests. Simplify how we do fix all tests. May 13, 2018
@CyrusNajmabadi
Copy link
Copy Markdown
Contributor Author

Tagging @dotnet/roslyn-ide @heejaechang @mavasani . When the test harness was updated to support fix-all, it was done in a somewhat unclean fashion. Specifically, tests had to have carnal knowledge of how the code action generated its internal state, and it they had to respecify that information so the harness could properly fix all.

This violated the intent of the code-action tests. The purpose of the tests is to simulate the experience of a user interacting with a lightbulb. In that regard all we want the test to specify is:

  1. the code the user is operating on.
  2. where in the file their cursor/selection is.
  3. which item in the lightbulb they choose (with the top one being the default).
  4. if they wanted fix-all in doc/proj/solution
  5. any special options they may have set.

Given that, we then should be able to run all the code as necessary and check the resultant code against hte baseline.

Importantly, if the user picks something like 'fix all in document', we now properly compute that by figuring out what action they wanted, then actually going the 'fix all' route with that action (and it's equivalence key) instead of hte test having to provide any low level details about this.

@CyrusNajmabadi
Copy link
Copy Markdown
Contributor Author

@sharwell as well. This is a test only change that helps simplify tests and cleans upa bunch of stuff.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@mavasani @sharwell can either of you tell me what the original bug actually was? I want to ensure we have a proper test for it. Importantly, a test should not have any clue about thigns like 'equivalence keys'. It's a low level internal implementation detail, and validating what that value it tells you nothing about if the fix is actually doing the right thing.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@mavasani @sharwell Can either of you comment on this? I want to ensure that there is a test in place for the internal VS issue. Thanks!

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This tests seems to have been added as part of feature work, rather than as a regression test (Bug 774321: CodeCoverage: (potential test hole) class 'ConversionGenerator' has zero coverage fixed with changeset 1331072: We now offer to generate conversion methods in VB and C#. I think deleting this test is the right thing to do.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah, thanks much! Definitely a bit of a wacky test.

@CyrusNajmabadi CyrusNajmabadi force-pushed the simplifyFixAllTests branch 2 times, most recently from 5d442d0 to f2bca24 Compare May 13, 2018 04:05
Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

note: i'm not a huge fan of this pattern i'm using here. Effectively, the test says "the user wants to invoke the action at index 'N'". We then go and pass that along to GetCodeActions which will find the action at that index. However, if fix-all is requested, a new code action for the fix-all is synthesized and it is that action that is returned. Since htat is the only action returned, we clearly cannot index into the returned list with the original "index", so the callee returns the index that should be used.

A much cleaner way to do this would be to just return the action to invoke. However, we have a bunch of code that examines all the actions (for thigns like checking spans/titles/etc.).

However, i think i can clean this up by returning the list of actions and the specific action to invoke. i will try to make that change next.

@CyrusNajmabadi CyrusNajmabadi force-pushed the simplifyFixAllTests branch 4 times, most recently from f67c66e to 0526a31 Compare May 13, 2018 06:49
@jinujoseph jinujoseph added Area-IDE Community The pull request was submitted by a contributor who is not a Microsoft employee. labels May 13, 2018
public async Task Tuple_IntroduceLocalForAllOccurrences()
{
// Cannot refactor tuple as local constant
await Assert.ThrowsAsync<Xunit.Sdk.InRangeException>(() =>
Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

these tests were just innapropriate. They were depending on test code actually throwing specific errors. I've replaced then with the actual descriptive methods they should be calling.

@CyrusNajmabadi
Copy link
Copy Markdown
Contributor Author

Tagging @dotnet/roslyn-ide @heejaechang @mavasani @sharwell Can you take a look at this test-only cleanup. Thanks!

Copy link
Copy Markdown
Contributor

@mavasani mavasani left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the cleanup!

@CyrusNajmabadi
Copy link
Copy Markdown
Contributor Author

@jinujoseph Can this test-only change go in?

@jinujoseph
Copy link
Copy Markdown
Contributor

Approved to merge for 15.8.Preview3

@heejaechang heejaechang merged commit 243dd30 into dotnet:master May 15, 2018
@CyrusNajmabadi
Copy link
Copy Markdown
Contributor Author

Thanks!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Approved to merge Area-IDE Community The pull request was submitted by a contributor who is not a Microsoft employee.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants