Skip to content

[@types/underscore] Collection and Array Tests - Map, Pluck, Filter, GroupBy, and Flatten#45763

Merged
typescript-bot merged 36 commits intoDefinitelyTyped:masterfrom
reubenrybnik:underscore-tests-map-pluck-filter-groupBy-flatten
Jul 5, 2020
Merged

[@types/underscore] Collection and Array Tests - Map, Pluck, Filter, GroupBy, and Flatten#45763
typescript-bot merged 36 commits intoDefinitelyTyped:masterfrom
reubenrybnik:underscore-tests-map-pluck-filter-groupBy-flatten

Conversation

@reubenrybnik
Copy link
Copy Markdown
Contributor

@reubenrybnik reubenrybnik commented Jun 28, 2020

  • Use a meaningful title for the pull request. Include the name of the package modified.
  • Test the change in your own code. (Compile and run.)
  • Add or edit tests to reflect the change. (Run with npm test.)
  • Follow the advice from the readme.
  • Avoid common mistakes.
  • Run npm run lint package-name (or tsc if no tslint.json is present).
  • Provide a URL to documentation or source code which provides context for the suggested changes: https://underscorejs.org/
  • Include tests for your changes

This PR continues the planned set that will together add up to #45201 and includes the following changes:

  • Adds tests for map, pluck, filter, groupBy, and flatten.
  • Updates iterator callback interfaces to provide a more specific collection type when one is available.
  • Adds an Iteratee type that handles all the cases Underscore allows for iteratees and drops the matcher shorthand of Dictionary<T> for map since that isn't accurate (it should be and now is Partial<T>).
  • Updates the return types for the map to IterateeResult to provide more specific return types for non-function iteratees where possible.
  • Updates map, filter, and groupBy to use Iteratee.
  • Updates pluck to result in a specific type rather than any if it is able to do so while still accepting unknown strings to handle type unions with differing properties and to avoid re-introducing [@types/underscore] v1.8.10 throws error on basic pluck usage. #33479.
  • Updates the return types of _Chain.filter, _Chain.groupBy, and special cases of _Chain.flatten to use the correct wrapped value type V to partially fix @types/underscore error TS2322 for _.chain after upgrade to v1.9 #36308.
  • Updates pluck and groupBy to work with both Lists and Dictionaries to partially fix Underscore collections functions should take objects #20623.
  • Updates flatten to use conditional types to be able to provide correctly flattened shallow and deep return types for arrays of up to two dimensions and deprecates _ChainOfArrays (also removing that interface as a return type for map and groupBy).
  • Replaces all collect overloads with a reference to map overloads and all select overloads with a reference to filter overloads.

Changes to filter and _Chain.tap are included in this PR because some existing chain tests that involve those functions followed by map became unhappy without updates to their return types. This is expected to be the largest and most complicated of the PRs in this set.

@typescript-bot typescript-bot added Popular package This PR affects a popular package (as counted by NPM download counts). Author is Owner The author of this PR is a listed owner of the package. labels Jun 28, 2020
@typescript-bot
Copy link
Copy Markdown
Contributor

typescript-bot commented Jun 28, 2020

@reubenrybnik Thank you for submitting this PR!

This is a live comment which I will keep updated.

Code Reviews

Because you edited one package and updated the tests (👏), I can help you merge this PR once someone else signs off on it.

Status

  • ✅ No merge conflicts
  • ✅ Continuous integration tests have passed
  • ✅ Most recent commit is approved by type definition owners or DT maintainers

All of the items on the list are green. To merge, you need to post a comment including the string "Ready to merge" to bring in your changes.


Diagnostic Information: What the bot saw about this PR
{
  "type": "info",
  "now": "-",
  "pr_number": 45763,
  "author": "reubenrybnik",
  "owners": [
    "borisyankov",
    "jbaldwin",
    "ccurrens",
    "confususs",
    "jgonggrijp",
    "ffflorian",
    "regevbr",
    "peterblazejewicz",
    "reubenrybnik"
  ],
  "dangerLevel": "ScopedAndTested",
  "headCommitAbbrOid": "6bd9cb7",
  "headCommitOid": "6bd9cb75d38e5ba10fcfcf81dbd3792c918bccf6",
  "mergeIsRequested": true,
  "stalenessInDays": 0,
  "lastCommitDate": "2020-07-04T22:34:09.000Z",
  "reopenedDate": "2020-06-30T16:35:01.000Z",
  "lastCommentDate": "2020-07-05T00:29:03.000Z",
  "reviewLink": "https://github.com/DefinitelyTyped/DefinitelyTyped/pull/45763/files",
  "hasMergeConflict": false,
  "authorIsOwner": true,
  "isFirstContribution": false,
  "popularityLevel": "Popular",
  "anyPackageIsNew": false,
  "packages": [
    "underscore"
  ],
  "files": [
    {
      "filePath": "types/underscore/index.d.ts",
      "kind": "definition",
      "package": "underscore"
    },
    {
      "filePath": "types/underscore/underscore-tests.ts",
      "kind": "test",
      "package": "underscore"
    }
  ],
  "hasDismissedReview": false,
  "ciResult": "pass",
  "lastReviewDate": "2020-07-05T00:24:12.000Z",
  "reviewersWithStaleReviews": [],
  "approvalFlags": 2,
  "isChangesRequested": false
}

@typescript-bot
Copy link
Copy Markdown
Contributor

🔔 @borisyankov @jbaldwin @ccurrens @confususs @jgonggrijp @ffflorian @regevbr @peterblazejewicz - please review this PR in the next few days. Be sure to explicitly select Approve or Request Changes in the GitHub UI so I know what's going on.

@typescript-bot typescript-bot added the The CI failed When GH Actions fails label Jun 28, 2020
@typescript-bot
Copy link
Copy Markdown
Contributor

@reubenrybnik The CI build failed! Please review the logs for more information.

Once you've pushed the fixes, the build will automatically re-run. Thanks!

@reubenrybnik
Copy link
Copy Markdown
Contributor Author

reubenrybnik commented Jun 28, 2020

This PR is dependent on the changes in #45748 for its CI check to succeed, but it was useful to be able to show the CI check failing, which is why I opened it early. I'm going to convert it to a draft for now, but reviewers may still feel free to start reviewing it if they'd like since aside from that check failure this PR is ready for review.

@jgonggrijp
Copy link
Copy Markdown
Contributor

Hey @reubenrybnik, I'll be happy to review this one as well, but I'll wait what happens in #45748 first.

@reubenrybnik
Copy link
Copy Markdown
Contributor Author

Sounds good, thanks for following up @jgonggrijp!

@typescript-bot
Copy link
Copy Markdown
Contributor

👋 Hi there! I’ve run some quick measurements against master and your PR. These metrics should help the humans reviewing this PR gauge whether it might negatively affect compile times or editor responsiveness for users who install these typings.

Let’s review the numbers, shall we?

Comparison details 📊
master #45763 diff
Batch compilation
Memory usage (MiB) 51.9 55.3 +6.7%
Type count 9729 10029 +3%
Assignability cache size 1495 1356 -9%
Language service
Samples taken 2160 4357 +102%
Identifiers in tests 2160 4357 +102%
getCompletionsAtPosition
    Mean duration (ms) 158.8 173.3 +9.1%
    Mean CV 11.2% 11.1%
    Worst duration (ms) 358.5 305.5 -14.8%
    Worst identifier _ mixedIterabilityValue
getQuickInfoAtPosition
    Mean duration (ms) 152.6 167.0 +9.4%
    Mean CV 11.4% 11.4% -0.2%
    Worst duration (ms) 375.2 306.6 -18.3%
    Worst identifier extractChainTypes extractChainTypes

It looks like nothing changed too much. I won’t post performance data again unless it gets worse.

@typescript-bot typescript-bot added the Perf: Same typescript-bot determined that this PR will not significantly impact compilation performance. label Jun 29, 2020
@typescript-bot typescript-bot removed Self Merge This PR can now be self-merged by the PR author or an owner Owner Approved A listed owner of this package signed off on the pull request. labels Jul 4, 2020
@reubenrybnik
Copy link
Copy Markdown
Contributor Author

Thanks for making this PR super extra excellent with the addition of Iteratee and IterateeResult @jgonggrijp! The only change I made in this last round of commits that you might not have expected is that I updated string | number constraints in the iteratee types to the built-in PropertyKey type of string | number | symbol. I tried out using symbols as iteratees in JS to sanity check that Underscore works with them before doing so and didn't run into any issues. Whether or not I made any changes that you might not approve of, though, DT still requires the most recent commit to be approved so I'm still going to need you to give this one more quick click to be able to merge it.

As I mentioned, I expected this to be the most complicated PR of the set; hopefully it will all be downhill from here 😄

@reubenrybnik
Copy link
Copy Markdown
Contributor Author

Hmm, darn, I plugged this into my company's solution to sanity check all these changes and a pair of unpleasant things happened that I need to look into.

  1. A collection of type any + a function iterator goes past the extends CollectionIterator check in IterateeResult so even with a function one ends up with boolean[] as a result.
  2. Making chain more value-centric when some chain values are still messed up breaks existing chains, which will eventually be fixed at the end of this PR set along with the previously-broken .value() calls for any chain ending with those functions, but it would still be nice to keep them working as well as they were until the end of this set is reached.

…atee function since that's really all that matters from its perspective to fix issues with using function iteratees on values of type any.
@reubenrybnik
Copy link
Copy Markdown
Contributor Author

reubenrybnik commented Jul 4, 2020

Hmm, darn, I plugged this into my company's solution to sanity check all these changes and a pair of unpleasant things happened that I need to look into.

  1. A collection of type any + a function iterator goes past the extends CollectionIterator check in IterateeResult so even with a function one ends up with boolean[] as a result.
  2. Making chain more value-centric when some chain values are still messed up breaks existing chains, which will eventually be fixed at the end of this PR set along with the previously-broken .value() calls for any chain ending with those functions, but it would still be nice to keep them working as well as they were until the end of this set is reached.
  1. Has permanent fix f84b2ea
  2. Has a temporary workaround da8ffa6 that introduces a _ChainIteratee temporary type that can be dropped at the end of this PR set

After applying these fixes, my company's solution now has four new and IMO correct errors when these changes are applied. One is a never iteratee error caused by calling filter on a non-collection, two are implicit any errors caused by calling filter on an any-type value and not explicitly providing a value type for the iteratee callback, and one is three are an issue with an incorrect non-collection chain return type in uniq (which will be fixed in a later PR) that is surfaced by pluck returning a specific and correct type now instead of any and by filter returning a correct array chain value type instead of a singleton.

reubenrybnik added a commit to reubenrybnik/DefinitelyTyped that referenced this pull request Jul 4, 2020
reubenrybnik added a commit to reubenrybnik/DefinitelyTyped that referenced this pull request Jul 4, 2020
Copy link
Copy Markdown
Contributor

@jgonggrijp jgonggrijp left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for making this PR super extra excellent with the addition of Iteratee and IterateeResult @jgonggrijp!

It was a pleasure! Though you should give me credit only for IterateeResult and the forthcoming reduction in overloads. Iteratee was your idea and it solves a problem I didn't foresee.

The only change I made in this last round of commits that you might not have expected is that I updated string | number constraints in the iteratee types to the built-in PropertyKey type of string | number | symbol. I tried out using symbols as iteratees in JS to sanity check that Underscore works with them before doing so and didn't run into any issues.

Symbols happen to work as iteratees, but I'm not comfortable with encouraging that. Currently, symbols are not actually supported in Underscore; all Underscore functions that proactively extract property names (through _.keys or _.allKeys) restrict themselves to enumerable keys, and properties with symbol keys are never enumerable. If you tell Underscore where to look, like when passing a symbol as iteratee, it will of course find the property, but this is an unfortunate coincidence rather than something you're supposed to do. This has confused people before, for example in jashkenas/underscore#2789.

Having a name for the key type union is useful though. Maybe you can set type EnumerableKey = string | number and then substitute that everywhere for PropertyKey.

Whether or not I made any changes that you might not approve of, though, DT still requires the most recent commit to be approved so I'm still going to need you to give this one more quick click to be able to merge it.

Oops, of course. Next time I expect more changes, I'll save myself the effort of approving ahead of time.

@typescript-bot typescript-bot added the Revision needed This PR needs code changes before it can be merged. label Jul 4, 2020
@typescript-bot
Copy link
Copy Markdown
Contributor

@reubenrybnik One or more reviewers has requested changes. Please address their comments. I'll be back once they sign off or you've pushed new commits or comments. If you disagree with the reviewer's comments, you can "dismiss" the review using GitHub's review UI. Thank you!

@reubenrybnik
Copy link
Copy Markdown
Contributor Author

Having a name for the key type union is useful though. Maybe you can set type EnumerableKey = string | number and then substitute that everywhere for PropertyKey.

Sounds reasonable to me @jgonggrijp, especially since one can still use a function iteratee if one wants to reference symbol properties. I've made the requested change.

@typescript-bot typescript-bot removed the Revision needed This PR needs code changes before it can be merged. label Jul 4, 2020
@reubenrybnik
Copy link
Copy Markdown
Contributor Author

Also just to make sure you had seen them @jgonggrijp after my original message I did end up making two other changes outside of the scope of originally discussed changes that aren't super huge but that you might want to take a quick peek at; see this comment for more details.

Copy link
Copy Markdown
Contributor

@jgonggrijp jgonggrijp left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Also just to make sure you had seen them @jgonggrijp after my original message I did end up making two other changes outside of the scope of originally discussed changes that aren't super huge but that you might want to take a quick peek at; see this comment for more details.

Yes I saw them and I'm not opposed.

Good to go, I think!

@typescript-bot typescript-bot added Owner Approved A listed owner of this package signed off on the pull request. Self Merge This PR can now be self-merged by the PR author or an owner labels Jul 5, 2020
@reubenrybnik
Copy link
Copy Markdown
Contributor Author

Ready to merge

@typescript-bot typescript-bot merged commit f4726d8 into DefinitelyTyped:master Jul 5, 2020
@typescript-bot
Copy link
Copy Markdown
Contributor

I just published @types/underscore@1.10.4 to npm.

ngbrown pushed a commit to ngbrown-forks/DefinitelyTyped that referenced this pull request Jul 11, 2020
…rray Tests - Map, Pluck, Filter, GroupBy, and Flatten by @reubenrybnik

* Adding type extractors and updating existing tests to use them.

* Improving handling of mixed-iterability types by simplifying the Underscore and chain functions to a single overload.

* Updating TypeOfCollection to check against List and Dictionary separately instead of using Collection since using the latter behaves oddly with JQuery objects.

* Renaming a variable to something slightly more friendly.

* Updating type definitions for map, pluck, filter, groupBy, and flatten and removing _ChainOfArrays and IterateeMatcherShorthand.

* Adding tests for map, pluck, filter, groupBy, and flatten.

* Fixing a test for @types/parse that breaks with these changes.

* Updating more places to use returns instead of return because I want to.

* Revising identifiers in tests.

* Switch from $ExpectType to assignability checks for now due to an odd issue with the linter that seems to affect TS 3.0 specifically.

* Updating a word in a comment that I forgot to update.

* Revert "Fixing a test for @types/parse that breaks with these changes."

This reverts commit 1955fc1.

* Fixing a typo in a variable name.

* Dropping KeysOfUnion and TypesOfUnionProperty for now in favor of a safer and simpler keyof T + string approach.

* Reducing the dimensions of a deep flatten and changing the names of the resulting types.

* Dropping array tests for most functions since testing lists should be sufficient to also cover arrays.

* Adding a comment to every test group.

* Update functions that take an iteratee to use an Iteratee type and updating map to use an IterateeResult type.

* Shortening the names of nested list variables.

* Also collapsing pluck to a single overload.

* Switching groupBy back to allowing any rather than attempting to constrain to typical valid key types.

* Dropping unnecessary `keyof T` constraints and updating `string` and `string | number` constraints to `PropertyKey` since from experimentation Underscore works just fine with symbols as well.

* Wrapping return values for multi-line funciton declarations to their own line.

* Updating UnderscoreStatic functions that don't technically need the collection type to use it anyway for consistency with those that do.

* Renaming some mixed-dimension array variables one more time.

* Adding property path tests for groupBy.

* Changing "iterator" to "iteratee" in test group comments.

* Adding property path tests for filter.

* Updating IterateeResult to only care about the return type of an iteratee function since that's really all that matters from its perspective to fix issues with using function iteratees on values of type any.

* Working around some _Chain return types incorrectly using non-collections as their wrapped values.

* Using `T` rather than `TypeOfCollection<V>` in `Underscore.map` since I have it.

* Applying a shorter variable name that I had missed copying over from DefinitelyTyped#45201.

* Fixing consistency issues with two more test variable names.

* Adding EnumerableKey and using that intstead of PropertyKey.
danielrearden pushed a commit to danielrearden/DefinitelyTyped that referenced this pull request Sep 22, 2020
…rray Tests - Map, Pluck, Filter, GroupBy, and Flatten by @reubenrybnik

* Adding type extractors and updating existing tests to use them.

* Improving handling of mixed-iterability types by simplifying the Underscore and chain functions to a single overload.

* Updating TypeOfCollection to check against List and Dictionary separately instead of using Collection since using the latter behaves oddly with JQuery objects.

* Renaming a variable to something slightly more friendly.

* Updating type definitions for map, pluck, filter, groupBy, and flatten and removing _ChainOfArrays and IterateeMatcherShorthand.

* Adding tests for map, pluck, filter, groupBy, and flatten.

* Fixing a test for @types/parse that breaks with these changes.

* Updating more places to use returns instead of return because I want to.

* Revising identifiers in tests.

* Switch from $ExpectType to assignability checks for now due to an odd issue with the linter that seems to affect TS 3.0 specifically.

* Updating a word in a comment that I forgot to update.

* Revert "Fixing a test for @types/parse that breaks with these changes."

This reverts commit 1955fc1.

* Fixing a typo in a variable name.

* Dropping KeysOfUnion and TypesOfUnionProperty for now in favor of a safer and simpler keyof T + string approach.

* Reducing the dimensions of a deep flatten and changing the names of the resulting types.

* Dropping array tests for most functions since testing lists should be sufficient to also cover arrays.

* Adding a comment to every test group.

* Update functions that take an iteratee to use an Iteratee type and updating map to use an IterateeResult type.

* Shortening the names of nested list variables.

* Also collapsing pluck to a single overload.

* Switching groupBy back to allowing any rather than attempting to constrain to typical valid key types.

* Dropping unnecessary `keyof T` constraints and updating `string` and `string | number` constraints to `PropertyKey` since from experimentation Underscore works just fine with symbols as well.

* Wrapping return values for multi-line funciton declarations to their own line.

* Updating UnderscoreStatic functions that don't technically need the collection type to use it anyway for consistency with those that do.

* Renaming some mixed-dimension array variables one more time.

* Adding property path tests for groupBy.

* Changing "iterator" to "iteratee" in test group comments.

* Adding property path tests for filter.

* Updating IterateeResult to only care about the return type of an iteratee function since that's really all that matters from its perspective to fix issues with using function iteratees on values of type any.

* Working around some _Chain return types incorrectly using non-collections as their wrapped values.

* Using `T` rather than `TypeOfCollection<V>` in `Underscore.map` since I have it.

* Applying a shorter variable name that I had missed copying over from DefinitelyTyped#45201.

* Fixing consistency issues with two more test variable names.

* Adding EnumerableKey and using that intstead of PropertyKey.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Author is Owner The author of this PR is a listed owner of the package. Owner Approved A listed owner of this package signed off on the pull request. Perf: Same typescript-bot determined that this PR will not significantly impact compilation performance. Popular package This PR affects a popular package (as counted by NPM download counts). Self Merge This PR can now be self-merged by the PR author or an owner

Projects

None yet

Development

Successfully merging this pull request may close these issues.

@types/underscore error TS2322 for _.chain after upgrade to v1.9 Underscore collections functions should take objects

3 participants