Skip to content

fix(@vitest/expect): support type-safe declaration of custom matchers#7656

Merged
sheremet-va merged 20 commits intovitest-dev:mainfrom
kettanaito:fix/custom-raw-matcher-fn
May 19, 2025
Merged

fix(@vitest/expect): support type-safe declaration of custom matchers#7656
sheremet-va merged 20 commits intovitest-dev:mainfrom
kettanaito:fix/custom-raw-matcher-fn

Conversation

@kettanaito
Copy link
Copy Markdown
Contributor

@kettanaito kettanaito commented Mar 11, 2025

Description

I am proposing this change to improve the type-safety experience when extending matchers.

Problem

Right now, the MatchersObject argument of expect.extend() is not open to extension. Although you can map your custom matchers by doing this...

interface CustomMatchers<R = unknown> {
  toBeFoo: (input: number) => R
}

declare module 'vitest' {
  interface Assertion<T = any> extends CustomMatchers<T> {}
}

...the argument you provide to expect.extend() is still not type-safe:

expect.extend({
  toBeFoo(received, expected) {}
})

The expected argument is not inferred to be a number. The type signature of the raw matchers you provide here is arbitrary and can contradict to the CustomMatchers you have easily without raising a type error.

The issue is that extending the Assertions interface is really meant only to have the custom matchers known on the expect() return type when using it, but not when declaring those custom matchers in expect.extend(). These two are controlled by different types.

Proposed solution

Important

Please see the discussion below and the diff for the up-to-date information on how this feature actually got implemented.

First, here's the experience I'm after:

import 'vitest'

interface CustomMatchers<R = unknown> {
	toBeFoo: (input: number) => R
}

declare module 'vitest' {
	interface Assertion<T = any> extends CustomMatchers<T> {}
}

declare module '@vitest/expect' {
	interface CustomMatchersObject extends CustomMatchers {}
}

expect.extend({
//                        👇 Hooray! This is finally inferred and type-safe!
	toBeFoo(received, expected) {
		return {
			pass: true,
			message: () => 'Passes',
		}
	},
})

This already works locally with the proposed changes and finally completes the type-safety story of custom matchers in Vitest.

The following changes are needed to make this experience happen:

  1. @vitest/expect introduces a placeholder interface CustomMatchersObject. This is meant purely to be extended in the userland as you can see me doing above.
  2. The MatchersObject type is preserved, but now it also remaps the custom matchers from CustomMatchersObject to RawMatcherFn, inferring the arguments (expected values).
  3. The RawMatcherFn has to be extended to accept a new type argument to represent a narrower type for the expected array of values.

You can find the implementation for this in this pull request.

I will document the new functionality and update the Extending matchers recipe once these changes are approved. Thank you.

Please don't delete this checklist! Before submitting the PR, please make sure you do the following:

  • It's really useful if your PR references an issue where it is discussed ahead of time. If the feature is substantial or introduces breaking changes without a discussion, PR might be closed.
  • Ideally, include a test that fails without this PR but passes with it.
  • Please, don't make changes to pnpm-lock.yaml unless you introduce a new test example.

Tests

  • Run the tests with pnpm test:ci.

Documentation

  • If you introduce new functionality, document it. You can run documentation with pnpm run docs command.

Changesets

  • Changes in changelog are generated from PR name. Please, make sure that it explains your changes in an understandable manner. Please, prefix changeset messages with feat:, fix:, perf:, docs:, or chore:.

@netlify
Copy link
Copy Markdown

netlify bot commented Mar 11, 2025

Deploy Preview for vitest-dev ready!

Name Link
🔨 Latest commit ed1243f
🔍 Latest deploy log https://app.netlify.com/projects/vitest-dev/deploys/682b29a66a88550008103eb4
😎 Deploy Preview https://deploy-preview-7656--vitest-dev.netlify.app
📱 Preview on mobile
Toggle QR Code...

QR Code

Use your smartphone camera to open QR code link.

To edit notification comments on pull requests, go to your Netlify project configuration.


export interface CustomMatchersObject extends Record<string, (...args: Array<any>) => any> {}

type CustomMatchersToRawMatchers<T extends CustomMatchersObject> = {
Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This mapping type is necessary because the userland should declare custom matchers like currently showcased in the docs:

interface CustomMatchers<R = unknown> {
  toBeFoo: (input: number) => R
}

The mapping is to correctly annotate the raw matchers that you declare in expect.extend()'s argument (correct this, correct received: unknown, and then inferred expected args).

@kettanaito
Copy link
Copy Markdown
Contributor Author

kettanaito commented Mar 11, 2025

Potential improvements

Ideally, I'd love for the CustomMatchersObject to be extended in vitest directly, but I'm not sure if that's possible. Appreciate any input on this matter.

// This would be perfect! 
declare module 'vitest' {
	interface Assertion<T = any> extends CustomMatchers<T> {}
	interface CustomMatchersObject extends CustomMatchers {}
}

However, given how the responsibility of these types differ, I'd say it does make sense to explicitly extend @vitest/expect because it's that package, and not vitest, that's in charge of expect.extend().

@kettanaito
Copy link
Copy Markdown
Contributor Author

I would love to add a type test for this, too. Please advise me where would be the best place to put it! Thanks.

@pkg-pr-new
Copy link
Copy Markdown

pkg-pr-new bot commented Mar 11, 2025

@vitest/browser

npm i https://pkg.pr.new/@vitest/browser@7656

@vitest/coverage-istanbul

npm i https://pkg.pr.new/@vitest/coverage-istanbul@7656

@vitest/expect

npm i https://pkg.pr.new/@vitest/expect@7656

@vitest/coverage-v8

npm i https://pkg.pr.new/@vitest/coverage-v8@7656

@vitest/mocker

npm i https://pkg.pr.new/@vitest/mocker@7656

@vitest/pretty-format

npm i https://pkg.pr.new/@vitest/pretty-format@7656

@vitest/runner

npm i https://pkg.pr.new/@vitest/runner@7656

@vitest/snapshot

npm i https://pkg.pr.new/@vitest/snapshot@7656

@vitest/spy

npm i https://pkg.pr.new/@vitest/spy@7656

@vitest/ui

npm i https://pkg.pr.new/@vitest/ui@7656

@vitest/utils

npm i https://pkg.pr.new/@vitest/utils@7656

vite-node

npm i https://pkg.pr.new/vite-node@7656

vitest

npm i https://pkg.pr.new/vitest@7656

@vitest/web-worker

npm i https://pkg.pr.new/@vitest/web-worker@7656

@vitest/ws-client

npm i https://pkg.pr.new/@vitest/ws-client@7656

commit: ed1243f

@sheremet-va
Copy link
Copy Markdown
Member

I recognize the issue, but I disagree with the solution. My issues with it:

  1. Type definition is declared several times. Why not infer it from a single definition? (from Assertion)
  2. CustomMatchersObject is augmented on @vitest/expect - this will require you to install the package manually if you are using pnpm or yarn pnp which is a big ask to be honest

@kettanaito
Copy link
Copy Markdown
Contributor Author

@sheremet-va, I'd love for it to infer the raw matcher functions from the Assertions type. Please show me where would be the correct way to infer that, and I will adjust the PR.

The difficulty here is that vitest doesn't control the MatchersObject type so I don't see how it would be able to affect the types we need. Perhaps what you're suggesting is to keep the proposed extensible type in @vitest/expect and only move the infer step to vitest, since it also can extend the CustomMatchersObject now. Was that what you meant?

@sheremet-va
Copy link
Copy Markdown
Member

The difficulty here is that vitest doesn't control the MatchersObject type so I don't see how it would be able to affect the types we need. Perhaps what you're suggesting is to keep the proposed extensible type in @vitest/expect and only move the infer step to vitest, since it also can extend the CustomMatchersObject now. Was that what you meant?

As long as it's exporting them, it should work fine. vitest also doesn't control Assertion type, but you augment vitest to add custom assertions

I'd love for it to infer the raw matcher functions from the Assertions type. Please show me where would be the correct way to infer that, and I will adjust the PR.

I cannot give any pointers, I'm sorry. Feel free to experiment.

@kettanaito
Copy link
Copy Markdown
Contributor Author

kettanaito commented Mar 11, 2025

I gave this a try in the latest commit, but I cannot get the inference to work for some reason. From what I can see, the newly added MatchersDeclaration interface should "fall through" the same way as Assertion does. Only it doesn't.

Adding properties onto the MatchersDeclaration on vitest level does work. So the issue must be in how the consumer extends that type. May also be related to npm linking vitest (although other changes and direct modification of the type works correctly).

Solution

Turns out, types have to be exported in order for this kind of augmentation to work. I've exported MatchersDeclaration from @vitest/expect, then did a proxy export for this type from vitest in the same way we handle the Assertion type.

Augmentation now works:

declare module 'vitest' {
	interface Assertion<T = any> extends CustomMatchers<T> {}
	interface MatchersDeclaration extends CustomMatchers {}
}

We can certainly discuss the interface name. I find MatchersDeclaration to be short and descriptive. MatchersObject means nothing to the end user.

@kettanaito kettanaito force-pushed the fix/custom-raw-matcher-fn branch from 81cab12 to a6e2edc Compare March 11, 2025 14:58
@kettanaito
Copy link
Copy Markdown
Contributor Author

kettanaito commented Mar 11, 2025

The next thing I'd love to do is collocate these two type augmentations into a single user action. You never want to have either of them, always both. So is should be a single augmentation you perform.

Something like

declare module 'vitest' {
  interface Matchers extends CustomMatchers
}

Why not utilize Assertion?

Assertion is a big existing type and it's virtually impossible for vitest to know how it has been augmented.

Instead, it makes more sense to introduce a new interface that would then extend Assertion and MatchersDeclaration under the hood. This top-down augmentation is much more manageable.

This approach is also backward-compatible as the Assertion type stays. For full type-safety, we could recommend augmenting the new Matchers type.

@sheremet-va, what do you think?

@kettanaito kettanaito changed the title fix(@vitest/expect): export a CustomMatchersObject for type-safe expect.extend() fix(@vitest/expect): support type-safe declaration of custom matchers Mar 11, 2025
@kettanaito
Copy link
Copy Markdown
Contributor Author

I will put aside some time to finish this in the upcoming weeks. Thanks for understanding.

@kettanaito
Copy link
Copy Markdown
Contributor Author

Update

Pushed a few changes. The most important one being this:

Now you can extend a single (new) Matcher type to provide both (1) the type of your matcher; (2) the type to infer the matcher implementation for expect.extend():

interface CustomMatchers<R = unknown> {
  toEqualSchema: (schema: Schema) => R
}

declare module 'vitest' {
  interface Matchers extends CustomMatchers {}
}

Such declaration will correctly annotate expect().toEqualSchema(schema) and infer the declaration type as well:

expect.extend({
  toEqualSchema(received, expected) {
    received // any
    expected // Schema
  }
})

All the changes introduced are backward-compatible. You can still extend the Assertion<T> type, and that will still leave expect.extend() without the inferred declaration types as before.

Dropping the T argument

Previously, when extending the Assertion type, you have to declare the T type argument for it to have the same type declaration as in the Vitest packages:

declare module 'vitest' {
  interface Assertion<T = any> extends CustomMatchers<T> {}
}

This type drilling is no longer needed with Matchers. The T type was only provided for the augmented types to match. It never actually annotated anything (T = any negates the type). Thus, removed from the Matchers type.

@kettanaito
Copy link
Copy Markdown
Contributor Author

I've also added some type tests for this. Would appreciate someone from the team to review this and, hopefully, move along with releasing it. Let's bring stellar type-safety to Vitest end-to-end! ✨

@kettanaito
Copy link
Copy Markdown
Contributor Author

It looks like running pnpm build/pnpm dev locally has messed up the lockfile. Does this happen often?

@sheremet-va
Copy link
Copy Markdown
Member

I did not review this yet, but I wanted to say that I am very grateful for this PR! I will check it when I am done with high priority tasks.

The type augmentation is something we needed to look into for some time now.

@kettanaito
Copy link
Copy Markdown
Contributor Author

@sheremet-va 🙏 I am glad to be able to improve Vitest, even if by a little. I would love to teach a better way to declare type-safe custom matchers, and this change is exactly what's missing. Would be grateful for your review whenever you have a moment. Thank you.

@kettanaito
Copy link
Copy Markdown
Contributor Author

kettanaito commented May 12, 2025

@sheremet-va, linting fails on some arbitrary license check:

0s
Run git diff --exit-code
diff --git a/packages/vitest/LICENSE.md b/packages/vitest/LICENSE.md
index 5c433bf..43[1](https://github.com/vitest-dev/vitest/actions/runs/14535368277/job/41232427629?pr=7656#step:6:1)4519 100644
--- a/packages/vitest/LICENSE.md
+++ b/packages/vitest/LICENSE.md
@@ -681,6 +681,35 @@ Repository: terkelg/prompts
 
 ---------------------------------------
 
+## quansync
+License: MIT
+By: Anthony Fu, 三咲智子 Kevin Deng
+Repository: git+https://github.com/quansync-dev/quansync.git
+
+> MIT License
+>
+> Copyright (c) 2025-PRESENT Anthony Fu <https://github.com/antfu> and Kevin Deng <https://github.com/sxzz>
+>
+> Permission is hereby granted, free of charge, to any person obtaining a copy
+> of this software and associated documentation files (the "Software"), to deal
+> in the Software without restriction, including without limitation the rights
+> to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+> copies of the Software, and to permit persons to whom the Software is
+> furnished to do so, subject to the following conditions:
+>
+> The above copyright notice and this permission notice shall be included in all
+> copies or substantial portions of the Software.
+>
+> THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+> IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+> FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+> AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+> LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+> OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+> SOFTWARE.
+
+---------------------------------------
+
 ## resolve-pkg-maps
 License: MIT
 By: Hiroki Osame

I don't believe my changes have anything to do with this. I wonder how isn't this failing on main?

I will rebase against main to see if that was already fixed there.

@sheremet-va
Copy link
Copy Markdown
Member

@sheremet-va, linting fails on some arbitrary license check:

I don't believe my changes have anything to do with this. I wonder how isn't this failing on main?

I will rebase against main to see if that was already fixed there.

It fails because you changed the license file when it shouldn't be changed

@kettanaito
Copy link
Copy Markdown
Contributor Author

@sheremet-va, that is extremely weird. I've never edited that file. I suggest you look into your automation, perhaps something auto-edits it in some cases. Will revert.

@kettanaito
Copy link
Copy Markdown
Contributor Author

kettanaito commented May 12, 2025

Got a regression in typechecking:

Error: test/core/test/jest-expect.test.ts(263,19): error TS2339: Property 'toBeDividedBy' does not exist on type 'ExpectStatic'.
Error: test/core/test/jest-expect.test.ts(264,23): error TS2339: Property 'toBeDividedBy' does not exist on type 'AsymmetricMatchersContaining'.

Looks like some of my latest changes broke the custom matcher forwarding if extending the Assertion type. Will look into it.

Edit: The error was caused because I was extending AsymmetricMatchersContaining in the wrong package (vitest instead of @vitest/expect).

@sheremet-va
Copy link
Copy Markdown
Member

@sheremet-va, that is extremely weird. I've never edited that file. I suggest you look into your automation, perhaps something auto-edits it in some cases. Will revert.

You probably ran build command at one point without the latest dependencies. This file is generated automatically

@kettanaito
Copy link
Copy Markdown
Contributor Author

Yeah, I think you're right! Still my bad I let it slip through into the commit. I will be more careful with the future changes.

Anyhow, I think the implementation is done on my part. I'm waiting for the tests to pass to confirm everything's okay. If you could please give this another review I would be grateful!

@kettanaito kettanaito requested a review from sheremet-va May 12, 2025 14:03
// Allow unused `T` to preserve its name for extensions.
// Type parameter names must be identical when extending those types.
// eslint-disable-next-line
interface Matchers<T> {}
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The changes in this file seem excessive, we already expose Matchers in @vitest/expect, we just need to also extend Assertion there. Is there a reason why you extend ExpectStatic, but don't extend Assertion in the expect package?

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

See

export interface Assertion<T = any>

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Without this, it seems like you can't use Matchers type only with the @vitest/expect package - the vitest is required

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is a great point. I put Matchers in vitest because I wasn't well familiar with the packages relationship back then. I followed your advice and moved the extension to @vitest/expect so that now these changes extend ExpectStatic and Assertion from @vitest/expect and do not touch the vitest changes at all. Type tests confirm that is enough. Awesome!

@kettanaito kettanaito requested a review from sheremet-va May 13, 2025 09:18
sheremet-va
sheremet-va previously approved these changes May 13, 2025
Copy link
Copy Markdown
Member

@sheremet-va sheremet-va left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM! Thank you 🙏

@kettanaito
Copy link
Copy Markdown
Contributor Author

Thank you so much for reviewing this! Is there anything I can do to help with getting this released this week?

@sheremet-va
Copy link
Copy Markdown
Member

Thank you so much for reviewing this! Is there anything I can do to help with getting this released this week?

You can help by writing a small documentation for this feature 😄

This PR will be part of 3.2, which is scheduled for May 19th. We can probably include this in the next beta release though.

@kettanaito
Copy link
Copy Markdown
Contributor Author

I propose we replace the current suggestions on extending the matchers with this. Would that work? If the docs are versioned, devs can reference the old solution (which still works) but new devs will interface with the new solution, which is shorter and fixes a few issues.

Yeah, even having this in beta is quite enough to unblock me so that sounds good!

@sheremet-va
Copy link
Copy Markdown
Member

I propose we replace the current suggestions on extending the matchers with this. Would that work? If the docs are versioned, devs can reference the old solution (which still works) but new devs will interface with the new solution, which is shorter and fixes a few issues.

Yeah, even having this in beta is quite enough to unblock me so that sounds good!

Docs are versioned only by major. 3.1 won't have this feature, so we need to specify that it is only available since 3.2.

@sheremet-va sheremet-va added the p2-nice-to-have Not breaking anything but nice to have (priority) label May 15, 2025
@sheremet-va sheremet-va added this to the 3.2.0 milestone May 15, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

cr-tracked p2-nice-to-have Not breaking anything but nice to have (priority)

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants