Conversation
RFC-lifecycle.md
Outdated
| 4. Add `nock.assertAll()` which does the equivalent of | ||
| `scopes.forEach(scope => scope.done())`. This probably should be invoked | ||
| in the test itself. | ||
| 5. Rename `scope.done()` to `scope.assert()` to harmonize with `assertAll()` |
There was a problem hiding this comment.
I’m not sure about the naming of scope.assert(), it reads as if I should pass in something. Maybe scope.assertDone() and scope.assertAllDone()?
There was a problem hiding this comment.
Hmm yea it does seem like it wants an argument. nock.assertAllDone() and scope.assertDone() could work. It's a little long but maybe that's fine.
For someone encountering the test code for the first time, it's definitely clearer than done().
There was a problem hiding this comment.
I’m a fan of clear method names, even if they are long. Code readability is king 👑
There was a problem hiding this comment.
Are there any other mock libraries with interfaces you like? I'm curious to compare. I looked at Sinon, but it doesn't really have assertion methods like this.
"Done" isn't the clearest either. Really what we want to say is "expectations fulfilled" or "requests made" or "mocks consumed." I'm 100% agreed that clear long names are better than short names, though I don't want the names to be longer than necessary.
There was a problem hiding this comment.
Another option is nock.verifyAll() / scope.verify().
I don’t think it has the same problem as assert of wanting an argument.
Or: nock.verifyAllRequests() / scope.verifyRequest()
There was a problem hiding this comment.
I changed nock.assertAll() to nock.assertAllMocksUsed() and scope.assert() to scope.assertMocksUsed().
There was a problem hiding this comment.
now seeing that, I wonder why one of the method is called assertAllMocksUsed and the other assertMocksUsed. Wouldn’t it be more consistent to call both assertMocksUsed and the "scope" of the method call would be clear based on wether you call it on a scope instance or the global nock object?
There was a problem hiding this comment.
It’s a good question. If the names are the same I think it’s too likely someone will call the wrong one. Since scope.done() was the way to do this for a long time, I don’t want to provide two different ways which are as confusable as scope.assertMocksUsed() and nock.assertMocksUsed(). Especially if those two functions are doing different things.
There was a problem hiding this comment.
Another option: scope.assertUsed(), nock.assertAllMocksUsed().
There was a problem hiding this comment.
Especially if those two functions are doing different things
Wouldn’t both functions do the same thing? nock.assertMocksUsed() would check globally, scope.assertMocksUsed() would check for only that scope?
|
For the reset case, I wrote:
To get back to the initial state, it also currently needs:
|
|
#1465 adds an |
|
I rewrote most of the text of this proposal, updated method names based on this discussion, and proposed a consolidated API for controlling nock's handling of unmocked requests. It's clear there is a lot of stuff to cover here! 👀 To make this easier to digest, I've split it up into four sub-proposals, each dealing with one of the main use cases:
So far we've talked the most about (3) so maybe we could try to approve that one first? |
This adds a global `afterEach` hook that resets nock global state to the condition it's in when it's initially `require`d. It's worth noting that there isn't a single function for doing this; I've got four function calls here to accomplish that. It's worth taking a moment here to validate that this is indeed the correct sequence of calls. See #1441 for some discussion on that. It removes most or all of the one-off global-state cleanup code that's sprinkled among the tests. - Tests may now assume that, at the time they're invoked, that nock's initial state will match the state it has when it has just been `require`d. - Tests are not responsible for restoring that state afterward. - If tests need nock in a _different_ starting state, it's up to them to set it. - If tests create _other_ state, such as starting a server, it's up to them to clean it up. I've included some miscellaneous adjacent style cleanup. I've deferred the various `test_back*` suites because they are all intertwingled, and require a bit more focused effort to unravel.
gr2m
left a comment
There was a problem hiding this comment.
in related news: I love how Node asks "Does anyone object" vs "Does anyone agree". I think we should follow their lead: https://twitter.com/jasnell/status/1100082440054304770
| #### Discussion | ||
|
|
||
| `nock.reset()` is suitable for running from `afterEach()` or `finally()` to | ||
| prepare for the next test. |
There was a problem hiding this comment.
I’d say let’s do it.
I wonder if it would be easier to move this out into its own RFC?
| #### Discussion | ||
|
|
||
| The new name harmonizes with `nock.activate()` which is being kept: these methods are | ||
| inverses of each other. |
There was a problem hiding this comment.
Awesome, let’s do it! No objections 👍
|
|
||
| Some developers may prefer to call it from an `afterEach()` hook to avoid | ||
| boilerplate. Since not all tests in a suite may use a mock, it should do the | ||
| natural thing when no mocks have been created, which is no-op. |
There was a problem hiding this comment.
I feel either way could cause confusion. Having just one method name would make it easier to remember. But in the end I think it’s better than what we have right now and if you feel strong about nock.assertAllMocksUsed() being better than nock.assertMocksUsed() than let’s go ahead with it :)
If we have two different methods anyway and want them to look different, we could also do scope.assertScopeMocksUsed(). It's repetitive, but clear.
There was a problem hiding this comment.
Agreed that assertMocksUsed() is better than assertAllMocksUsed(). 👍
Right, my concern is offering two identically named methods which act on different scopes. Most developers won't digest the whole API, they'll just absorb the bits they need. If they know the method is assertMocksUsed(), they could get unexpected results if they don't realize that calling it on scope will have a more limited effect than calling it on nock.
Having reflected on this a while, I'm okay with naming them both assertMocksUsed(). However I prefer your suggestion of nock.assertMocksUsed() for the global one and scope.assertScopeMocksUsed() for the limited one:
- It encourages people to use the global
nock.assertMocksUsed()where possible. - It makes the limited scope of
scope.assertScopeMocksUsed()100% clear. - It helps clarify that the intent of
scope.assertScopeMocksUsed()is for cases which can't be handled using the global method.
There was a problem hiding this comment.
If we do two different method names, then I’d prefer nock.assertAllMocksUsed over nock.assertMocksUsed. The latter might suggest I can pass an argument
| 2. Optionally pass an argument `nock.simulateUnreachability({ allowedHosts })` | ||
| to specify exceptions, i.e. hosts which should still be reachable. | ||
| 3. Add `nock.forbidUnmockedRequests()` which causes any unmocked request | ||
| to be considered a programmer error. |
There was a problem hiding this comment.
Maybe we can call it .throwOnUnmockedRequests() might be more clear of what the method does?
On a big picture perspective, I feel like things like this should be an option when instantiating nock, which we currently don’t do, but I really think we should. We can discuss that in another RFC :D
| #### Proposed solution | ||
|
|
||
| 1. Rename `nock.disableNetConnect()` to `nock.simulateUnreachability()`. | ||
| 2. Optionally pass an argument `nock.simulateUnreachability({ allowedHosts })` |
There was a problem hiding this comment.
maybe we could add two options:
- hosts: for a block list
- excludeHost: for an unblock list
Also I wonder if we could come up with a simpler alternative to Unreachability, it’s quite hard to spell :D
| #### Discussion | ||
|
|
||
| `nock.simulateUnreachability()` emits an unreachable-like error through normal | ||
| HTTP error channels. |
There was a problem hiding this comment.
I wonder whether this should be available on a scope? So for example
nock('https://google.com')
.get('/')
.simulateConnectionError();Which would allow for some more granular control over which requests produce a network error and which don’t
There was a problem hiding this comment.
That's an interesting idea, though there would also need to be a way to configure unreachability for unmocked requests.
I think .replyWithError() could be used for this purpose today, and would want to think about whether it's worth adding a second way to do it.
There was a problem hiding this comment.
Yeah, I think it’s totally reasonable to also want to turn off all network connections as well. But yeah, I think it’s nice to be able to test how your application responds to one API being down/unavailable without it being blanketly based on the host.
The use case here is that I have two requests that go through an API gateway, in my case api.ft.com, behind that are different services which potentially could be individually unavailable. So api.ft.com/foo is available but api.ft.com/bar uses a different surface under the hood which is down.
Currently (as far as I know) I could only simulate all of api.ft.com being unavailable rather than just api.ft.com/bar.
There was a problem hiding this comment.
Makes sense. Although, doesn't .replyWithError() solve that problem?
There was a problem hiding this comment.
Yeah, I didn’t actually know that existed until you’d just mentioned it.
It does, however it might be nice to be able to not have to provide a message and have it just fail in the network and let whatever library (eg node-fetch) build the correct network error message, that it would if it was real, if that makes sense?
There was a problem hiding this comment.
Right, makes sense. It's syntactic sugar, though it keeps the dev from needing to know what kind of error is thrown when a server is unreachable.
| Okay, so I'm not thrilled with these proposals. While these use cases are | ||
| important and need to be distinguished, having similar behavior triggered by | ||
| two different functions is confusing. Let me try to propose an alternative | ||
| with a combined API. |
There was a problem hiding this comment.
Agreed. I feel we can come up with a better way here
| By putting a wildcard match in the `fail` list, you know your tests will | ||
| immediately choke if they try to hit live servers. Since the client library | ||
| and application are kept out of the loop, this _provides confidence that a | ||
| mock is configured for every request made by the application and tests_. |
There was a problem hiding this comment.
whenUnmocked feels like it should be an event handler somehow ... 🤔 I’m not sure about this one, I feel there is a better way. Sorry I know that’s not helpful but all I got right now 🤷♂️
|
This is awesome feedback. It's going to take me a minute to digest it and respond. Thanks for sharing the link about Node governance! "Does anyone object" is not how I'm used to working though I agree it makes sense, and could be really good for nock. In particular (from that thread):
|
|
Should we check the accepted RFCs into the repository? The artifact of that discussion could be useful to link to from the release notes for users who are upgrading. I propose we do that. Since this discussion and scope have gotten large, I could pull all the intro material + analysis into one PR (maybe |
|
That’s be great! |
This adds a global `afterEach` hook that resets nock global state to the condition it's in when it's initially `require`d. It's worth noting that there isn't a single function for doing this; I've got four function calls here to accomplish that. It's worth taking a moment here to validate that this is indeed the correct sequence of calls. See #1441 for some discussion on that. It removes most or all of the one-off global-state cleanup code that's sprinkled among the tests. - Tests may now assume that, at the time they're invoked, that nock's initial state will match the state it has when it has just been `require`d. - Tests are not responsible for restoring that state afterward. - If tests need nock in a _different_ starting state, it's up to them to set it. - If tests create _other_ state, such as starting a server, it's up to them to clean it up. I've included some miscellaneous adjacent style cleanup. I've deferred the various `test_back*` suites because they are all intertwingled, and require a bit more focused effort to unravel.
This adds a global `afterEach` hook that resets nock global state to the condition it's in when it's initially `require`d. It's worth noting that there isn't a single function for doing this; I've got four function calls here to accomplish that. It's worth taking a moment here to validate that this is indeed the correct sequence of calls. See #1441 for some discussion on that. It removes most or all of the one-off global-state cleanup code that's sprinkled among the tests. - Tests may now assume that, at the time they're invoked, that nock's initial state will match the state it has when it has just been `require`d. - Tests are not responsible for restoring that state afterward. - If tests need nock in a _different_ starting state, it's up to them to set it. - If tests create _other_ state, such as starting a server, it's up to them to clean it up. I've included some miscellaneous adjacent style cleanup. I've deferred the various `test_back*` suites because they are all intertwingled, and require a bit more focused effort to unravel.
This adds a global `afterEach` hook that resets nock global state to the condition it's in when it's initially `require`d. It's worth noting that there isn't a single function for doing this; I've got four function calls here to accomplish that. It's worth taking a moment here to validate that this is indeed the correct sequence of calls. See #1441 for some discussion on that. It removes most or all of the one-off global-state cleanup code that's sprinkled among the tests. - Tests may now assume that, at the time they're invoked, that nock's initial state will match the state it has when it has just been `require`d. - Tests are not responsible for restoring that state afterward. - If tests need nock in a _different_ starting state, it's up to them to set it. - If tests create _other_ state, such as starting a server, it's up to them to clean it up. I've included some miscellaneous adjacent style cleanup. I've deferred the various `test_back*` suites because they are all intertwingled, and require a bit more focused effort to unravel.
This adds a global `afterEach` hook that resets nock global state to the condition it's in when it's initially `require`d. It's worth noting that there isn't a single function for doing this; I've got four function calls here to accomplish that. It's worth taking a moment here to validate that this is indeed the correct sequence of calls. See nock#1441 for some discussion on that. It removes most or all of the one-off global-state cleanup code that's sprinkled among the tests. - Tests may now assume that, at the time they're invoked, that nock's initial state will match the state it has when it has just been `require`d. - Tests are not responsible for restoring that state afterward. - If tests need nock in a _different_ starting state, it's up to them to set it. - If tests create _other_ state, such as starting a server, it's up to them to clean it up. I've included some miscellaneous adjacent style cleanup. I've deferred the various `test_back*` suites because they are all intertwingled, and require a bit more focused effort to unravel.
Thought I would open a proposal for this in the style of #1429 / #1413.
This aims to improve replace Nock's lifecycle methods with new ones which better fit real-life use cases from my experience with the library. It gives them clearer, less ambiguous names. I include a solution to #884 which I've reopened.
These use cases are the ones that make sense to me, though I'm curious if other folks have different use cases which should be considered as "first class." Though I'm not really removing anything in the API, so if folks have an unusual use case where they need to do things a different way, that's still possible.
I'd like to ship these in 11.x, though we could consider backporting some of them to 10.x (minus the deprecation warnings).
These encourage changes to the installed test code base, which is something I don't take lightly. Before we release it, I'd like to validate the new interface with beta users, and be really sure that it makes their tests clearer and seems worth their effort to update.