Skip to content
This repository was archived by the owner on Sep 30, 2025. It is now read-only.

Attempt to debug drifting fixtures#69

Merged
pbrisbin merged 6 commits intomasterfrom
pb/fixtures-debug
Aug 27, 2019
Merged

Attempt to debug drifting fixtures#69
pbrisbin merged 6 commits intomasterfrom
pb/fixtures-debug

Conversation

@pbrisbin
Copy link
Owner

In #67, we found an out-of-date-mock was leading to a diff in the expected
output for the upgrade test. Updating the mock this trivial, but why did it
drift?

This PR makes some improvements to help debug this:

  • Prevent implicit fixtures-recording on CI

    This will ensure that any change that introduces drift (assuming it goes via
    PR with CI) will be identified then and there. I assume the immediate feedback
    will make it simple to understand why and take appropriate action.

  • Capture /arguments when fixtures are written

    When we do encounter the drift error, we can now see which calls led to which
    existing fixtures. Perhaps this will be useful for identifying cause.

  • Remove stale fixtures

    Any fixtures without /arguments after the last complete run are no longer used
    by the test suite. They are just noise and so have been removed.

Unfortunately, this will only help the next case of this, since we're not able
to capture arguments from the stale fixtures to debug, nor were we enforcing no
implicit fixture updates on CI before this PR.

Therefore, I have to give up and actually just commit the original update,
without any explanation for it.

This way, we can attempt to find which fixture is out of date if/when we
bump into a FIXTURES_NO_RECORD error.
I can't explain why this is needed.
@pbrisbin pbrisbin merged commit de3b4f3 into master Aug 27, 2019
@pbrisbin pbrisbin deleted the pb/fixtures-debug branch August 27, 2019 13:49
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant