Skip to content

review unit tests and make config setup more explicit#6972

Merged
shiftkey merged 6 commits intodevelopmentfrom
make-unit-tests-more-robust
Mar 15, 2019
Merged

review unit tests and make config setup more explicit#6972
shiftkey merged 6 commits intodevelopmentfrom
make-unit-tests-more-robust

Conversation

@shiftkey
Copy link
Member

@shiftkey shiftkey commented Feb 27, 2019

Overview

Closes #6924

Description

$ git config --global pull.rebase
true
$ yarn test:unit
yarn run v1.13.0
$ cross-env ELECTRON_RUN_AS_NODE=1 ./node_modules/.bin/electron ./node_modules/jest/bin/jest --detectOpenHandles --silent --config ./app/jest.unit.config.js
...

Test Suites: 85 passed, 85 total
Tests:       3 skipped, 724 passed, 727 total
Snapshots:   0 total
Time:        14.635s
✨  Done in 15.57s.

And the opposite:

$ git config --global pull.rebase
false
$ yarn test:unit                                                                                                                                            
yarn run v1.13.0
$ cross-env ELECTRON_RUN_AS_NODE=1 ./node_modules/.bin/electron ./node_modules/jest/bin/jest --detectOpenHandles --silent --config ./app/jest.unit.config.js
...

Test Suites: 85 passed, 85 total
Tests:       3 skipped, 724 passed, 727 total
Snapshots:   0 total
Time:        14.865s
✨  Done in 15.64s.

Release notes

Notes: no-notes

@shiftkey shiftkey added tech-debt Issues and pull requests related to addressing technical debt or improving the codebase ready-for-review Pull Requests that are ready to be reviewed by the maintainers labels Feb 27, 2019
Copy link
Contributor

@outofambit outofambit left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@shiftkey do you think it's worth taking the time to build some sort of default config to be used in our testing environment instead of being vulnerable to the underlying system's settings? it seems like this would be something we'll run into for other features and test cases too.

@shiftkey
Copy link
Member Author

do you think it's worth taking the time to build some sort of default config to be used in our testing environment instead of being vulnerable to the underlying system's settings?

I gather you're thinking of something like this?

describe('global config', () => {
const HOME = mkdirSync('global-config-here')
const env = { HOME }
const expectedConfigPath = Path.normalize(Path.join(HOME, '.gitconfig'))
const baseArgs = ['config', '-f', expectedConfigPath]
describe('getGlobalConfigPath', () => {
beforeEach(async () => {
// getGlobalConfigPath requires at least one entry, so the
// test needs to setup an existing config value
await GitProcess.exec([...baseArgs, 'user.name', 'bar'], __dirname)
})
it('gets the config path', async () => {
const path = await getGlobalConfigPath(env)
expect(path).toBe(expectedConfigPath)
})
})

I worry that using a pre-canned config with tests would prevent us from catching issues with config set at the system level (set by the Git distribution, and can differ by OS) or global level by users that Desktop should handle.

I think in this case what I should have done was ensure the local config for a repository is set correctly for what is being tested (which overrides whatever may be set globally). That's the approach I continue to recommend, particularly with complex areas with plenty of config options like git pull.

These tests were there to help me wrap my head around the current pull behaviour based on expected config values as I dove into "pull with rebase" and seeing them fail now was a good chance to revisit them and question what value they actually had to the feature. I worry that abstracting away the global config away will not give us a chance to identify and re-evaluate our tests as things evolve.

We also don't have an easy way to inject the config for each test (getGlobalConfigValue was important to get under test otherwise we'd be overwriting the user's config), and I worry this might add complexity to the overall Git API surface.

@outofambit
Copy link
Contributor

@shiftkey I'm not sure how the code snippet above relates to the question?

I understand your concerns but I think ideally we would enumerate our conditions and assumptions in our test harnesses instead of relying on variations in underlying system variance to catch bugs. (This can lead to difficult to reproduce test failures that don't provide much data to us and can be frustrating for our collaborators.)

For this PR, I'd prefer a fixture-ish approach that uses helper functions for setting up certain config conditions. Perhaps returning an entire config is a bit too much to take on now, but it would be nice to start collecting our different configs and modifiers for our testing into a single place.

Perhaps something like a setPullRebaseConfig(repositoryPath, boolean) or even PullRebaseButNotFF(repositoryPath) in a helper file?

@shiftkey
Copy link
Member Author

shiftkey commented Mar 4, 2019

I'm not sure how the code snippet above relates to the question?

This is the ceremony I had in mind when you were asking about a "testing environment instead of being vulnerable to the underlying system's settings" - creating a new global config on disk, assigning this to the HOME environment variable, and then passing that through to the Git command to represent the "global" configuration (each API under test would need to support this).

That sounds different to this:

For this PR, I'd prefer a fixture-ish approach that uses helper functions for setting up certain config conditions.

This is interesting to me from the perspective of making setup more clear:

diff --git a/app/test/helpers/local-config.ts b/app/test/helpers/local-config.ts
new file mode 100644
index 000000000..0283d80f9
--- /dev/null
+++ b/app/test/helpers/local-config.ts
@@ -0,0 +1,11 @@
+import { Repository } from '../../src/models/repository'
+import { GitProcess } from 'dugite'
+
+export async function setupLocalConfig(
+  repository: Repository,
+  localConfig: Iterable<[string, string]>
+) {
+  for (const [key, value] of localConfig) {
+    await GitProcess.exec(['config', key, value], repository.path)
+  }
+}
diff --git a/app/test/unit/git/pull/ahead-and-behind-test.ts b/app/test/unit/git/pull/ahead-and-behind-test.ts
index 6ecff4bb4..356e61c8a 100644
--- a/app/test/unit/git/pull/ahead-and-behind-test.ts
+++ b/app/test/unit/git/pull/ahead-and-behind-test.ts
@@ -13,6 +13,7 @@ import {
 } from '../../../helpers/repository-scaffolding'
 import { getTipOrError, getRefOrError } from '../../../helpers/tip'
 import { GitProcess } from 'dugite'
+import { setupLocalConfig } from '../../../helpers/local-config'
 
 const featureBranch = 'this-is-a-feature'
 const origin = 'origin'
@@ -59,14 +60,10 @@ describe('git/pull', () => {
       let newTip: Commit
 
       beforeEach(async () => {
-        await GitProcess.exec(
-          ['config', '--local', 'pull.rebase', 'false'],
-          repository.path
-        )
-        await GitProcess.exec(
-          ['config', '--local', 'pull.ff', 'false'],
-          repository.path
-        )
+        await setupLocalConfig(repository, [
+          ['pull.rebase', 'false'],
+          ['pull.ff', 'false'],
+        ])
 
         previousTip = await getTipOrError(repository)

It doesn't quite address the presence of the global config problem - we'd still need to pass in the new HOME environment variable for each Git operation under test.

Would an API like ☝️ help unblock this PR?

@outofambit
Copy link
Contributor

This is the ceremony I had in mind

@shiftkey ah! thanks for clarifying. yeah i'm thinking we can override global, system-level configs with local configs in the repositories we use in tests.

Would an API like ☝️ help unblock this PR?

👍 👍 👍 yes! this looks great.

@shiftkey
Copy link
Member Author

shiftkey commented Mar 4, 2019

@outofambit ok cool, I'll add that work into this PR and see how it changes things. There's some places where we change the config on-the-fly that I'll call out as being different to this sort of "setup" case we're discussing here.

@shiftkey shiftkey removed the ready-for-review Pull Requests that are ready to be reviewed by the maintainers label Mar 4, 2019
@shiftkey shiftkey changed the title ensure unit tests work when a global pull.rebase is set review unit tests and make config setup more explicit Mar 4, 2019
@shiftkey shiftkey force-pushed the make-unit-tests-more-robust branch from ac7b206 to 33d1d02 Compare March 4, 2019 17:05
)

// change config on-the-fly to trigger the line endings change warning
await GitProcess.exec(['config', 'core.autocrlf', 'true'], repo.path)
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is the only place I've left untouched by this PR, but I've added a better comment here to explain why it's necessary to do this mid-test rather than as part of the repository setupo.

// ensure the default config is to try and show signatures
// this should be overriden by the `getCommits` function as it may not
// have a valid GPG agent configured
await setupLocalConfig(repository, [['log.showSignature', 'true']])
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The comment attached to this setup wasn't even remotely accurate to what was being tested, so I updated that to better explain this test case.

@shiftkey shiftkey added the ready-for-review Pull Requests that are ready to be reviewed by the maintainers label Mar 4, 2019
@shiftkey shiftkey requested a review from outofambit March 4, 2019 19:53
@shiftkey shiftkey force-pushed the make-unit-tests-more-robust branch from 2564283 to 4f43f0d Compare March 7, 2019 15:04
@iAmWillShepherd
Copy link
Contributor

Since this is tech-debt and @outofambit was involved in guiding the direction of this PR, I'm going to let it hang around until she gets a chance to take a look.

@iAmWillShepherd iAmWillShepherd self-assigned this Mar 8, 2019
@shiftkey shiftkey merged commit cfec99b into development Mar 15, 2019
@shiftkey shiftkey deleted the make-unit-tests-more-robust branch March 26, 2019 21:17
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

ready-for-review Pull Requests that are ready to be reviewed by the maintainers tech-debt Issues and pull requests related to addressing technical debt or improving the codebase

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants