Skip to content

Commit 09b1b2c

Browse files
jbudzelasticmachine
jbudz
and
elasticmachine
committed
Refactor test entry by runner (#44679)
Over the last few years we've increased the number of test runners. Entry points by test type have become too lage of a category for unique names, so this moves top level test scripts under yarn and replaces test types with the runner name. e.g. `yarn test:browser` -> `yarn test:karma` Closes #41133 Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>
1 parent d7a0074 commit 09b1b2c

30 files changed

Lines changed: 88 additions & 136 deletions

File tree

‎CONTRIBUTING.md‎

Lines changed: 13 additions & 31 deletions
Original file line numberDiff line numberDiff line change
@@ -458,11 +458,11 @@ The following table outlines possible test file locations and how to invoke them
458458

459459
| Test runner | Test location | Runner command (working directory is kibana root) |
460460
| ----------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------- | --------------------------------------------------------------------------------------- |
461-
| Jest | `src/**/*.test.js`<br>`src/**/*.test.ts` | `node scripts/jest -t regexp [test path]` |
462-
| Jest (integration) | `**/integration_tests/**/*.test.js` | `node scripts/jest_integration -t regexp [test path]` |
461+
| Jest | `src/**/*.test.js`<br>`src/**/*.test.ts` | `yarn test:jest -t regexp [test path]` |
462+
| Jest (integration) | `**/integration_tests/**/*.test.js` | `yarn test:jest_integration -t regexp [test path]` |
463463
| Mocha | `src/**/__tests__/**/*.js`<br>`!src/**/public/__tests__/*.js`<br>`packages/kbn-datemath/test/**/*.js`<br>`packages/kbn-dev-utils/src/**/__tests__/**/*.js`<br>`tasks/**/__tests__/**/*.js` | `node scripts/mocha --grep=regexp [test path]` |
464-
| Functional | `test/*integration/**/config.js`<br>`test/*functional/**/config.js`<br>`test/accessibility/config.js` | `node scripts/functional_tests_server --config test/[directory]/config.js`<br>`node scripts/functional_test_runner --config test/[directory]/config.js --grep=regexp` |
465-
| Karma | `src/**/public/__tests__/*.js` | `npm run test:dev` |
464+
| Functional | `test/*integration/**/config.js`<br>`test/*functional/**/config.js`<br>`test/accessibility/config.js` | `yarn test:ftr:server --config test/[directory]/config.js`<br>`yarn test:ftr:runner --config test/[directory]/config.js --grep=regexp` |
465+
| Karma | `src/**/public/__tests__/*.js` | `yarn test:karma:debug` |
466466

467467
For X-Pack tests located in `x-pack/` see [X-Pack Testing](x-pack/README.md#testing)
468468

@@ -473,56 +473,38 @@ Test runner arguments:
473473
Examples:
474474
- Run the entire elasticsearch_service test suite:
475475
```
476-
node scripts/jest src/core/server/elasticsearch/elasticsearch_service.test.ts
476+
yarn test:jest src/core/server/elasticsearch/elasticsearch_service.test.ts
477477
```
478478
- Run the jest test case whose description matches `stops both admin and data clients`:
479479
```
480-
node scripts/jest -t 'stops both admin and data clients' src/core/server/elasticsearch/elasticsearch_service.test.ts
480+
yarn test:jest -t 'stops both admin and data clients' src/core/server/elasticsearch/elasticsearch_service.test.ts
481481
```
482482
- Run the api integration test case whose description matches the given string:
483483
```
484-
node scripts/functional_tests_server --config test/api_integration/config.js
485-
node scripts/functional_test_runner --config test/api_integration/config.js --grep='should return 404 if id does not match any sample data sets'
484+
yarn test:ftr:server --config test/api_integration/config.js
485+
yarn test:ftr:runner --config test/api_integration/config.js --grep='should return 404 if id does not match any sample data sets'
486486
```
487487
488488
### Debugging Unit Tests
489489
490490
The standard `yarn test` task runs several sub tasks and can take several minutes to complete, making debugging failures pretty painful. In order to ease the pain specialized tasks provide alternate methods for running the tests.
491491
492-
To execute both server and browser tests, but skip linting, use `yarn test:quick`.
493-
494-
```bash
495-
yarn test:quick
496-
```
497-
498-
Use `yarn test:mocha` when you want to run the mocha tests.
499-
500-
```bash
501-
yarn test:mocha
502-
```
503-
504-
When you'd like to execute individual server-side test files, you can use the command below. Note that this command takes care of configuring Mocha with Babel compilation for you, and you'll be better off avoiding a globally installed `mocha` package. This command is great for development and for quickly identifying bugs.
505-
506-
```bash
507-
node scripts/mocha <file>
508-
```
509-
510492
You could also add the `--debug` option so that `node` is run using the `--debug-brk` flag. You'll need to connect a remote debugger such as [`node-inspector`](https://github.com/node-inspector/node-inspector) to proceed in this mode.
511493
512494
```bash
513495
node scripts/mocha --debug <file>
514496
```
515497

516-
With `yarn test:browser`, you can run only the browser tests. Coverage reports are available for browser tests by running `yarn test:coverage`. You can find the results under the `coverage/` directory that will be created upon completion.
498+
With `yarn test:karma`, you can run only the browser tests. Coverage reports are available for browser tests by running `yarn test:coverage`. You can find the results under the `coverage/` directory that will be created upon completion.
517499

518500
```bash
519-
yarn test:browser
501+
yarn test:karma
520502
```
521503

522-
Using `yarn test:dev` initializes an environment for debugging the browser tests. Includes an dedicated instance of the kibana server for building the test bundle, and a karma server. When running this task the build is optimized for the first time and then a karma-owned instance of the browser is opened. Click the "debug" button to open a new tab that executes the unit tests.
504+
Using `yarn test:karma:debug` initializes an environment for debugging the browser tests. Includes an dedicated instance of the kibana server for building the test bundle, and a karma server. When running this task the build is optimized for the first time and then a karma-owned instance of the browser is opened. Click the "debug" button to open a new tab that executes the unit tests.
523505

524506
```bash
525-
yarn test:dev
507+
yarn test:karma:debug
526508
```
527509

528510
In the screenshot below, you'll notice the URL is `localhost:9876/debug.html`. You can append a `grep` query parameter to this URL and set it to a string value which will be used to exclude tests which don't match. For example, if you changed the URL to `localhost:9876/debug.html?query=my test` and then refreshed the browser, you'd only see tests run which contain "my test" in the test description.
@@ -538,7 +520,7 @@ To run the tests for just your particular plugin run the following command from
538520

539521
```bash
540522
yarn test:mocha
541-
yarn test:browser --dev # remove the --dev flag to run them once and close
523+
yarn test:karma:debug # remove the debug flag to run them once and close
542524
```
543525

544526
### Automated Accessibility Testing

‎package.json‎

Lines changed: 5 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -40,23 +40,21 @@
4040
"kbn": "node scripts/kbn",
4141
"es": "node scripts/es",
4242
"test": "grunt test",
43-
"test:dev": "grunt test:dev",
44-
"test:quick": "grunt test:quick",
45-
"test:browser": "grunt test:browser",
43+
"test:karma": "grunt test:karma",
44+
"test:karma:debug": "grunt test:karmaDebug",
4645
"test:jest": "node scripts/jest",
4746
"test:mocha": "node scripts/mocha",
4847
"test:mocha:coverage": "grunt test:mochaCoverage",
49-
"test:ui": "node scripts/functional_tests",
50-
"test:ui:server": "node scripts/functional_tests_server",
51-
"test:ui:runner": "node scripts/functional_test_runner",
48+
"test:ftr": "node scripts/functional_tests",
49+
"test:ftr:server": "node scripts/functional_tests_server",
50+
"test:ftr:runner": "node scripts/functional_test_runner",
5251
"test:coverage": "grunt test:coverage",
5352
"typespec": "typings-tester --config x-pack/legacy/plugins/canvas/public/lib/aeroelastic/tsconfig.json x-pack/legacy/plugins/canvas/public/lib/aeroelastic/__fixtures__/typescript/typespec_tests.ts",
5453
"checkLicenses": "node scripts/check_licenses --dev",
5554
"build": "node scripts/build --all-platforms",
5655
"start": "node --trace-warnings --throw-deprecation scripts/kibana --dev",
5756
"debug": "node --nolazy --inspect scripts/kibana --dev",
5857
"debug-break": "node --nolazy --inspect-brk scripts/kibana --dev",
59-
"karma": "karma start",
6058
"lint": "yarn run lint:es && yarn run lint:sass",
6159
"lint:es": "node scripts/eslint",
6260
"lint:sass": "node scripts/sasslint",

‎packages/kbn-plugin-generator/README.md‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -71,7 +71,7 @@ Generated plugins receive a handful of scripts that can be used during developme
7171
7272
Build a distributable archive of your plugin.
7373
74-
- `yarn test:browser`
74+
- `yarn test:karma`
7575
7676
Run the browser tests in a real web browser.
7777

‎packages/kbn-plugin-generator/integration_tests/generate_plugin.test.js‎

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -69,8 +69,8 @@ describe(`running the plugin-generator via 'node scripts/generate_plugin.js plug
6969
});
7070

7171
describe(`then running`, () => {
72-
it(`'yarn test:browser' should exit 0`, async () => {
73-
await execa('yarn', ['test:browser'], {
72+
it(`'yarn test:karma' should exit 0`, async () => {
73+
await execa('yarn', ['test:karma'], {
7474
cwd: generatedPath,
7575
env: {
7676
DISABLE_JUNIT_REPORTER: '1',

‎packages/kbn-plugin-generator/sao_template/template/README.md‎

Lines changed: 1 addition & 29 deletions
Original file line numberDiff line numberDiff line change
@@ -8,32 +8,4 @@
88

99
## development
1010

11-
See the [kibana contributing guide](https://github.com/elastic/kibana/blob/master/CONTRIBUTING.md) for instructions setting up your development environment. Once you have completed that, use the following yarn scripts.
12-
13-
- `yarn kbn bootstrap`
14-
15-
Install dependencies and crosslink Kibana and all projects/plugins.
16-
17-
> ***IMPORTANT:*** Use this script instead of `yarn` to install dependencies when switching branches, and re-run it whenever your dependencies change.
18-
19-
- `yarn start`
20-
21-
Start kibana and have it include this plugin. You can pass any arguments that you would normally send to `bin/kibana`
22-
23-
```
24-
yarn start --elasticsearch.hosts http://localhost:9220
25-
```
26-
27-
- `yarn build`
28-
29-
Build a distributable archive of your plugin.
30-
31-
- `yarn test:browser`
32-
33-
Run the browser tests in a real web browser.
34-
35-
- `yarn test:mocha`
36-
37-
Run the server tests using mocha.
38-
39-
For more information about any of these commands run `yarn ${task} --help`. For a full list of tasks checkout the `package.json` file, or run `yarn run`.
11+
See the [kibana contributing guide](https://github.com/elastic/kibana/blob/master/CONTRIBUTING.md) for instructions setting up your development environment.

‎packages/kbn-plugin-helpers/README.md‎

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -30,8 +30,8 @@ $ plugin-helpers help
3030
start Start kibana and have it include this plugin
3131
build [options] [files...] Build a distributable archive
3232
test Run the server and browser tests
33-
test:browser [options] Run the browser tests in a real web browser
34-
test:server [files...] Run the server tests using mocha
33+
test:karma [options] Run the browser tests in a real web browser
34+
test:mocha [files...] Run the server tests using mocha
3535

3636
Options:
3737

‎packages/kbn-plugin-helpers/cli.js‎

Lines changed: 6 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -66,24 +66,24 @@ program
6666
.action(createCommanderAction('testAll'));
6767

6868
program
69-
.command('test:browser')
69+
.command('test:karma')
7070
.description('Run the browser tests in a real web browser')
7171
.option('--dev', 'Enable dev mode, keeps the test server running')
7272
.option('-p, --plugins <plugin-ids>', "Manually specify which plugins' test bundles to run")
73-
.on('--help', docs('test/browser'))
73+
.on('--help', docs('test/karma'))
7474
.action(
75-
createCommanderAction('testBrowser', command => ({
75+
createCommanderAction('testKarma', command => ({
7676
dev: Boolean(command.dev),
7777
plugins: command.plugins,
7878
}))
7979
);
8080

8181
program
82-
.command('test:server [files...]')
82+
.command('test:mocha [files...]')
8383
.description('Run the server tests using mocha')
84-
.on('--help', docs('test/server'))
84+
.on('--help', docs('test/mocha'))
8585
.action(
86-
createCommanderAction('testServer', (command, files) => ({
86+
createCommanderAction('testMocha', (command, files) => ({
8787
files: files,
8888
}))
8989
);

‎packages/kbn-plugin-helpers/lib/index.d.ts‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -21,6 +21,6 @@ export function babelRegister(): void;
2121
export function resolveKibanaPath(path: string): string;
2222
export function readFtrConfigFile(path: string): any;
2323
export function run(
24-
task: 'build' | 'start' | 'testAll' | 'testBrowser' | 'testServer' | 'postinstall',
24+
task: 'build' | 'start' | 'testAll' | 'testKarma' | 'testMocha' | 'postinstall',
2525
options: any
2626
): Promise<void>;

‎packages/kbn-plugin-helpers/lib/tasks.js‎

Lines changed: 4 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -20,15 +20,15 @@
2020
const buildTask = require('../tasks/build');
2121
const startTask = require('../tasks/start');
2222
const testAllTask = require('../tasks/test/all');
23-
const testBrowserTask = require('../tasks/test/browser');
24-
const testServerTask = require('../tasks/test/server');
23+
const testKarmaTask = require('../tasks/test/karma');
24+
const testMochaTask = require('../tasks/test/mocha');
2525
const postinstallTask = require('../tasks/postinstall');
2626

2727
module.exports = {
2828
build: buildTask,
2929
start: startTask,
3030
testAll: testAllTask,
31-
testBrowser: testBrowserTask,
32-
testServer: testServerTask,
31+
testKarma: testKarmaTask,
32+
testMocha: testMochaTask,
3333
postinstall: postinstallTask,
3434
};

‎packages/kbn-plugin-helpers/tasks/test/all/README.md‎

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -1,3 +1,3 @@
1-
Runs both the server and browser tests, in that order.
1+
Runs both the mocha and karma tests, in that order.
22

3-
This is just a simple caller to both `test/server` and `test/browser`
3+
This is just a simple caller to both `test/mocha` and `test/karma`

0 commit comments

Comments
 (0)