Is this for visual regression testing? Should we also fail the tests for CI if a new screenshot is generated? That way we can ensure the modified screenshots are committed?

@tylersmalley Right, exactly. I'm planning on using these in conjunction with our screenshot-comparison tool (#7395) to protect against visual regressions while I refactor our styles.

I think it would be great to incorporate it into our automated testing, but I think we need to use the tool manually for awhile before we can use it to fail builds. For example, right now it can fail due to a resource not loading in time (e.g. a font), which results in a false positive, so we'd need to fix that issue first.

That sounds great. Seeing how this is simply adding more screenshots, this LGTM.

I am getting a error when running test:visualRegression`:

./node_modules/.bin/grunt  test:visualRegression
Compiling all doT templates...
Compiling visual_regression_gallery.dot to function
Running "test:visualRegression" task
(97.57%) screenshot-LineChart.png
(95.99%) screenshot-VerticalBarChart.png
(97.38%) screenshot-DataTable.png
(95.48%) screenshot-AreaChart.png
{ [Error: ENOENT: no such file or directory, open '/Users/tyler/Code/tylersmalley/kibana/test/screenshots/baseline/screenshot-LineChart.png']
  cause:
   { [Error: ENOENT: no such file or directory, open '/Users/tyler/Code/tylersmalley/kibana/test/screenshots/baseline/screenshot-LineChart.png']
     errno: -2,
     code: 'ENOENT',
     syscall: 'open',
     path: '/Users/tyler/Code/tylersmalley/kibana/test/screenshots/baseline/screenshot-LineChart.png' },
  isOperational: true,
  errno: -2,
  code: 'ENOENT',
  syscall: 'open',
  path: '/Users/tyler/Code/tylersmalley/kibana/test/screenshots/baseline/screenshot-LineChart.png' }
Warning: Task "test:visualRegression" failed. Use --force to continue.

Aborted due to warnings.

Ah, that's because the task looks in your screenshots/session directory and tries to compare against files with the same name in the screenshots/baseline directory. I changed the way the screenshots are named, so you'll need to run the functional tests first (to re-generate all the session screenshots), and then run test:visualRegression again.

This ran as expected after the clean:screenshots task was implemented.

The gallery looks great, and I could definitely see us adding filtering down the road.