chore: add visual reporting option (#13511)

wswebcreation · web-flow · commit f38407401c41 · 2024-09-03T10:11:52.000+02:00
diff --git a/website/docs/visual-testing/service-options.md b/website/docs/visual-testing/service-options.md
@@ -14,14 +14,14 @@ export const config = {
     // =====
     services: [
         [
-            'visual',
+            "visual",
             {
                 // The options
-            }
-        ]
-    ]
+            },
+        ],
+    ],
     // ...
-}
+};
 ```
 
 ## Default Options
@@ -110,9 +110,9 @@ The directory will hold all the baseline images that are used during the compari
 ```js
 {
     baselineFolder: (options) => {
-        const testFolder = path.dirname(options.specs[0])
-        return path.join(testFolder, 'snapshots', type)
-    }
+        const testFolder = path.dirname(options.specs[0]);
+        return path.join(testFolder, "snapshots", type);
+    };
 }
 ```
 
@@ -129,6 +129,131 @@ Delete runtime folder (`actual` & `diff) on initialization
 This will only work when the [`screenshotPath`](#screenshotpath) is set through the plugin options, and **WILL NOT WORK** when you set the folders in the methods
 :::
 
+### `createJsonReportFiles` **(NEW)**
+
+-   **Type:** `boolean`
+-   **Mandatory:** No
+-   **Default:** `false`
+
+You now have the option to export the compare results into a JSON report file. By providing the option `createJsonReportFiles: true`, each image that is compared will create a report stored in the `actual` folder, next to each `actual` image result. The output will look like this:
+
+```json
+{
+    "parent": "check methods",
+    "test": "should fail comparing with a baseline",
+    "tag": "examplePageFail",
+    "instanceData": {
+        "browser": {
+            "name": "chrome-headless-shell",
+            "version": "126.0.6478.183"
+        },
+        "platform": {
+            "name": "mac",
+            "version": "not-known"
+        }
+    },
+    "commandName": "checkScreen",
+    "boundingBoxes": {
+        "diffBoundingBoxes": [
+            {
+                "left": 1088,
+                "top": 717,
+                "right": 1186,
+                "bottom": 730
+            }
+            //....
+        ],
+        "ignoredBoxes": [
+            {
+                "left": 159,
+                "top": 652,
+                "right": 356,
+                "bottom": 703
+            }
+            //...
+        ]
+    },
+    "fileData": {
+        "actualFilePath": "/Users/wdio/visual-testing/.tmp/actual/desktop_chrome-headless-shellexamplePageFail-local-chrome-latest-1366x768.png",
+        "baselineFilePath": "/Users/wdio/visual-testing/localBaseline/desktop_chrome-headless-shellexamplePageFail-local-chrome-latest-1366x768.png",
+        "diffFilePath": "/Users/wdio/visual-testing/.tmp/diff/desktop_chrome-headless-shell/examplePageFail-local-chrome-latest-1366x768png",
+        "fileName": "examplePageFail-local-chrome-latest-1366x768.png",
+        "size": {
+            "actual": {
+                "height": 768,
+                "width": 1366
+            },
+            "baseline": {
+                "height": 768,
+                "width": 1366
+            },
+            "diff": {
+                "height": 768,
+                "width": 1366
+            }
+        }
+    },
+    "misMatchPercentage": "12.90",
+    "rawMisMatchPercentage": 12.900729014153246
+}
+```
+
+When all tests are executed, a new JSON file with the collection of the comparisons will be generated and can be found in the root of your `actual` folder. The data is grouped by:
+
+-   `describe` for Jasmine/Mocha or `Feature` for CucumberJS
+-   `it` for Jasmine/Mocha or `Scenario` for CucumberJS
+    and then sorted by:
+-   `commandName`, which are the compare method names used to compare the images
+-   `instanceData`, browser first, then device, then platform
+    it will look like this
+
+```json
+[
+    {
+        "description": "check methods",
+        "data": [
+            {
+                "test": "should fail comparing with a baseline",
+                "data": [
+                    {
+                        "tag": "examplePageFail",
+                        "instanceData": {},
+                        "commandName": "checkScreen",
+                        "framework": "mocha",
+                        "boundingBoxes": {
+                            "diffBoundingBoxes": [],
+                            "ignoredBoxes": []
+                        },
+                        "fileData": {},
+                        "misMatchPercentage": "14.34",
+                        "rawMisMatchPercentage": 14.335403703025868
+                    },
+                    {
+                        "tag": "exampleElementFail",
+                        "instanceData": {},
+                        "commandName": "checkElement",
+                        "framework": "mocha",
+                        "boundingBoxes": {
+                            "diffBoundingBoxes": [],
+                            "ignoredBoxes": []
+                        },
+                        "fileData": {},
+                        "misMatchPercentage": "1.34",
+                        "rawMisMatchPercentage": 1.335403703025868
+                    }
+                ]
+            }
+        ]
+    }
+]
+```
+
+The report data will give you the opportunity to build your own visual report without doing all the magic and data collection yourself.
+
+:::info NOTE
+You need to use `@wdio/visual-testing` version `5.2.0` or higher
+:::
+
 ### `disableCSSAnimation`
 
 -   **Type:** `boolean`
@@ -146,12 +271,12 @@ and reset when done
 -   **Default:** `false`
 -   **Supported:** Web
 
-This will hide all text on a page so only the layout will be used for comparison. Hiding will be done by adding the style `'color': 'transparent !important'` to __each__ element.
+This will hide all text on a page so only the layout will be used for comparison. Hiding will be done by adding the style `'color': 'transparent !important'` to **each** element.
 
 For the output see [Test Output](./test-output#enablelayouttesting)
 
 :::info
-By using this flag each element that contains text (so not only `p, h1, h2, h3, h4, h5, h6, span, a, li`, but also `div|button|..`) will get this property. There is __no__ option to tailor this.
+By using this flag each element that contains text (so not only `p, h1, h2, h3, h4, h5, h6, span, a, li`, but also `div|button|..`) will get this property. There is **no** option to tailor this.
 :::
 
 ### `formatImageName`
