The Reporting plugin has the ability to take a URL to a Kibana page, and a few fields of metadata, and return an observable that resolves to a screenshot of the page, in the form of buffer data.
To execute this task, several things happen in within the pipeline of the observable:
openUrl: Open a browser page to the URL using the Puppeteer library
- Authentication is provided via
Basic authentication in the request headers. In order not to reveal the authentication headers outside of the server origin, Reporting intercepts each request made by the browser to check if authentication is required for the request.
- Reporting knows if it has found the expected page by waiting for the
.application selector to resolve.
skipTelemetry(): Attempt to have the browser instance avoid Telemetry calls by injecting data to into localStorage that telemetry does not need to be sent [1]
getNumberOfItems(): Read data from the DOM of the Kibana page to get a count of how many visualization items are on the page
- It needs that info to know how many visualizations should "check in" with custom events to signal to Reporting that they are done rendering.
driver.setViewport()Set the browser viewport and "zoom" based on the metadata
waitForElementsToBeInDOM: Wait for all visualization elements to be in the DOM
injectCss(): Inject custom CSS that hacks the Kibana page into a Report style
- hides the navigation and page menus
- hides interactive elements such as buttons and filter inputs
layout.positionElements(): Conditional step: if the visualization elements need to be re-arranged for "print mode," do it here
waitForRenderComplete(): Wait for all visualization elements to be done rendering
getElementPositionAndAttributes: Find (a) the x/y position of each visualization, and (b) title from the DOM for each visualization, that will be captured on the page
- Note: the element position / data can be a set of multiple items for "print" mode of a dashboard
getTimeRange: Read the time range from the filter bar DOM so it can be added to the screenshot title [2]
getScreenshots: Capture screenshots of each visualization on the page, using the element position info and return it, along with the date range and titles info
[1] This is a problematic arrangement because it forces Reporting to be aware of Telemetry and how to talk to it by manipulating Telemetry data in localStorage - also, it doesn't work properly.
[2] Steps 9 and 10 happen in parallel
Once the buffer data is available to the higher level code that runs the capture flow, it gets converted to base64 data, and the Reporting job document gets updated with the base64 and the "status" and "completed_at" fields updated.
Between the first and last step, there are parts that do things to the page to prepare it for reports, and parts that check the progress of the preparations by waiting for CSS selectors.
Plan to improve performance.
We need a plan to improve performance in Reporting for these steps.
Benchmark the current state of performance
We can define a "typical" dashboard as one of the sample data dashboards. We'll run many tests of generating a PNG report, while capture verbose logs with timestamps. Based on that information, we can find the average time taken by each part of the steps in the pipeline.
Benchmarks will help us focus on fixing the parts where the most time gets wasted, and gives us something to compare against after the improvements are done.
Tweak the pipeline
Based on the results of the benchmarking, we can make small changes to the pipeline to chase some improvements in the timing. Presumably, running more things in parallel could be something done as part of this.
Add a "Screenshot Mode Service" to Kibana core
There are steps in the pipeline that can be moved away to other parts of Kibana that "prepare" the page for Kibana. These parts wouldn't be needed if the preparations happen automatically when Reporting opens the page. That would be a huge performance boost for a few reasons:
- No wasted rendering that gets replaced with the Reporting styles
- No "interaction" components loading or fetching data in the background, which consumes resources.
Screenshot Mode Service RFC: #59084
This issue will be updated with some benchmark results.
The Reporting plugin has the ability to take a URL to a Kibana page, and a few fields of metadata, and return an observable that resolves to a screenshot of the page, in the form of buffer data.
To execute this task, several things happen in within the pipeline of the observable:
openUrl: Open a browser page to the URL using the Puppeteer libraryBasicauthentication in the request headers. In order not to reveal the authentication headers outside of the server origin, Reporting intercepts each request made by the browser to check if authentication is required for the request..applicationselector to resolve.skipTelemetry(): Attempt to have the browser instance avoid Telemetry calls by injecting data to into localStorage that telemetry does not need to be sent [1]getNumberOfItems(): Read data from the DOM of the Kibana page to get a count of how many visualization items are on the pagedriver.setViewport()Set the browser viewport and "zoom" based on the metadatawaitForElementsToBeInDOM: Wait for all visualization elements to be in the DOMinjectCss(): Inject custom CSS that hacks the Kibana page into a Report stylelayout.positionElements(): Conditional step: if the visualization elements need to be re-arranged for "print mode," do it herewaitForRenderComplete(): Wait for all visualization elements to be done renderinggetElementPositionAndAttributes: Find (a) the x/y position of each visualization, and (b) title from the DOM for each visualization, that will be captured on the pagegetTimeRange: Read the time range from the filter bar DOM so it can be added to the screenshot title [2]getScreenshots: Capture screenshots of each visualization on the page, using the element position info and return it, along with the date range and titles info[1] This is a problematic arrangement because it forces Reporting to be aware of Telemetry and how to talk to it by manipulating Telemetry data in localStorage - also, it doesn't work properly.
[2] Steps 9 and 10 happen in parallel
Once the buffer data is available to the higher level code that runs the capture flow, it gets converted to base64 data, and the Reporting job document gets updated with the base64 and the "status" and "completed_at" fields updated.
Between the first and last step, there are parts that do things to the page to prepare it for reports, and parts that check the progress of the preparations by waiting for CSS selectors.
Plan to improve performance.
We need a plan to improve performance in Reporting for these steps.
Benchmark the current state of performance
We can define a "typical" dashboard as one of the sample data dashboards. We'll run many tests of generating a PNG report, while capture verbose logs with timestamps. Based on that information, we can find the average time taken by each part of the steps in the pipeline.
Benchmarks will help us focus on fixing the parts where the most time gets wasted, and gives us something to compare against after the improvements are done.
Tweak the pipeline
Based on the results of the benchmarking, we can make small changes to the pipeline to chase some improvements in the timing. Presumably, running more things in parallel could be something done as part of this.
Add a "Screenshot Mode Service" to Kibana core
There are steps in the pipeline that can be moved away to other parts of Kibana that "prepare" the page for Kibana. These parts wouldn't be needed if the preparations happen automatically when Reporting opens the page. That would be a huge performance boost for a few reasons:
Screenshot Mode Service RFC: #59084
This issue will be updated with some benchmark results.