[Reporting] Remove Chome stdout/stderr observables, Add Browser Logger observable by tsullivan · Pull Request #44359 · elastic/kibana

tsullivan · 2019-08-29T07:56:04Z

Summary

Previous code was not successfully capturing browser messages from the web inspector console, or for stdout/stderr of the Chromium process.

This PR fixes the log forwarding of the headless browser, wherever I found it to be possible. Any message tagged as error is tagged as an error in the Kibana logs. Any other message is tagged as debug.

This PR also completely removes the old handling of text data stream handling that were available prior to integrating Puppeteer: nssError$, fontError$, noUsableSandbox$. We can file an issue in Puppeteer to get an investigation of why we can't check for stdout messages.

The additional logging is tagged as headless-browser-console. Here are some samples:

Replaces: #44056

Remove handling of nssError$, fontError$, noUsableSandbox$

elasticmachine · 2019-08-29T07:56:06Z

Pinging @elastic/kibana-stack-services

tsullivan · 2019-08-29T07:56:43Z

x-pack/legacy/plugins/reporting/server/browsers/chromium/driver_factory/index.ts

+      logger.error(
+        `The Reporting plugin encountered issues launching Chromium in a self-test. You may have trouble generating reports.`
      );
+      logger.error(error);


This fixes an unrelated problem where an error object was getting concatenated with a string, making it unreadable

elasticmachine · 2019-08-29T09:23:20Z

💔 Build Failed

continuous-integration/kibana-ci/pull-request

…-browser-console-logging

elasticmachine · 2019-08-30T19:36:28Z

💚 Build Succeeded

continuous-integration/kibana-ci/pull-request

elasticmachine · 2019-09-03T22:31:51Z

💔 Build Failed

continuous-integration/kibana-ci/pull-request

joelgriffith

LGTM

tsullivan · 2019-09-05T17:16:22Z

x-pack/legacy/plugins/reporting/server/browsers/chromium/driver_factory/index.ts

-
-      const [consoleMessage$, message$] = partition(
-        (msg: string) => !!msg.match(/\[\d+\/\d+.\d+:\w+:CONSOLE\(\d+\)\]/)
-      )(stderr$);


Removing this code as it does not work with the puppeteer driver. I believe this was written at a time when we launched Chromium directly, without a helper. As of now, we no longer have a way to scan the browser's stdin / stderr as a stream.

I believe this only works if you use dumpio in puppeteer launch: https://pptr.dev/#?product=Puppeteer&version=v1.19.0&show=api-puppeteerlaunchoptions. We don't expose an option to do this, so I'm cool with removing this

Same here. As we have explored using dumpio before and found that it doesn't provide a stream of helpful info for us.

elasticmachine · 2019-09-05T19:04:24Z

💚 Build Succeeded

continuous-integration/kibana-ci/pull-request

tsullivan · 2019-09-05T20:18:15Z

@joelgriffith and I have discussed the risk of removing the stdout/stderr observables, and we both agree that is dead code per limitations in Puppeteer. We are going to move forward with this. I will file an issue that we must try to restore the behavior that can scan text on the reason for Chromium failing to run and expose that to the user on a job-per-job basis.

…r observable (elastic#44359) * Add browserLogger$, processLogger$ Remove handling of nssError$, fontError$, noUsableSandbox$ * clean up messy logs being forwarded

…r observable (#44359) (#44940) * Add browserLogger$, processLogger$ Remove handling of nssError$, fontError$, noUsableSandbox$ * clean up messy logs being forwarded

…ete-for-distance_feature * 'master' of github.com:elastic/kibana: (89 commits) Replace TSVB timeseries charts with elastic-charts (elastic#33558) [TSVB][Top N aggregation] Unable to deal with negative values (elastic#43581) [alerting] Adds Action Type configuration support and whitelisting (elastic#44483) FTR: fix WebDriver Actions calls (elastic#44605) [Code] add NodeRepositoriesService to watch new repositories on local node (elastic#44677) [skip-ci][Maps] Improve Maps intro page (elastic#44721) [Maps] Update titles and descriptions for data sources (elastic#44833) Types + Extract Integration Util (elastic#44433) Downgrade log level from info to debug for cases when we cannot handle authentication attempt. (elastic#44933) [Reporting] Remove Chome stdout/stderr observables, Add Browser Logger observable (elastic#44359) Update Jest script to output coverage (elastic#44447) [ftr] support --kibana-install-dir flag (elastic#44552) [WATCHER] Allow user to set a threshold value of 0 (elastic#44810) Remove injectI18n in dashboard plugin. (elastic#44580) [Graph] Save modal (elastic#44261) Use external script for the OIDC Implicit flow handler page. (elastic#44866) disable router prefixing with pluginId (elastic#44855) [SIEM] Fix bug on url + inspect functionality on hosts/hostDetails page (elastic#44671) [ML] File data viz limiting uploaded doc chunk size (elastic#44768) [code] Append go env variable 'GOCACHE' to go lsp spawn command. (elastic#44864) ...

…plate * 'master' of github.com:elastic/kibana: (91 commits) [APM] Make number of x ticks responsive to the plot width (elastic#44870) [ML] Single metric viewer: Fix top nav refresh behaviour. (elastic#44860) Replace TSVB timeseries charts with elastic-charts (elastic#33558) [TSVB][Top N aggregation] Unable to deal with negative values (elastic#43581) [alerting] Adds Action Type configuration support and whitelisting (elastic#44483) FTR: fix WebDriver Actions calls (elastic#44605) [Code] add NodeRepositoriesService to watch new repositories on local node (elastic#44677) [skip-ci][Maps] Improve Maps intro page (elastic#44721) [Maps] Update titles and descriptions for data sources (elastic#44833) Types + Extract Integration Util (elastic#44433) Downgrade log level from info to debug for cases when we cannot handle authentication attempt. (elastic#44933) [Reporting] Remove Chome stdout/stderr observables, Add Browser Logger observable (elastic#44359) Update Jest script to output coverage (elastic#44447) [ftr] support --kibana-install-dir flag (elastic#44552) [WATCHER] Allow user to set a threshold value of 0 (elastic#44810) Remove injectI18n in dashboard plugin. (elastic#44580) [Graph] Save modal (elastic#44261) Use external script for the OIDC Implicit flow handler page. (elastic#44866) disable router prefixing with pluginId (elastic#44855) [SIEM] Fix bug on url + inspect functionality on hosts/hostDetails page (elastic#44671) ...

tsullivan added 2 commits August 29, 2019 00:10

Add browserLogger$, processLogger$

82b17d5

Remove handling of nssError$, fontError$, noUsableSandbox$

clean up messy logs being forwarded

570f411

tsullivan added release_note:enhancement review zDeprecated Feature:Reporting Use Reporting:Screenshot, Reporting:CSV, or Reporting:Framework instead v8.0.0 Team:Stack Services v7.5.0 labels Aug 29, 2019

tsullivan requested a review from joelgriffith August 29, 2019 07:56

tsullivan commented Aug 29, 2019

View reviewed changes

Merge branch 'master' of github.com:elastic/kibana into reporting/fix…

041d864

…-browser-console-logging

tsullivan mentioned this pull request Aug 30, 2019

[Reporting] Fix headless browser console log capturing #44056

Closed

7 tasks

Merge branch 'master' into reporting/fix-browser-console-logging

121e87e

tsullivan changed the title ~~Add browserLogger$, processLogger$~~ [Reporting] Remove Chome stdin observables, Add Browser Logger observable Sep 3, 2019

joelgriffith approved these changes Sep 5, 2019

View reviewed changes

tsullivan commented Sep 5, 2019

View reviewed changes

Merge branch 'master' into reporting/fix-browser-console-logging

1377459

tsullivan changed the title ~~[Reporting] Remove Chome stdin observables, Add Browser Logger observable~~ [Reporting] Remove Chome stdout/stderr observables, Add Browser Logger observable Sep 5, 2019

tsullivan merged commit 24137e8 into elastic:master Sep 5, 2019

tsullivan deleted the reporting/fix-browser-console-logging branch September 5, 2019 20:21

This was referenced Sep 5, 2019

[7.x] [Reporting] Remove Chome stdout/stderr observables, Add Browser Logger observable (#44359) #44940

Merged

[Reporting] Restore handling of Chromium stdout / stderr messages #44942

Closed

tsullivan mentioned this pull request Sep 13, 2019

[Reporting] Browser Driver is incorrectly logging browser console messages #43295

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[Reporting] Remove Chome stdout/stderr observables, Add Browser Logger observable#44359

[Reporting] Remove Chome stdout/stderr observables, Add Browser Logger observable#44359
tsullivan merged 5 commits intoelastic:masterfrom
tsullivan:reporting/fix-browser-console-logging

tsullivan commented Aug 29, 2019 •

edited

Loading

Uh oh!

elasticmachine commented Aug 29, 2019

Uh oh!

tsullivan Aug 29, 2019

Uh oh!

elasticmachine commented Aug 29, 2019

Uh oh!

elasticmachine commented Aug 30, 2019

Uh oh!

elasticmachine commented Sep 3, 2019

Uh oh!

joelgriffith left a comment

Uh oh!

tsullivan Sep 5, 2019

Uh oh!

joelgriffith Sep 5, 2019

Uh oh!

tsullivan Sep 5, 2019

Uh oh!

elasticmachine commented Sep 5, 2019

Uh oh!

tsullivan commented Sep 5, 2019

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Conversation

tsullivan commented Aug 29, 2019 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Uh oh!

elasticmachine commented Aug 29, 2019

Uh oh!

tsullivan Aug 29, 2019

Choose a reason for hiding this comment

Uh oh!

elasticmachine commented Aug 29, 2019

💔 Build Failed

Uh oh!

elasticmachine commented Aug 30, 2019

💚 Build Succeeded

Uh oh!

elasticmachine commented Sep 3, 2019

💔 Build Failed

Uh oh!

joelgriffith left a comment

Choose a reason for hiding this comment

Uh oh!

tsullivan Sep 5, 2019

Choose a reason for hiding this comment

Uh oh!

joelgriffith Sep 5, 2019

Choose a reason for hiding this comment

Uh oh!

tsullivan Sep 5, 2019

Choose a reason for hiding this comment

Uh oh!

elasticmachine commented Sep 5, 2019

💚 Build Succeeded

Uh oh!

tsullivan commented Sep 5, 2019

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

tsullivan commented Aug 29, 2019 •

edited

Loading