Fixes Failing smoke test artifacts snapshot: Visualize Visualize Reporting Screenshots PNG reports: sample data created in 7.6 TSVB Gauge: PNG file matches the baseline image by ymao1 · Pull Request #226067 · elastic/kibana

ymao1 · 2025-07-01T18:50:44Z

Summary

Test was failing with TypeError: Invalid URL, only on the smoke tests that are run on this pipeline, not locally.

The smoke tests run against real cloud deployments, so there is a difference in URLs between real deployments and local test runs. The issue occurs here:

kibana/x-pack/platform/test/functional/page_objects/reporting_page.ts

Lines 97 to 102 in dc09cb5

    
           const baseURL = formatUrl({ 
        
             ...kibanaServerConfig, 
        
             auth: false, 
        
           }); 
        
           const urlWithoutBase = fullUrl.replace(baseURL, ''); 
        
           const res = await this.security.testUserSupertest.get(urlWithoutBase);

For the smoke test, the fullUrl looks like https://kibana-pr-226067.kb.us-west2.gcp.elastic-cloud.com/internal/reporting/jobs/download/8446e53e-bafe-400d-8f28-4f923bfeadd8?elasticInternalOrigin=true but the baseUrl looks like https://kibana-pr-226067.kb.us-west2.gcp.elastic-cloud.com:443 because it's generated with the port. This causes the urlWithoutBase replacement to fail because the fullUrl does not include the port number.

For local testing, everything works fine because the fullUrl is http://localhost:5620/internal/reporting/jobs/download/1d9b0e64-b706-4916-855d-a5bbec9a5f02?elasticInternalOrigin=true and the baseUrl is http://localhost:5620 so the urlWithoutBase correctly ends up being /internal/reporting/jobs/download/1d9b0e64-b706-4916-855d-a5bbec9a5f02?elasticInternalOrigin=true

This fix catches any errors from the original request and retries them without the port in the baseUrl.

To Verify

Check out this PR branch
Get the credentials for the cloud deployment.
The smoke test is run using this script: .buildkite/scripts/steps/artifacts/cloud.sh. To recreate it, set your environment variables in your terminal with the cloud deployment info:

export TEST_KIBANA_PROTOCOL=https
export TEST_KIBANA_HOSTNAME=kibana-pr-226067.kb.us-west2.gcp.elastic-cloud.com
export TEST_KIBANA_PORT=443
export TEST_KIBANA_USERNAME=elastic
export TEST_KIBANA_PASSWORD=<password>

export TEST_ES_PROTOCOL=https
export TEST_ES_HOSTNAME=kibana-pr-226067.es.us-west2.gcp.elastic-cloud.com
export TEST_ES_PORT=443
export TEST_ES_USERNAME=elastic
export TEST_ES_PASSWORD=<password>

export TEST_BROWSER_HEADLESS=1
export NODE_TLS_REJECT_UNAUTHORIZED=0

Then run node --no-warnings scripts/functional_test_runner.js --config x-pack/test/functional/apps/visualize/config.ts --include-tag=smoke --grep "TSVB Gauge: PNG file matches the baseline image"

The test should pass.

elasticmachine · 2025-07-02T11:33:06Z

Pinging @elastic/response-ops (Team:ResponseOps)

x-pack/platform/test/functional/page_objects/reporting_page.ts

…189590

markov00

Thanks for the changes, LGTM!

ymao1 · 2025-07-08T12:03:51Z

@elasticmachine merge upstream

elasticmachine · 2025-07-08T12:58:57Z

💚 Build Succeeded

Metrics [docs]

✅ unchanged

History

💚 Build #315670 succeeded 238bbe0
💔 Build #315444 failed 7267981
💔 Build #315407 failed 418f86e
💚 Build #314734 succeeded d4b4ae0
💚 Build #314687 succeeded d3b2c6f

cc @ymao1

JiaweiWu

LGTM!

…rting Screenshots PNG reports: sample data created in 7.6 TSVB Gauge: PNG file matches the baseline image (elastic#226067) Fixes elastic#189590 ## Summary Test was failing with `TypeError: Invalid URL`, only on the smoke tests that are run on [this pipeline](https://buildkite.com/elastic/kibana-artifacts-snapshot), not locally. The smoke tests run against real cloud deployments, so there is a difference in URLs between real deployments and local test runs. The issue occurs here: https://github.com/elastic/kibana/blob/dc09cb5d8abe1d6a52977d0a7d8c69778c1eabd0/x-pack/platform/test/functional/page_objects/reporting_page.ts#L97-L102 For the smoke test, the `fullUrl` looks like `https://kibana-pr-226067.kb.us-west2.gcp.elastic-cloud.com/internal/reporting/jobs/download/8446e53e-bafe-400d-8f28-4f923bfeadd8?elasticInternalOrigin=true` but the `baseUrl` looks like `https://kibana-pr-226067.kb.us-west2.gcp.elastic-cloud.com:443` because it's generated with the port. This causes the `urlWithoutBase` replacement to fail because the `fullUrl` does not include the port number. For local testing, everything works fine because the `fullUrl` is `http://localhost:5620/internal/reporting/jobs/download/1d9b0e64-b706-4916-855d-a5bbec9a5f02?elasticInternalOrigin=true` and the `baseUrl` is `http://localhost:5620` so the `urlWithoutBase` correctly ends up being `/internal/reporting/jobs/download/1d9b0e64-b706-4916-855d-a5bbec9a5f02?elasticInternalOrigin=true` This fix catches any errors from the original request and retries them without the port in the `baseUrl`. ## To Verify 1. Check out this PR branch 2. Get the credentials for the cloud deployment. 3. The smoke test is run using this script: `.buildkite/scripts/steps/artifacts/cloud.sh`. To recreate it, set your environment variables in your terminal with the cloud deployment info: ``` export TEST_KIBANA_PROTOCOL=https export TEST_KIBANA_HOSTNAME=kibana-pr-226067.kb.us-west2.gcp.elastic-cloud.com export TEST_KIBANA_PORT=443 export TEST_KIBANA_USERNAME=elastic export TEST_KIBANA_PASSWORD=<password> export TEST_ES_PROTOCOL=https export TEST_ES_HOSTNAME=kibana-pr-226067.es.us-west2.gcp.elastic-cloud.com export TEST_ES_PORT=443 export TEST_ES_USERNAME=elastic export TEST_ES_PASSWORD=<password> export TEST_BROWSER_HEADLESS=1 export NODE_TLS_REJECT_UNAUTHORIZED=0 ``` Then run `node --no-warnings scripts/functional_test_runner.js --config x-pack/test/functional/apps/visualize/config.ts --include-tag=smoke --grep "TSVB Gauge: PNG file matches the baseline image"` The test should pass. --------- Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>

ymao1 added the ci:cloud-deploy Create or update a Cloud deployment label Jul 1, 2025

ymao1 force-pushed the reporting-test-189590 branch from 23fc205 to df50546 Compare July 2, 2025 01:14

Fixing invalid url

d4b4ae0

ymao1 force-pushed the reporting-test-189590 branch from df50546 to d4b4ae0 Compare July 2, 2025 01:15

ymao1 changed the title ~~Reporting test 189590~~ Fixes Failing smoke test artifacts snapshot: Visualize Visualize Reporting Screenshots PNG reports: sample data created in 7.6 TSVB Gauge: PNG file matches the baseline image Jul 2, 2025

ymao1 self-assigned this Jul 2, 2025

ymao1 marked this pull request as ready for review July 2, 2025 11:33

ymao1 requested review from a team as code owners July 2, 2025 11:33

markov00 reviewed Jul 3, 2025

View reviewed changes

x-pack/platform/test/functional/page_objects/reporting_page.ts Outdated Show resolved Hide resolved

ymao1 added 2 commits July 3, 2025 08:25

PR feedback

3b1acac

Merge branch 'main' of github.com:elastic/kibana into reporting-test-…

418f86e

…189590

ymao1 requested a review from markov00 July 3, 2025 12:26

ymao1 added 3 commits July 3, 2025 09:25

Fixing lint errors

7267981

Handling url fragments

eb7db87

Merge branch 'main' of github.com:elastic/kibana into reporting-test-…

ddd5198

…189590

ymao1 requested a review from a team as a code owner July 4, 2025 02:35

Cleanup

238bbe0

ymao1 removed the request for review from a team July 4, 2025 02:36

markov00 approved these changes Jul 7, 2025

View reviewed changes

Merge branch 'main' into reporting-test-189590

bc5c41b

JiaweiWu approved these changes Jul 8, 2025

View reviewed changes

ymao1 merged commit df6f6c4 into elastic:main Jul 8, 2025
12 checks passed

ymao1 deleted the reporting-test-189590 branch July 8, 2025 17:21

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Fixes Failing smoke test artifacts snapshot: Visualize Visualize Reporting Screenshots PNG reports: sample data created in 7.6 TSVB Gauge: PNG file matches the baseline image#226067

Fixes Failing smoke test artifacts snapshot: Visualize Visualize Reporting Screenshots PNG reports: sample data created in 7.6 TSVB Gauge: PNG file matches the baseline image#226067
ymao1 merged 8 commits intoelastic:mainfrom
ymao1:reporting-test-189590

ymao1 commented Jul 1, 2025 •

edited

Loading

Uh oh!

elasticmachine commented Jul 2, 2025

Uh oh!

Uh oh!

markov00 left a comment

Uh oh!

ymao1 commented Jul 8, 2025

Uh oh!

elasticmachine commented Jul 8, 2025 •

edited

Loading

Uh oh!

JiaweiWu left a comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

	const baseURL = formatUrl({
	...kibanaServerConfig,
	auth: false,
	});
	const urlWithoutBase = fullUrl.replace(baseURL, '');
	const res = await this.security.testUserSupertest.get(urlWithoutBase);

Conversation

ymao1 commented Jul 1, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

To Verify

Uh oh!

elasticmachine commented Jul 2, 2025

Uh oh!

Uh oh!

markov00 left a comment

Choose a reason for hiding this comment

Uh oh!

ymao1 commented Jul 8, 2025

Uh oh!

elasticmachine commented Jul 8, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

💚 Build Succeeded

Metrics [docs]

History

Uh oh!

JiaweiWu left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

ymao1 commented Jul 1, 2025 •

edited

Loading

elasticmachine commented Jul 8, 2025 •

edited

Loading