/botio-linux preview

From: Bot.io (Linux m4)

Received

Command cmd_preview from @timvandermeij received. Current queue size: 1

Live output at: http://54.241.84.105:8877/694a426049b12bc/output.txt

From: Bot.io (Linux m4)

Success

Full output at http://54.241.84.105:8877/694a426049b12bc/output.txt

Total script time: 1.19 mins

Published

• Viewer: http://54.241.84.105:8877/694a426049b12bc/web/viewer.html
• Viewer (legacy): http://54.241.84.105:8877/694a426049b12bc/legacy/web/viewer.html

/botio test

From: Bot.io (Linux m4)

Received

Command cmd_test from @timvandermeij received. Current queue size: 0

Live output at: http://54.241.84.105:8877/edcb325c096ae1d/output.txt

From: Bot.io (Windows)

Received

Command cmd_test from @timvandermeij received. Current queue size: 1

Live output at: http://54.193.163.58:8877/9324bc9eae55e48/output.txt

From: Bot.io (Linux m4)

Failed

Full output at http://54.241.84.105:8877/edcb325c096ae1d/output.txt

Total script time: 32.04 mins

• Unit tests: Passed
• Integration Tests: FAILED
• Regression tests: FAILED

  different ref/snapshot: 15
  different first/second rendering: 1

Image differences available at: http://54.241.84.105:8877/edcb325c096ae1d/reftest-analyzer.html#web=eq.log

From: Bot.io (Windows)

Failed

Full output at http://54.193.163.58:8877/9324bc9eae55e48/output.txt

Total script time: 46.86 mins

• Unit tests: Passed
• Integration Tests: FAILED
• Regression tests: FAILED

  different ref/snapshot: 4

Image differences available at: http://54.193.163.58:8877/9324bc9eae55e48/reftest-analyzer.html#web=eq.log

The Error: Cannot dismiss dialog which is already handled! integration test issue is a new one and most likely caused by the Puppeteer update. I'll look into what's causing that: it could be an issue upstream in Puppeteer, or indicate something that's not good in our test code that just surfaced because of the update. It's interesting that for these three tests the dialog.dismiss() call seems to fail, but that same call does work for other tests we have...

The closest upstream PR I could find related to this is puppeteer/puppeteer#12744 which touched this file recently (but in a seemingly trivial way).

I reproduced the issue locally and confirmed only Firefox is impacted by it: the validation dialog stays open while it should have closed. However, I noticed puppeteer/puppeteer#12744 (comment) and figured my Firefox 130 Nightly version might not have that fix yet, so I purged my local cache and indeed the newly downloaded Firefox 130 Nightly version works. We should remove the cached Firefox 130 Nightly versions from both the Windows and Linux bots so we can retry this.

@calixteman Could you perform this action on both bots? The paths to the Firefox binaries are printed at the top of the bot logs above. Thanks!

/botio test

From: Bot.io (Linux m4)

Received

Command cmd_test from @calixteman received. Current queue size: 0

Live output at: http://54.241.84.105:8877/93e7a69aae3f35b/output.txt

From: Bot.io (Windows)

Received

Command cmd_test from @calixteman received. Current queue size: 0

Live output at: http://54.193.163.58:8877/28bbafde9141f63/output.txt

From: Bot.io (Linux m4)

Failed

Full output at http://54.241.84.105:8877/93e7a69aae3f35b/output.txt

Total script time: 15.32 mins

• Unit tests: Passed
• Integration Tests: FAILED
• Regression tests: FAILED

Image differences available at: http://54.241.84.105:8877/93e7a69aae3f35b/reftest-analyzer.html#web=eq.log

From: Bot.io (Windows)

Failed

Full output at http://54.193.163.58:8877/28bbafde9141f63/output.txt

Total script time: 48.38 mins

• Unit tests: Passed
• Integration Tests: FAILED
• Regression tests: FAILED

  different ref/snapshot: 2

Image differences available at: http://54.193.163.58:8877/28bbafde9141f63/reftest-analyzer.html#web=eq.log

/botio test

From: Bot.io (Windows)

Received

Command cmd_test from @timvandermeij received. Current queue size: 0

Live output at: http://54.193.163.58:8877/dec6810b8cb3389/output.txt

From: Bot.io (Linux m4)

Received

Command cmd_test from @timvandermeij received. Current queue size: 0

Live output at: http://54.241.84.105:8877/3372d921fce1c12/output.txt

I have converted this to draft because I'll update it later today with the latest updates from the last few days.

From: Bot.io (Linux m4)

Failed

Full output at http://54.241.84.105:8877/3372d921fce1c12/output.txt

Total script time: 36.39 mins

• Unit tests: FAILED
• Integration Tests: Passed
• Regression tests: FAILED

  errors: 1860
  different ref/snapshot: 9
  different first/second rendering: 4

Image differences available at: http://54.241.84.105:8877/3372d921fce1c12/reftest-analyzer.html#web=eq.log

From: Bot.io (Windows)

Failed

Full output at http://54.193.163.58:8877/dec6810b8cb3389/output.txt

Total script time: 45.89 mins

• Unit tests: Passed
• Integration Tests: Passed
• Regression tests: FAILED

  different ref/snapshot: 3

Image differences available at: http://54.193.163.58:8877/dec6810b8cb3389/reftest-analyzer.html#web=eq.log