this should probably also be added to

failure is
Download failed: server closed connection. URL: https://cdn.playwright.dev/dbazure/download/playwright/builds/chromium/1155/chromium-linux.zip
@radical do we have enough retries on this ? I've seen it before.

failure is Download failed: server closed connection. URL: https://cdn.playwright.dev/dbazure/download/playwright/builds/chromium/1155/chromium-linux.zip @radical do we have enough retries on this ? I've seen it before.

Saw the same error from the test runs on the first commit to this PR. The URL works fine from my home network; test pool getting rate limited?

@radical copilot says htere's no way to request retries of the playwright download short of looping in msbuild

<PropertyGroup>
  <MaxRetries>3</MaxRetries>
  <RetryCount>0</RetryCount>
</PropertyGroup>

<Target Name="InstallPlaywrightWithRetry">
  <Exec Command="dotnet exec --runtimeconfig $(MSBuildThisFileDirectory)Microsoft.Playwright.runtimeconfig.json $(_MicrosoftPlaywrightDllPath) install @(BrowsersForPlaywrightToInstall, ' ')"
        EnvironmentVariables="@(_EnvVarsForPlaywrightInstall)"
        IgnoreExitCode="true">
    <Output TaskParameter="ExitCode" PropertyName="PlaywrightExitCode" />
  </Exec>

  <Message Text="Playwright install attempt $(RetryCount), ExitCode: $(PlaywrightExitCode)" Importance="high" />

  <Error Condition="'$(PlaywrightExitCode)' != '0' and '$(RetryCount)' == '$(MaxRetries)'" Text="Playwright install failed after $(MaxRetries) attempts." />

  <PropertyGroup>
    <RetryCount Condition="'$(PlaywrightExitCode)' != '0'">$(RetryCount)</RetryCount>
    <RetryCount Condition="'$(PlaywrightExitCode)' != '0'">$([MSBuild]::Add($(RetryCount), 1))</RetryCount>
  </PropertyGroup>

  <CallTarget Condition="'$(PlaywrightExitCode)' != '0' and '$(RetryCount)' != '$(MaxRetries)'" Targets="InstallPlaywrightWithRetry" />
</Target>

Green tests, so I'll merge this to unblock the other PRs hitting test failures.

thanks @danegsta . is there a product bug here for 9.3?

Are we sure this isn’t a product issue?

thanks @danegsta . is there a product bug here for 9.3?

Best guess is that Docker on the test pool is hanging on the docker info command. We added the workaround we're using here in 9.2 after a couple users reported similar behavior (but we couldn't reproduce at the time). I do want to make a change to switch from using docker info in our runtime health check to checking if Docker is a symlink instead, but I don't know if we want to take that change in 9.3 if we aren't running into it outside of the test machines.

This sounds related to #8737 - I see tests intermittently fail on 9.2 for RuntimeUnhealthy, even though other containers have started up successfully (had one fail like that this morning). Also see similar with the dashboard, although dashboard is less impacted as the RuntimeHealthy resources eventually retry and start up.

I've made a change in DCP main to avoid calling docker info in the runtime checks.