Skip to content

Fix OmniSharp tests silently passing on script execution failures#3954

Merged
mthalman merged 8 commits intomainfrom
copilot/fix-omnisharp-tests-failure
Jan 13, 2026
Merged

Fix OmniSharp tests silently passing on script execution failures#3954
mthalman merged 8 commits intomainfrom
copilot/fix-omnisharp-tests-failure

Conversation

Copy link
Contributor

Copilot AI commented Dec 18, 2025

OmniSharp tests were passing when the run script failed with "mono: command not found". The test expects non-zero exit from killing the process after timeout, but script failures also produce non-zero exit codes, causing false positives.

Changes

  • Added assertions to detect shell execution failures in stderr:
    • "command not found" - catches missing executables (mono, bash, etc.)
    • "No such file or directory" - catches missing files/scripts
  • Both checks use case-insensitive comparison to handle error message variations
Assert.DoesNotContain("command not found", executeResult.StdErr, StringComparison.OrdinalIgnoreCase);
Assert.DoesNotContain("No such file or directory", executeResult.StdErr, StringComparison.OrdinalIgnoreCase);

Context

This issue is specific to main branch due to ExecuteHelper.cs being imported from dotnet/sdk in commit 94383bd. The release/10.0.1xx branch uses a different process execution implementation.

Original prompt

This section details on the original issue you should resolve

<issue_title>OmniSharp tests have been failing silently</issue_title>
<issue_description>OmniSharp tests show the following output:

Executing: /__w/1/s/artifacts/obj/extracted-dotnet-sdk/dotnet new <template> --name OmniSharpTests_<template> --output /__w/1/s/artifacts/bin/Microsoft.DotNet.SourceBuild.Tests/Release/projects-202512151127508377/OmniSharpTests_<template>
Executing: /__w/1/s/artifacts/bin/Microsoft.DotNet.SourceBuild.Tests/Release/OmniSharpTests/run -s /__w/1/s/artifacts/bin/Microsoft.DotNet.SourceBuild.Tests/Release/projects-202512151127508377/OmniSharpTests_<template>
/__w/1/s/artifacts/bin/Microsoft.DotNet.SourceBuild.Tests/Release/OmniSharpTests/run: line 19: mono: command not found

The underlying OmniSharp runner script fails to execute, but the test suite still reports the result as a success instead of a failure or skip.</issue_description>

<agent_instructions>Two things that need to be done:

  • Update the test so that it produce a failure instead of silently passing
  • Figure out why this is failing in the main branch but not the release/10.0.1xx branch.</agent_instructions>

Comments on the Issue (you are @copilot in this section)

@mthalman This behavior is only occurring in main branch.

💬 We'd love your input! Share your thoughts on Copilot coding agent in our 2 minute survey.

Copilot AI and others added 2 commits December 18, 2025 19:35
Adds checks for "command not found" and "No such file or directory" in stderr to prevent tests from silently passing when the OmniSharp run script fails to execute properly.

Co-authored-by: mthalman <15789599+mthalman@users.noreply.github.com>
Applied code review feedback to make error message detection more robust by using case-insensitive comparison.

Co-authored-by: mthalman <15789599+mthalman@users.noreply.github.com>
Copilot AI changed the title [WIP] Fix silent failure in OmniSharp tests execution Fix OmniSharp tests silently passing on script execution failures Dec 18, 2025
Copilot AI requested a review from mthalman December 18, 2025 19:40
@mthalman mthalman marked this pull request as ready for review January 13, 2026 18:35
@mthalman mthalman requested a review from a team as a code owner January 13, 2026 18:35
Copilot AI review requested due to automatic review settings January 13, 2026 18:35
@mthalman mthalman requested a review from a team as a code owner January 13, 2026 18:35
Copy link
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR fixes OmniSharp tests that were silently passing when script execution failed with errors like "mono: command not found". The test was designed to expect a non-zero exit code from killing the OmniSharp process after a timeout, but script execution failures also produce non-zero exit codes, leading to false positives.

Changes:

  • Changed OmniSharp execution from shell script to direct .NET execution via dotnet OmniSharp.dll
  • Added assertions to detect shell execution failures in stderr (case-insensitive checks for "command not found" and "No such file or directory")
  • Updated OmniSharp tarball download to use the net6.0 variant instead of the mono-based version
  • Added omnisharp.json configuration file to enable prerelease .NET SDK support

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

@mthalman mthalman merged commit adc54b2 into main Jan 13, 2026
16 of 17 checks passed
@mthalman mthalman deleted the copilot/fix-omnisharp-tests-failure branch January 13, 2026 20:13
@mthalman
Copy link
Member

/backport to release/10.0.1xx

@github-actions
Copy link
Contributor

Started backporting to release/10.0.1xx (link to workflow run)

@mthalman
Copy link
Member

/backport to release/10.0.2xx

@github-actions
Copy link
Contributor

Started backporting to release/10.0.2xx (link to workflow run)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

OmniSharp tests have been failing silently

4 participants