Skip to content

[Tests] CLI command tests — Hardcoded English message assertions break on any Strings.resx edit #744

@Christophe-Rogiers

Description

@Christophe-Rogiers

Severity: Info

Files (samples — pattern is repo-wide in CLI tests):

  • tests/Servy.CLI.UnitTests/Commands/InstallServiceCommandTests.cs line 53, 139, 174
  • tests/Servy.CLI.UnitTests/Commands/ExportServiceCommandTests.cs
  • tests/Servy.CLI.UnitTests/Commands/ImportServiceCommandTests.cs

Description:
The CLI command tests assert on the English text of user-facing messages, not on their resource key, result code, or exception type:

Assert.Equal("Service 'TestService' installed successfully.", result.Message);
Assert.Contains("Access Denied", result.Message);
Assert.Contains("Failed to install service 'TestService'", result.Message);

The messages are defined in each project's Resources/Strings.resx and flow through Strings.Msg_*. If any of the following happens, every one of these assertions breaks even though the SUT is behaving correctly:

  • A non-English culture is added (any .resx satellite — de, fr, nl, etc.).
  • The English wording is retuned (e.g., "installed successfully" → "installation succeeded").
  • A reviewer changes a punctuation mark in a Strings entry.

This is a chronic brittleness tax on every Strings edit, and it also discourages future localisation work because the CI fallout is "obvious" only after the PR is proposed.

Suggested fix:
Assert on structured fields that survive localisation:

  • Assert.True(result.Success) / Assert.False(result.Success) / Assert.Equal(ExitCode.Xxx, result.ExitCode) — outcome, not wording.
  • For error paths, assert on the exception type or a typed ErrorCode enum on result.
  • Where you genuinely want to pin the message (e.g., a user-facing summary), read the expected string from Strings.Msg_XXX in the test itself and compare — then the assertion updates automatically with the resource file and also validates cross-culture by swapping the test's CultureInfo.CurrentUICulture.

A targeted sweep of grep -rnE 'Assert\.(Equal|Contains)\("[A-Z]' tests/ will surface the majority of offenders.

Metadata

Metadata

Assignees

Labels

testsChanges to test code and coverage

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions