Skip to content

Add anti-pattern check for usage of have_admin_callout (matcher strategy) - deprecated#16003

Closed
andreslucena wants to merge 6 commits intodevelopfrom
test/have_admin_callout_antipattern
Closed

Add anti-pattern check for usage of have_admin_callout (matcher strategy) - deprecated#16003
andreslucena wants to merge 6 commits intodevelopfrom
test/have_admin_callout_antipattern

Conversation

@andreslucena
Copy link
Copy Markdown
Member

@andreslucena andreslucena commented Feb 5, 2026

🎩 What? Why?

While giving an eye to #15994, I found the usage of this antipattern:

  expect(page).to have_admin_callout("successfully")

This is something that is well extended in the codebase, and it actually can also ignore a minor bug in the process: as the spec was copy and pasted, the flash can also be copy pasted and we can be showing a wrong message in the flash (something like "Meeting successfully updated" when it should really be a Proposal).

As this is really extended through the codebase, I think we should introduce a rubocop or a failure in the CI for this. I think just adding the failure in the matcher is the cleanest way of doing it.

📌 Related Issues

Testing

  1. (Before) Do a search for this string in the codebase
grep -r 'have_admin_callout."successfully' --include="*.rb" decidim-*/ | wc -l
206
  1. Apply the patch
  2. (After) Do the search and see it on 0

♥️ Thank you!

Summary by CodeRabbit

  • Tests

    • Added comprehensive tests for admin callout validation covering antipatterns, length rules, edge cases, and helpful failure messages.
  • Chores

    • Replaced the previous admin-callout test helper with a stricter validation matcher that enforces clearer admin notification wording and provides actionable guidance when messages are too short or ambiguous.

Copilot AI review requested due to automatic review settings February 5, 2026 11:38
@andreslucena andreslucena added the type: internal PRs that aren't necessary to add to the CHANGELOG for implementers label Feb 5, 2026
@andreslucena
Copy link
Copy Markdown
Member Author

Doing a quick run to see all the failures first

@coderabbitai
Copy link
Copy Markdown
Contributor

coderabbitai bot commented Feb 5, 2026

📝 Walkthrough

Walkthrough

Adds a new RSpec matcher have_admin_callout with antipattern and minimum-length validation for admin flash messages, and removes the previous have_admin_callout helper from component test helpers. Includes tests covering validation, edge cases, and helpful error hints.

Changes

Cohort / File(s) Summary
New Matcher Implementation
decidim-dev/lib/decidim/dev/test/rspec_support/admin_callout_matchers.rb
Adds have_admin_callout(expected) matcher; defines MIN_LENGTH_THRESHOLD = 12 and SINGLE_WORD_ANTI_PATTERNS list; validates texts (rejects short texts and single-word antipatterns), raises descriptive antipattern errors, and provides grep search hints.
Helper Removal
decidim-dev/lib/decidim/dev/test/rspec_support/helpers.rb
Removes the old have_admin_callout(text) method from Decidim::ComponentTestHelpers.
Matcher Tests
decidim-dev/spec/lib/decidim/dev/admin_callout_matchers_spec.rb
Adds comprehensive specs for the matcher: antipattern detection, minimum-length boundary tests, nil/symbol handling, example replacements, and grep-based hint presence in error messages.

Sequence Diagram(s)

sequenceDiagram
  participant Spec as RSpec Example
  participant Matcher as have_admin_callout matcher
  participant Flash as Admin Flash Messages (DOM)
  Spec->>Matcher: invoke have_admin_callout(expected)
  Matcher->>Matcher: validate_text(expected) (nil/symbol/length/anti-patterns)
  alt validation passes
    Matcher->>Flash: within_flash_messages { find expected text }
    Flash-->>Matcher: match found
    Matcher-->>Spec: pass
  else validation fails
    Matcher-->>Spec: raise AntipatternError with grep hint
  end
Loading

Estimated code review effort

🎯 3 (Moderate) | ⏱️ ~25 minutes

Poem

🐰 I hopped through specs with a keen little eye,
Banishing "Done" and short phrases that fly.
I whisper a grep to help find the spot,
And nudge callouts to say something hot.
Tests green, I munch carrots — nice work, let's not stop! ✨

🚥 Pre-merge checks | ✅ 2 | ❌ 1
❌ Failed checks (1 warning)
Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 0.00% which is insufficient. The required threshold is 80.00%. Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (2 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title accurately summarizes the main change: adding an anti-pattern check to the have_admin_callout matcher to prevent misuse.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing touches
  • 📝 Generate docstrings
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Post copyable unit tests in a comment
  • Commit unit tests in branch test/have_admin_callout_antipattern

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

github-actions[bot]
github-actions bot previously approved these changes Feb 5, 2026
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR introduces antipattern detection for the have_admin_callout matcher to prevent developers from using generic, vague flash message assertions (like have_admin_callout("successfully")) that don't verify the actual feedback shown to users. The validation enforces a minimum message length of 12 characters and explicitly blocks common single-word antipatterns.

Changes:

  • Replaces the simple have_admin_callout helper method with a full RSpec custom matcher that validates message quality
  • Adds validation rules to detect and reject generic single-word messages and very short strings
  • Includes comprehensive test coverage for the antipattern detection logic

Reviewed changes

Copilot reviewed 3 out of 3 changed files in this pull request and generated 6 comments.

File Description
decidim-dev/lib/decidim/dev/test/rspec_support/admin_callout_matchers.rb New file implementing the have_admin_callout custom RSpec matcher with antipattern detection logic
decidim-dev/spec/lib/decidim/dev/admin_callout_matchers_spec.rb New test file verifying antipattern detection for various invalid inputs
decidim-dev/lib/decidim/dev/test/rspec_support/helpers.rb Removes the old simple have_admin_callout method (6 lines) that is being replaced by the new matcher

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment on lines +5 to +103
MIN_LENGTH_THRESHOLD = 12

SINGLE_WORD_ANTIPATTERNS = %w(
successfully
successfully.
problem
error
warning
done
complete
finished
ok
okay
saved
updated
created
deleted
removed
published
unpublished
).freeze

RSpec::Matchers.define :have_admin_callout do |expected|
include Decidim::ComponentTestHelpers

match do |_page|
# Short-circuit for nil/symbols to avoid calling within_flash_messages
return true if expected.nil? || expected.is_a?(Symbol)

validate_text(expected)

within_flash_messages do
have_content expected
end
end

def validate_text(text)
return true if text.nil? || text.is_a?(Symbol)

stripped_text = text.to_s.gsub(/[[:punct:]]/, "")
is_single_word = text.to_s.strip !~ /\s/
is_too_short = stripped_text.length < MIN_LENGTH_THRESHOLD

if is_single_word && is_too_short
raise <<~ERROR
Antipattern detected: Using generic single-word text "#{text}" in admin callout assertions.

Problem: Generic single-word messages don't verify the actual feedback shown to users.

#{format_find_command(text)}
ERROR
end

if is_single_word && SINGLE_WORD_ANTIPATTERNS.include?(stripped_text.downcase)
raise <<~ERROR
Antipattern detected: Using generic single-word text "#{text}" in admin callout assertions.

Problem: Generic single-word messages don't verify the actual feedback shown to users.
Single-word messages like "successfully" or "problem" are too vague.

Example: "Meeting successfully published" is better than just "successfully"

#{format_find_command(text)}
ERROR
end

if is_too_short
raise <<~ERROR
Antipattern detected: Using very short text "#{text}" (#{stripped_text.length} chars) in admin callout assertions.

Problem: Very short messages are likely generic and don't provide meaningful feedback.
Messages should be at least #{MIN_LENGTH_THRESHOLD} characters to convey useful information.

Examples of proper messages:
- "Meeting successfully published"
- "Budget successfully created."
- "There was a problem saving the form"
- "Your changes have been saved successfully"

#{format_find_command(text)}
ERROR
end

true
end

def format_find_command(text)
<<~COMMAND

To find similar instances:
grep -r 'have_admin_callout.*#{text}' --include="*.rb" decidim-*/
COMMAND
end

failure_message do
@failure_message || super()
end

diffable
Copy link

Copilot AI Feb 5, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The constants MIN_LENGTH_THRESHOLD and SINGLE_WORD_ANTIPATTERNS are defined at the top level (global namespace) instead of being scoped within a module or the matcher definition. This can lead to namespace pollution and potential conflicts with other code that might use similar constant names.

Consider moving these constants inside the matcher definition or wrapping them in a module namespace. For example, you could define them as class-level constants within a module, or define them as constants within the matcher block itself.

Suggested change
MIN_LENGTH_THRESHOLD = 12
SINGLE_WORD_ANTIPATTERNS = %w(
successfully
successfully.
problem
error
warning
done
complete
finished
ok
okay
saved
updated
created
deleted
removed
published
unpublished
).freeze
RSpec::Matchers.define :have_admin_callout do |expected|
include Decidim::ComponentTestHelpers
match do |_page|
# Short-circuit for nil/symbols to avoid calling within_flash_messages
return true if expected.nil? || expected.is_a?(Symbol)
validate_text(expected)
within_flash_messages do
have_content expected
end
end
def validate_text(text)
return true if text.nil? || text.is_a?(Symbol)
stripped_text = text.to_s.gsub(/[[:punct:]]/, "")
is_single_word = text.to_s.strip !~ /\s/
is_too_short = stripped_text.length < MIN_LENGTH_THRESHOLD
if is_single_word && is_too_short
raise <<~ERROR
Antipattern detected: Using generic single-word text "#{text}" in admin callout assertions.
Problem: Generic single-word messages don't verify the actual feedback shown to users.
#{format_find_command(text)}
ERROR
end
if is_single_word && SINGLE_WORD_ANTIPATTERNS.include?(stripped_text.downcase)
raise <<~ERROR
Antipattern detected: Using generic single-word text "#{text}" in admin callout assertions.
Problem: Generic single-word messages don't verify the actual feedback shown to users.
Single-word messages like "successfully" or "problem" are too vague.
Example: "Meeting successfully published" is better than just "successfully"
#{format_find_command(text)}
ERROR
end
if is_too_short
raise <<~ERROR
Antipattern detected: Using very short text "#{text}" (#{stripped_text.length} chars) in admin callout assertions.
Problem: Very short messages are likely generic and don't provide meaningful feedback.
Messages should be at least #{MIN_LENGTH_THRESHOLD} characters to convey useful information.
Examples of proper messages:
- "Meeting successfully published"
- "Budget successfully created."
- "There was a problem saving the form"
- "Your changes have been saved successfully"
#{format_find_command(text)}
ERROR
end
true
end
def format_find_command(text)
<<~COMMAND
To find similar instances:
grep -r 'have_admin_callout.*#{text}' --include="*.rb" decidim-*/
COMMAND
end
failure_message do
@failure_message || super()
end
diffable
module Decidim
module Dev
module Test
MIN_LENGTH_THRESHOLD = 12
SINGLE_WORD_ANTIPATTERNS = %w(
successfully
successfully.
problem
error
warning
done
complete
finished
ok
okay
saved
updated
created
deleted
removed
published
unpublished
).freeze
RSpec::Matchers.define :have_admin_callout do |expected|
include Decidim::ComponentTestHelpers
match do |_page|
# Short-circuit for nil/symbols to avoid calling within_flash_messages
return true if expected.nil? || expected.is_a?(Symbol)
validate_text(expected)
within_flash_messages do
have_content expected
end
end
def validate_text(text)
return true if text.nil? || text.is_a?(Symbol)
stripped_text = text.to_s.gsub(/[[:punct:]]/, "")
is_single_word = text.to_s.strip !~ /\s/
is_too_short = stripped_text.length < MIN_LENGTH_THRESHOLD
if is_single_word && is_too_short
raise <<~ERROR
Antipattern detected: Using generic single-word text "#{text}" in admin callout assertions.
Problem: Generic single-word messages don't verify the actual feedback shown to users.
#{format_find_command(text)}
ERROR
end
if is_single_word && SINGLE_WORD_ANTIPATTERNS.include?(stripped_text.downcase)
raise <<~ERROR
Antipattern detected: Using generic single-word text "#{text}" in admin callout assertions.
Problem: Generic single-word messages don't verify the actual feedback shown to users.
Single-word messages like "successfully" or "problem" are too vague.
Example: "Meeting successfully published" is better than just "successfully"
#{format_find_command(text)}
ERROR
end
if is_too_short
raise <<~ERROR
Antipattern detected: Using very short text "#{text}" (#{stripped_text.length} chars) in admin callout assertions.
Problem: Very short messages are likely generic and don't provide meaningful feedback.
Messages should be at least #{MIN_LENGTH_THRESHOLD} characters to convey useful information.
Examples of proper messages:
- "Meeting successfully published"
- "Budget successfully created."
- "There was a problem saving the form"
- "Your changes have been saved successfully"
#{format_find_command(text)}
ERROR
end
true
end
def format_find_command(text)
<<~COMMAND
To find similar instances:
grep -r 'have_admin_callout.*#{text}' --include="*.rb" decidim-*/
COMMAND
end
failure_message do
@failure_message || super()
end
diffable
end
end
end

Copilot uses AI. Check for mistakes.
Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As I mentioned in my comment :

Doing a quick run to see all the failures first

So, this is actually part of the process :)

Comment on lines +36 to +38
within_flash_messages do
have_content expected
end
Copy link

Copilot AI Feb 5, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The match block is not correctly evaluating whether the content exists in the flash messages. The code calls have_content expected within the within_flash_messages block, but this just returns a matcher object without evaluating it. Since matcher objects are always truthy, this will always return true (assuming validation passes).

The match block should actually check if the content exists. Consider using one of these approaches:

  1. Use Capybara's predicate methods: page.has_content?(expected)
  2. Delegate to Capybara's built-in matcher and return the result of checking the scoped element
  3. Use an expectation and return true/false based on whether it passes

For example, you could capture the result within the within_flash_messages block by checking if the current scope has the content.

Suggested change
within_flash_messages do
have_content expected
end
result = nil
within_flash_messages do
result = page.has_content?(expected)
end
result

Copilot uses AI. Check for mistakes.
Comment on lines +27 to +104
RSpec::Matchers.define :have_admin_callout do |expected|
include Decidim::ComponentTestHelpers

match do |_page|
# Short-circuit for nil/symbols to avoid calling within_flash_messages
return true if expected.nil? || expected.is_a?(Symbol)

validate_text(expected)

within_flash_messages do
have_content expected
end
end

def validate_text(text)
return true if text.nil? || text.is_a?(Symbol)

stripped_text = text.to_s.gsub(/[[:punct:]]/, "")
is_single_word = text.to_s.strip !~ /\s/
is_too_short = stripped_text.length < MIN_LENGTH_THRESHOLD

if is_single_word && is_too_short
raise <<~ERROR
Antipattern detected: Using generic single-word text "#{text}" in admin callout assertions.

Problem: Generic single-word messages don't verify the actual feedback shown to users.

#{format_find_command(text)}
ERROR
end

if is_single_word && SINGLE_WORD_ANTIPATTERNS.include?(stripped_text.downcase)
raise <<~ERROR
Antipattern detected: Using generic single-word text "#{text}" in admin callout assertions.

Problem: Generic single-word messages don't verify the actual feedback shown to users.
Single-word messages like "successfully" or "problem" are too vague.

Example: "Meeting successfully published" is better than just "successfully"

#{format_find_command(text)}
ERROR
end

if is_too_short
raise <<~ERROR
Antipattern detected: Using very short text "#{text}" (#{stripped_text.length} chars) in admin callout assertions.

Problem: Very short messages are likely generic and don't provide meaningful feedback.
Messages should be at least #{MIN_LENGTH_THRESHOLD} characters to convey useful information.

Examples of proper messages:
- "Meeting successfully published"
- "Budget successfully created."
- "There was a problem saving the form"
- "Your changes have been saved successfully"

#{format_find_command(text)}
ERROR
end

true
end

def format_find_command(text)
<<~COMMAND

To find similar instances:
grep -r 'have_admin_callout.*#{text}' --include="*.rb" decidim-*/
COMMAND
end

failure_message do
@failure_message || super()
end

diffable
end
Copy link

Copilot AI Feb 5, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This PR introduces antipattern detection that will cause hundreds of existing tests to fail. A search of the codebase shows 206+ instances of have_admin_callout("successfully") and many more instances of similar antipatterns like have_admin_callout("updated"), have_admin_callout("created"), etc.

The PR description states "Do the search and see it on 0" suggesting all instances should be fixed, but this PR doesn't include any fixes to the affected test files. This will cause a massive test failure across the entire codebase once merged.

This is a breaking change that requires one of the following approaches:

  1. Include fixes for all affected tests in this same PR
  2. Split into two PRs: first fix all the antipattern usages, then add the validation
  3. Add a temporary flag or environment variable to enable/disable the antipattern detection, allowing for gradual migration
  4. Provide a clear migration guide and accept that tests will fail until they're updated

Without addressing this, merging this PR will break the CI/CD pipeline and block development until all affected tests are fixed.

Copilot uses AI. Check for mistakes.
Copy link
Copy Markdown
Contributor

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 3

🤖 Fix all issues with AI agents
In `@decidim-dev/lib/decidim/dev/test/rspec_support/admin_callout_matchers.rb`:
- Around line 7-25: Rename the constant SINGLE_WORD_ANTIPATTERNS to a
lint-friendly name like SINGLE_WORD_ANTI_PATTERNS and update any code that
references it (e.g., in admin_callout_matchers.rb) to use the new constant;
replace any user-facing text that uses “antipattern/Antipattern” with
“anti-pattern” and change contractions like “don’t” to “do not” in the
error/message strings so spelling/lint checks pass; update all specs that assert
on these messages to expect the new wording and adjust any other occurrences in
the same file (including the other constants/messages around the block where
SINGLE_WORD_ANTIPATTERNS was defined) to the new names/strings.
- Around line 30-38: The match block currently returns an unevaluated RSpec
matcher (have_content expected) which always passes; inside the
within_flash_messages block (and keeping validate_text(expected) and the
surrounding match do |_page| intact) replace the matcher construction with a
boolean check such as has_text?(expected) (i.e., call has_text?(expected) inside
within_flash_messages) so the block returns a true/false based on actual
presence of the text rather than a matcher object.

In `@decidim-dev/spec/lib/decidim/dev/admin_callout_matchers_spec.rb`:
- Around line 5-12: Update the spec strings and error-message expectations to
use the spelled form "anti-pattern" instead of "antipattern": search for
occurrences in the RSpec example group description ("antipattern detection"),
context/descriptions, and any regex expectations (e.g., the
raise_error(/Antipattern detected/) assertion) and replace them with
"anti-pattern" or a regex matching /Anti-pattern detected/ so they align with
the matcher have_admin_callout's wording; ensure all other examples in the same
file are updated similarly.

Comment on lines +30 to +38
match do |_page|
# Short-circuit for nil/symbols to avoid calling within_flash_messages
return true if expected.nil? || expected.is_a?(Symbol)

validate_text(expected)

within_flash_messages do
have_content expected
end
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟠 Major

🧩 Analysis chain

🏁 Script executed:

# First, let's read the entire file to understand the context
cat -n decidim-dev/lib/decidim/dev/test/rspec_support/admin_callout_matchers.rb

Repository: decidim/decidim

Length of output: 3456


🏁 Script executed:

# Search for within_flash_messages definition
rg -n "def within_flash_messages" --type ruby

Repository: decidim/decidim

Length of output: 149


🏁 Script executed:

# Search for how within_flash_messages is used elsewhere in the codebase
rg -n "within_flash_messages" --type ruby -A 2 | head -50

Repository: decidim/decidim

Length of output: 3688


🏁 Script executed:

# Search for other RSpec.Matchers.define patterns in the codebase to see how match blocks are typically structured
rg -n "RSpec::Matchers\.define" --type ruby -A 10 | head -100

Repository: decidim/decidim

Length of output: 9119


🏁 Script executed:

# Look for examples of match blocks in custom matchers
rg -n "match do" --type ruby -A 3 | grep -E "(match do|expect|has_|\.call)" | head -50

Repository: decidim/decidim

Length of output: 2511


🏁 Script executed:

# Verify that have_content is an RSpec matcher that needs expect() to be evaluated
rg -n "have_content" decidim-dev/lib/decidim/dev/test/rspec_support/helpers.rb

Repository: decidim/decidim

Length of output: 109


Match block returns unevaluated matcher instead of boolean assertion

The have_content expected statement inside within_flash_messages only builds a matcher object and returns it without evaluating the assertion. RSpec matcher objects are truthy, so the block will always pass regardless of whether the text is actually present. Wrap the assertion with expect() or use a boolean method like has_text?() to properly evaluate the condition.

Proposed fix
  match do |_page|
    # Short-circuit for nil/symbols to avoid calling within_flash_messages
    return true if expected.nil? || expected.is_a?(Symbol)

    validate_text(expected)

    within_flash_messages do
-     have_content expected
+     page.has_text?(expected)
    end
  end
🤖 Prompt for AI Agents
In `@decidim-dev/lib/decidim/dev/test/rspec_support/admin_callout_matchers.rb`
around lines 30 - 38, The match block currently returns an unevaluated RSpec
matcher (have_content expected) which always passes; inside the
within_flash_messages block (and keeping validate_text(expected) and the
surrounding match do |_page| intact) replace the matcher construction with a
boolean check such as has_text?(expected) (i.e., call has_text?(expected) inside
within_flash_messages) so the block returns a true/false based on actual
presence of the text rather than a matcher object.

@andreslucena andreslucena changed the title Add antipattern check for usage of have_admin_callout Add anti-pattern check for usage of have_admin_callout Feb 5, 2026
@andreslucena andreslucena changed the title Add anti-pattern check for usage of have_admin_callout Add anti-pattern check for usage of have_admin_callout (matcher strategy) - deprecated Feb 6, 2026
@andreslucena
Copy link
Copy Markdown
Member Author

Closing this one in favour of #16011 - that seems much cleaner

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

module: dev type: internal PRs that aren't necessary to add to the CHANGELOG for implementers

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants