Skip to content

test: unblock post-25957 shared CI#26048

Merged
ethernet8023 merged 2 commits into
NousResearch:mainfrom
stephenschoettler:fix/discord-e2e-history-mock
May 15, 2026
Merged

test: unblock post-25957 shared CI#26048
ethernet8023 merged 2 commits into
NousResearch:mainfrom
stephenschoettler:fix/discord-e2e-history-mock

Conversation

@stephenschoettler

@stephenschoettler stephenschoettler commented May 15, 2026

Copy link
Copy Markdown
Contributor

What does this PR do?

Fixes the remaining shared CI failures after #25957.

There are two small test-isolation fixes here:

  1. Discord e2e mock surface: history backfill now catches discord.Forbidden and uses discord.MessageType / discord.Object, but the e2e mock still exposed those as generic MagicMock attributes. That made the expected fallback path crash with TypeError: catching classes that do not inherit from BaseException is not allowed.
  2. Nous provider parity helper: a few provider-parity tests instantiate the Nous Portal path without passing a model. After the minimum-context guard, that empty model resolves to a 15K fallback context and fails before the tests reach the provider-format assertions. The helper now uses gpt-5 for Nous parity tests, matching what those tests assert against later anyway.

Related Issue

Follow-up to #25957 and latest main Tests run 25895188143. Does not close a separate issue.

Type of Change

  • 🐛 Bug fix (non-breaking change that fixes an issue)
  • ✨ New feature (non-breaking change that adds functionality)
  • 🔒 Security fix
  • 📝 Documentation update
  • ✅ Tests (adding or improving test coverage)
  • ♻️ Refactor (no behavior change)
  • 🎯 New skill (bundled or hub)

Changes Made

  • tests/e2e/conftest.py
    • Adds discord.Forbidden as an exception class in the e2e Discord mock.
    • Adds deterministic discord.MessageType.default / reply values for history-context filtering.
    • Adds a minimal discord.Object(id=...) mock for cached-history after= support.
  • tests/run_agent/test_provider_parity.py
    • Makes the shared test helper pass gpt-5 for Nous provider tests when a model is not provided explicitly.

How to Test

  1. Reproduce the failed latest-main e2e nodes from run 25895188143.
  2. Run the failed provider-parity nodes from PR CI.
  3. Run the focused Discord e2e file.
  4. Run the full e2e suite plus provider-parity file.

Validation run locally with scripts/run_tests.sh:

./scripts/run_tests.sh -n 0 tests/e2e/test_discord_adapter.py -q --tb=short
# 6 passed

./scripts/run_tests.sh -n 0 \
  tests/run_agent/test_provider_parity.py::TestDeveloperRoleSwap::test_developer_role_via_nous_portal \
  tests/run_agent/test_provider_parity.py::TestBuildApiKwargsNousPortal::test_includes_nous_product_tags \
  tests/run_agent/test_provider_parity.py::TestBuildApiKwargsNousPortal::test_uses_chat_completions_format \
  tests/e2e/test_discord_adapter.py \
  -q --tb=short
# 9 passed

./scripts/run_tests.sh -n 0 tests/run_agent/test_provider_parity.py tests/e2e -q --tb=short
# 149 passed, 7 skipped

git diff --check origin/main...HEAD
/home/w0lf/hermes-agent/venv/bin/python -m compileall -q tests/e2e/conftest.py tests/run_agent/test_provider_parity.py
/home/w0lf/hermes-agent/venv/bin/python -m ruff check tests/e2e/conftest.py tests/run_agent/test_provider_parity.py
# All checks passed

Validation Status

  • Reproduced current-main e2e failure: 4 failed, 2 passed in tests/e2e/test_discord_adapter.py.
  • Focused Discord e2e after fix: 6 passed in 2.65s.
  • Newly failed provider-parity nodes plus Discord e2e after fix: 9 passed in 5.03s.
  • Full e2e suite plus provider parity after fix: 149 passed, 7 skipped in 74.56s.
  • git diff --check origin/main...HEAD passed.
  • python -m compileall -q tests/e2e/conftest.py tests/run_agent/test_provider_parity.py passed.
  • python -m ruff check tests/e2e/conftest.py tests/run_agent/test_provider_parity.py passed.
  • Full pytest tests/ -q is not claimed here. This is intentionally a narrow CI unblocker.

Checklist

Code

  • I've read the Contributing Guide
  • My commit messages follow Conventional Commits (fix(scope):, feat(scope):, etc.)
  • I searched for existing PRs to make sure this isn't a duplicate
  • My PR contains only changes related to this fix/feature (no unrelated commits)
  • I've run pytest tests/ -q and all tests pass
  • I've added tests for my changes (required for bug fixes, strongly encouraged for features)
  • I've tested on my platform: Arch Linux, Python 3.11 venv via scripts/run_tests.sh

Documentation & Housekeeping

  • I've updated relevant documentation (README, docs/, docstrings) — N/A, test fixture/helper only
  • I've updated cli-config.yaml.example if I added/changed config keys — N/A
  • I've updated CONTRIBUTING.md or AGENTS.md if I changed architecture or workflows — N/A
  • I've considered cross-platform impact (Windows, macOS) per the compatibility guide — test-only changes, no runtime platform impact
  • I've updated tool descriptions/schemas if I changed tool behavior — N/A

For New Skills

N/A

Screenshots / Logs

FAILED tests/e2e/test_discord_adapter.py::TestMentionStrippedCommandDispatch::test_mention_then_command - TypeError: catching classes that do not inherit from BaseException is not allowed
FAILED tests/e2e/test_discord_adapter.py::TestMentionStrippedCommandDispatch::test_nickname_mention_then_command - TypeError: catching classes that do not inherit from BaseException is not allowed
FAILED tests/e2e/test_discord_adapter.py::TestAutoThreadingPreservesCommand::test_command_detected_after_auto_thread - TypeError: catching classes that do not inherit from BaseException is not allowed
FAILED tests/e2e/test_discord_adapter.py::TestMentionStrippedCommandDispatch::test_text_before_command_not_detected - TypeError: catching classes that do not inherit from BaseException is not allowed

FAILED tests/run_agent/test_provider_parity.py::TestDeveloperRoleSwap::test_developer_role_via_nous_portal - ValueError: Model  has a context window of 15,000 tokens
FAILED tests/run_agent/test_provider_parity.py::TestBuildApiKwargsNousPortal::test_includes_nous_product_tags - ValueError: Model  has a context window of 15,000 tokens
FAILED tests/run_agent/test_provider_parity.py::TestBuildApiKwargsNousPortal::test_uses_chat_completions_format - ValueError: Model  has a context window of 15,000 tokens

149 passed, 7 skipped in 74.56s
All checks passed!

@alt-glitch alt-glitch added P3 Low — cosmetic, nice to have type/test Test coverage or test infrastructure platform/discord Discord bot adapter labels May 15, 2026
@stephenschoettler stephenschoettler changed the title test(e2e): fix Discord mock exception surface test(ci): fix Discord e2e and Nous provider parity May 15, 2026
@stephenschoettler stephenschoettler changed the title test(ci): fix Discord e2e and Nous provider parity test: unblock post-25957 shared CI May 15, 2026

@ethernet8023 ethernet8023 left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm, ship it. narrow scope, no surprises.
gpt-5 change makes sense, the nous portal tests assert against gpt-5 downstream anyway

also, not a blocker, just noting for whoever picks up future cleanup in this file: tests/run_agent/test_provider_parity.py already has three orphan lines on main immediately below your new elif block:

nice n tight. ty

@ethernet8023 ethernet8023 merged commit 9e67c8e into NousResearch:main May 15, 2026
14 checks passed
gweeteve pushed a commit to gweeteve/hermes-agent that referenced this pull request Jun 2, 2026
…ord-e2e-history-mock

test: unblock post-25957 shared CI
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

P3 Low — cosmetic, nice to have platform/discord Discord bot adapter type/test Test coverage or test infrastructure

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants