Skip to content

[FIX][PLUGINS]: Allow external plugins to be tested without symbolic link and config hack#3022

Merged
crivetimihai merged 2 commits intoIBM:mainfrom
tedhabeck:performance-test-plugins-external-plugin-fix
Feb 20, 2026
Merged

[FIX][PLUGINS]: Allow external plugins to be tested without symbolic link and config hack#3022
crivetimihai merged 2 commits intoIBM:mainfrom
tedhabeck:performance-test-plugins-external-plugin-fix

Conversation

@tedhabeck
Copy link
Copy Markdown
Collaborator

@tedhabeck tedhabeck commented Feb 18, 2026

🔗 Related Issue

Closes #

📝 Summary

Previously, running performance benchmarks on external plugins required soft linking the external plugin's plugin-manifest.yaml folder to the project root, and adding the plugins spec from resources/plugins/config.yaml to tests/performance/plugins/config.yaml. This process was counter-intuitive and inconsistent with how external plugins are registered.

External plugins can now be registered in tests/performance/plugins/config.yaml the same way they are registered in plugins/config.yaml

E.g. for the LLMGuardPlugin:

  - name: "LLMGuardPlugin"
    kind: "external"
    mode: "enforce"  # Don't fail if the server is unavailable
    priority: 20 # adjust the priority
    mcp:
      proto: STREAMABLEHTTP
      url: http://127.0.0.1:8001/mcp

🏷️ Type of Change

  • Bug fix
  • Feature / Enhancement
  • Documentation
  • Refactor
  • Chore (deps, CI, tooling)
  • Other (describe below)

🧪 Verification

Check Command Status
Lint suite make lint
Unit tests make test
Coverage ≥ 80% make coverage

✅ Checklist

  • Code formatted (make black isort pre-commit)
  • Tests added/updated for changes
  • No secrets or credentials committed

📓 Notes (optional)

Screenshots, design decisions, or additional context.

@tedhabeck tedhabeck requested a review from araujof February 18, 2026 00:44
@tedhabeck tedhabeck marked this pull request as ready for review February 18, 2026 00:45
crivetimihai
crivetimihai previously approved these changes Feb 19, 2026
View reviewed changes
Copy link
Copy Markdown
Member

@crivetimihai crivetimihai left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Makes sense — plugin_config.hooks contains the raw config hook type enums, while plugin_ref.hooks contains the actual loaded hook references. Using the plugin ref ensures external plugins without symlinks can be profiled correctly.

@crivetimihai crivetimihai changed the title fix: allow external plugins to be tested w/o symbolic link and config… [FIX][PLUGINS]: Allow external plugins to be tested without symbolic link and config hack Feb 20, 2026
@crivetimihai crivetimihai added SHOULD P2: Important but not vital; high-value items that are not crucial for the immediate release bug Something isn't working labels Feb 20, 2026
@crivetimihai crivetimihai added this to the Release 1.0.0-GA milestone Feb 20, 2026
tedhabeck and others added 2 commits February 20, 2026 16:47
@tedhabeck @crivetimihai
fix: allow external plugins to be tested w/o symbolic link and config…
59546f6 
… hack

Signed-off-by: habeck <habeck@us.ibm.com>
@crivetimihai
fix: add None check for get_plugin return value in performance profiler
bc56c02 
Signed-off-by: Mihai Criveti <crivetimihai@gmail.com>
@crivetimihai crivetimihai force-pushed the performance-test-plugins-external-plugin-fix branch from 8d11908 to bc56c02 Compare February 20, 2026 16:53
@crivetimihai
Copy link
Copy Markdown
Member

crivetimihai commented Feb 20, 2026

Review Changes

Rebased onto main and made the following changes:

Triage

  • Title: renamed from fix: allow external plugins to be tested w/o symbolic link and config… to [FIX][PLUGINS]: Allow external plugins to be tested without symbolic link and config hack
  • Labels: added SHOULD, bug
  • Milestone: set to Release 1.0.0-GA

Fixes

  • tests/performance/test_plugins_performance.py:230-233: Added a None check for the manager.get_plugin() return value. This method returns Optional[Plugin], so dereferencing .hooks without a guard would raise AttributeError if a configured plugin wasn't found in the registry. Now prints a warning and skips the plugin gracefully.

Notes

  • The core approach is correct: for external plugins, hooks are discovered at runtime during initialize() when connecting to the remote MCP server. The YAML config's hooks field may be empty for external plugins, but after initialization the Plugin object's config has the merged hooks from the remote server. Using manager.get_plugin(name).hooks instead of plugin_config.hooks resolves this.
  • No additional unit tests needed — the change is in a performance profiling script, not production code.
  • All checks passing locally.

@tedhabeck — please review the changes and let me know if anything looks off.

@crivetimihai crivetimihai merged commit 3c0d42c into IBM:main Feb 20, 2026
39 checks passed
vishu-bh pushed a commit that referenced this pull request Feb 24, 2026
@tedhabeck @crivetimihai
[FIX][PLUGINS]: Allow external plugins to be tested without symbolic …
c4f9575 
…link and config hack (#3022)

* fix: allow external plugins to be tested w/o symbolic link and config hack

Signed-off-by: habeck <habeck@us.ibm.com>

* fix: add None check for get_plugin return value in performance profiler

Signed-off-by: Mihai Criveti <crivetimihai@gmail.com>

---------

Signed-off-by: habeck <habeck@us.ibm.com>
Signed-off-by: Mihai Criveti <crivetimihai@gmail.com>
Co-authored-by: Mihai Criveti <crivetimihai@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@crivetimihai crivetimihai crivetimihai left review comments

@araujof araujof Awaiting requested review from araujof

@kevalmahajan kevalmahajan Awaiting requested review from kevalmahajan kevalmahajan is a code owner

@madhav165 madhav165 Awaiting requested review from madhav165 madhav165 is a code owner

Assignees

No one assigned

Labels

bug Something isn't working SHOULD P2: Important but not vital; high-value items that are not crucial for the immediate release

Projects

None yet

Milestone

Release 1.0.0-RC2

Development

Successfully merging this pull request may close these issues.

2 participants

@tedhabeck @crivetimihai