Overview

Create a Harmful Content Detector Plugin that identifies and blocks harmful, offensive, or inappropriate content in tool outputs and resource content.

Plugin Requirements

Plugin Details

• Name: HarmfulContentDetectorPlugin
• Type: Self-contained (native) plugin
• File Location: plugins/harmful_content_detector/
• Complexity: High

Functionality

• Detect harmful, offensive, or inappropriate content
• Support multiple content categories and severity levels
• Configurable detection thresholds
• Context-aware analysis
• Multi-language support

Hook Integration

• Primary Hooks: tool_post_invoke, prompt_post_fetch, resource_post_fetch
• Purpose: Detect and filter harmful content
• Behavior: Block or warn on harmful content based on configuration

Acceptance Criteria

• Plugin implements HarmfulContentDetectorPlugin class
• Content categorization and scoring
• Configurable severity thresholds
• Multi-language detection support
• Context-aware analysis
• Plugin manifest and documentation created
• Unit tests with >90% coverage

Priority

High - Safety feature

Dependencies

• Content analysis libraries
• Natural language processing utilities

Security Considerations

• Privacy-preserving content analysis
• Secure handling of sensitive content
• Audit logging for detection events