Analysis:

Plugin                            P:post       P:pre      R:post       R:pre      T:post       T:pre
----------------------------------------------------------------------------------------------------
Baseline:
LLMGuardPlugin                  22.748ms   116.705ms           —           —           —           —

After cache update:
LLMGuardPlugin                  23.611ms   130.061ms           —           —           —           —

After scanner threading update:
LLMGuardPlugin                   0.148ms     0.120ms           —           —           —           —

After pickle threading:
LLMGuardPlugin                   0.215ms     0.249ms           —           —           —           —

After eval() replaced with ast and orjson update:
LLMGuardPlugin                   0.153ms     0.139ms           —           —           —           —

Rebase and Review Summary

I've rebased this PR onto main and made the following adjustments:

Changes Made During Rebase

1. Resolved conflicts with merged PRs llmguard: switch from pickle to orjson for cache serialization #2179 and llmguard: replace raw eval() with a safe AST-based evaluator #2180

   • Kept orjson serialization (from llmguard: switch from pickle to orjson for cache serialization #2179) instead of pickle
   • Removed pickle-related optimizations (optimal_dump/optimal_load) as orjson is already fast and doesn't need async threading for serialization
   • Kept the AST-based policy evaluator (from llmguard: replace raw eval() with a safe AST-based evaluator #2180)

2. Security Fix: Fail-closed behavior for scanner errors

   • Original implementation skipped failed scanners, which caused a fail-open security issue
   • When a scanner failed, its result was missing from policy evaluation
   • Policy evaluation returned "Invalid expression" (truthy), causing requests to be allowed
   • Fix: Failed scanners now return is_valid=False with risk_score=1.0, ensuring denial

3. Test fixes

   • Updated tests to use prompt_id instead of deprecated name parameter
   • Added environment variable support for redis host/port in tests

Remaining Questions

1. Thread safety of llm_guard scanners: Are filter scanners guaranteed thread-safe when called concurrently via asyncio.to_thread? If scanners mutate shared state, concurrent requests could race. The parallelization only applies to filters (likely stateless), while sanitizers still run sequentially.

2. Regression test for fail-closed path: Should we add a test that injects a failing scanner and asserts denial to prevent reintroducing fail-open behavior?

Files Changed

• cache.py - Async redis with orjson (not pickle)
• llmguard.py - Parallel scanners with fail-closed error handling
• plugin.py - Added await calls
• tests/test_llmguardplugin.py - Fixed payload parameters