@tedhabeck FYI

Pickle performance:

Plugin                            P:post       P:pre      R:post       R:pre      T:post       T:pre
----------------------------------------------------------------------------------------------------
LLMGuardPlugin                  26.069ms   128.718ms           —           —           —           —

orjson performance:

LLMGuardPlugin                  24.170ms   128.957ms           —           —           —           —

Hi @RinZ27 - thank you for this contribution! I've rebased it, and made some modifications. I think this still needs a discussion around sanitization of data, but looking forward to your input on this.

Review Changes Applied

This PR has been rebased onto main and enhanced during review. Here's a summary of the changes made beyond the original commit:

Fixes Applied

Final Diff Summary

• Robust error handling for serialization and deserialization
• PII-safe logging throughout
• Type-correct return values

Trade-offs: orjson vs pickle

Why This Change is Important

Security (Primary Motivation):

• pickle.loads() can execute arbitrary code during deserialization — while this would require write access to Redis (cache poisoning), it's better to use a different approach.
• orjson only deserializes JSON data types — no code execution possible

Comparison

Trade-offs Accepted

1. Type Restrictions: Vault data must be JSON-serializable. Verified that vault tuples are always List[Tuple[str, str, str]] containing only strings — no issue for this use case.

2. Tuple Conversion: JSON doesn't have a tuple type, so List[Tuple] becomes List[List] during serialization. The _convert_to_list_of_tuples() function handles the conversion back.

3. Behavioral Change: If non-JSON-serializable data is ever stored (e.g., bytes, UUID, custom objects), the cache write will now fail gracefully instead of working. This is intentional — it surfaces data shape issues early rather than silently storing potentially problematic data.

Alternatives Considered

• json stdlib: Slower than orjson, same type restrictions
• msgpack: Binary format, faster than JSON, but adds a new dependency
• orjson with default handler: Could serialize more types, but adds complexity and may mask data issues

Conclusion

The security benefits of removing pickle outweigh the minor complexity of tuple conversion. The vault data structure is well-defined (string tuples only), making orjson a clean fit. Error handling ensures graceful degradation if assumptions are ever violated.

@crivetimihai One question that needs to be answered at the plugin level is do we want to throw error/PluginError when cache serialization fails or just log it? Right now, it's logging the serialization error and sending a False, False to the plugin.py.

Rest looks good to me.

Just jumping in with my two cents here... I agree with the current approach (logging the error and returning False, False). Since caching is an optimization/auxiliary mechanism here, a serialization failure shouldn't block the main processing flow or crash the plugin execution. Graceful degradation seems safer for production.

Appreciate the rebase and the extra polish on the function names and types, @crivetimihai. It's definitely looking much cleaner now. Also, thanks @tedhabeck for running those benchmarks—glad to see orjson giving us that slight performance edge too.