AI Owasp Middleware

AI-powered Honojs middleware for real-time threat detection, monitoring API endpoint activity, and risks. Uses Vercel AI SDK, worker threads, and ollama for efficient owasp security insights.

Comment

Inspiration

The inspiration for this project comes from a real-world incident on X (formerly Twitter) involving Mark Lou @marc_louvion, a solo entrepreneur who discovered security vulnerabilities in his product only after they were pointed out publicly. Initially skeptical, he eventually hired a security expert to fix the issues. This scenario highlights a common pitfall in web development: security often takes a backseat until it's too late.

For many developers, the typical journey follows this path:

  1. Pick a programming language (e.g., Python, JavaScript).
  2. Specialize in frontend or backend.
  3. Build projects.
  4. Years later, suffer a security breach.

Web and API security are frequently overlooked until they become urgent problems. Our goal is to integrate security best practices seamlessly into the development process, preventing these crises before they occur. More info.

What it does

AI Owasp Middleware acts as a passive security layer between the client and the server. It monitors API interactions in real time, providing developers with security feedback, potential vulnerabilities, and OWASP best practice recommendations. Developers can optionally send this data to an external service for deeper analysis and categorization using AI. This approach enables proactive security measures without disrupting the development workflow.

How we built it

The middleware is built using JavaScript and currently supports the Hono.js framework, which provides efficient API routing strategies. By designing it as a reusable package, developers can integrate it seamlessly into any Hono.js-powered Node.js server.

To prevent LLM API calls from blocking request-response cycles, we leverage Node.js worker_threads, ensuring that expensive AI computations run in a separate thread. This design maintains a smooth developer experience while minimizing performance overhead.

Technologies Used:

  1. Hono.js – Provides the middleware logic.
  2. Vercel AI SDK – Standardized API for accessing various LLMs. AI SDK
  3. Next.js – Powers the log server, allowing security teams to review AI-generated logs.
  4. Convex – A real-time backend service for securely storing AI logs. Convex.dev
  5. Ollama – Enables local AI model execution, reducing reliance on cloud-based LLMs. Ollama

Challenges we ran into

  1. Developer Experience (DX) vs. Performance – While real-time security feedback is valuable, blocking API requests for AI analysis created performance issues. We solved this by offloading AI computations to worker_threads.
  2. Customizability – Developers needed control over which data was sent to the LLM. We addressed this by allowing users to configure the middleware via closures, passing parameters such as model selection and payload customization.
  3. Data Privacy – API requests often contain sensitive information. We provided a transformRequest function, enabling developers to filter out confidential properties (e.g., API keys, headers, or route sources) before sending data to the LLM.

Accomplishments that we're proud of

  • Seamless AI model swapping – Thanks to Vercel AI SDK, developers can easily switch between different LLMs.
  • Performance optimization – Worker threads keep expensive AI operations separate from core business logic.
  • Unopinionated design – The middleware is flexible and integrates smoothly into any Hono.js setup.
  • Powerful middleware pattern – Facilitates distribution, integration, and enhances developer experience.
  • Log server implementation – Provides a dedicated interface for reviewing AI-generated security insights.

What we learned

Through this project, we gained valuable insights into optimizing AI performance in real-time applications. We learned that:

  • Domain-specific AI models yield better results – Instead of a general-purpose AI trying to analyze everything, focusing it on security-specific tasks improves accuracy.
  • Threading is crucial for real-time AI – Moving expensive operations to separate worker threads enhances both performance and usability.
  • Security must be proactive, not reactive – By embedding AI security monitoring directly into the development workflow, we can prevent vulnerabilities before they become exploits.

What's next for AI Owasp Middleware

  • Stabilizing worker_threads – Addressing existing bugs and optimizing thread management.
  • Implementing worker pools – Queuing AI tasks to prevent bottlenecks and improve efficiency.
  • Expanding framework support – Adding compatibility with other web frameworks beyond Hono.js.
  • Enhanced AI feedback – Improving recommendation accuracy and expanding OWASP rule coverage.

By making security an integral part of the development cycle, AI Owasp Middleware aims to shift the industry’s mindset from reactive patching to proactive protection.

Built With

  • convex
  • honojs
  • node.js
  • ollama
  • vercel-ai-sdk
Share this project:

Updates