Continuously Monitor the Business Appropriateness, Risk, and Quality of AI
Citadel AI Inc. is launching Citadel Radar, a firewall and monitoring product that evaluates every prompt and response from your generative AI applications to detect, control, and monitor risk.
As generative AI adoption expands across enterprises, simply checking error rates, availability, and system status, as traditional IT monitoring does, is no longer sufficient for safe AI operations. With generative AI applications, companies need to continuously verify the interactions between the application and its users: whether those interactions are appropriate for business use, whether they carry risk, and whether they deliver the expected value.
Citadel Radar addresses these monitoring challenges unique to generative AI applications, and serves as a firewall and monitoring product for operationalizing cross-functional AI governance. It evaluates prompts and responses against defined assessment metrics, consolidating detection, blocking, flagging, notifications, and analysis into a single platform.
AI Monitoring Requires More than IT Teams
Traditional IT system monitoring has largely been the domain of software engineers and other technical teams. Generative AI monitoring goes further: in addition to traditional metrics like uptime, organizations need to assess whether user interactions are appropriate for business use, whether they carry risk, and whether there are quality issues.
As a result, operating generative AI applications involves not just development teams, but multiple stakeholders across business units, operations, risk management, compliance, and beyond, many of whom are non-technical.
Citadel Radar gives all these stakeholders a single dashboard to manage risk and trends for each application, enabling the organization to maintain continuous visibility into how its generative AI applications are being used. By standardizing how risk is evaluated across teams and roles, through common evaluation metrics and operational logs, it helps build a shared organizational language for generative AI governance and helps accumulate institutional knowledge.
Continuously Detect Risk Based on AI Guidelines
Citadel Radar ships with built-in metrics based on AI safety guidelines and major industry frameworks. This allows organizations to begin monitoring against widely recognized AI risk criteria without having to build their own evaluation standards from the ground up.
For example, it evaluates both prompts and responses for risks that require continuous monitoring, such as disclosure of confidential or personal information, inappropriate or harmful instructions, and language that may reflect bias or discrimination.
You can also create custom metrics tailored to your own business rules, internal policies, industry guidelines, and compliance requirements. Since evaluation criteria can be defined in plain language, your operations and risk management teams can shape the evaluation design directly, not just engineers.
Evaluate, Control, and Visualize AI Interactions
Citadel Radar connects to your generative AI applications via API, receives their prompt and response logs, and evaluates them against your configured metrics. It enables real-time detection and control of risks as they arise in day-to-day operations.
You can choose between two response modes: Block, which prevents a problematic prompt from being processed or a response from being sent to the user, or Flag, which logs the interaction for post-hoc review without disrupting the user experience. This lets you distinguish between risks that must be stopped outright and those that should first be monitored and analyzed, calibrated to each application’s use case and your organization’s policies.
The Traffic page provides a unified view of every application’s prompt and response logs, evaluation results, detected risk categories, and Block/Flag status. Drilling into individual interaction entries reveals per-metric verdicts and metadata, and an AI-powered explanation feature makes it clear which specific elements triggered the result.
Email notifications can be configured based on thresholds, enabling timely notification to relevant stakeholders when detections spike within a given period or when a high-priority risk is identified.
Detection results and risk trends accumulated during operation can also be connected to Citadel Lens, our existing evaluation and testing product, enabling a continuous improvement cycle that links pre-deployment evaluation with post-deployment monitoring.
Key Features of Citadel Radar
- Evaluate prompt and response logs from any generative AI application via API
- Continuously monitor every interaction for business appropriateness, risk, and quality
- Built-in metrics aligned with major AI safety guidelines and industry frameworks
- Custom metrics based on your own business rules, industry guidelines, and compliance requirements
- Flexible Block and Flag controls based on evaluation results
- Evaluation packages to bundle related metrics and operational settings
- Playground to test and verify metric accuracy
- Traffic page showing individual logs, evaluation verdicts, and metadata
- Analytics dashboard for detection trends, Block/Flag counts, and risk distribution
- Integration with Citadel Lens for a continuous loop between pre-deployment testing and live monitoring
- Threshold-based email alerts
- API key management per application
- Role-based and resource-based permissions for managing access across your organization
About Citadel AI
Citadel AI is a Tokyo-based global startup with a mission to build trust in AI. Led by a team of world-class engineers from Google and other leading AI companies, we provide AI governance tools covering pre-deployment risk assessments to real-time guardrails. Our technology is adopted by industry leaders, such as the British Standards Institution, and is trusted by customers globally.
