chore(ai_guard): add support for multimodal prompts#15939
Conversation
Codeowners resolved as |
b02ff15 to
f208cd9
Compare
Performance SLOsComparing candidate malvarez/ai-guard-multimodal (89be681) with baseline main (f81b2d1) 📈 Performance Regressions (2 suites)📈 iastaspectsospath - 24/24✅ ospathbasename_aspectTime: ✅ 504.561µs (SLO: <700.000µs 📉 -27.9%) vs baseline: 📈 +19.0% Memory: ✅ 42.349MB (SLO: <43.500MB -2.6%) vs baseline: +5.0% ✅ ospathbasename_noaspectTime: ✅ 426.577µs (SLO: <700.000µs 📉 -39.1%) vs baseline: -1.4% Memory: ✅ 42.389MB (SLO: <43.500MB -2.6%) vs baseline: +4.7% ✅ ospathjoin_aspectTime: ✅ 623.262µs (SLO: <700.000µs 📉 -11.0%) vs baseline: -0.4% Memory: ✅ 42.330MB (SLO: <43.500MB -2.7%) vs baseline: +5.0% ✅ ospathjoin_noaspectTime: ✅ 626.779µs (SLO: <700.000µs 📉 -10.5%) vs baseline: -0.5% Memory: ✅ 42.251MB (SLO: <43.500MB -2.9%) vs baseline: +4.5% ✅ ospathnormcase_aspectTime: ✅ 351.033µs (SLO: <700.000µs 📉 -49.9%) vs baseline: -1.1% Memory: ✅ 42.428MB (SLO: <43.500MB -2.5%) vs baseline: +5.1% ✅ ospathnormcase_noaspectTime: ✅ 352.804µs (SLO: <700.000µs 📉 -49.6%) vs baseline: -2.0% Memory: ✅ 42.546MB (SLO: <43.500MB -2.2%) vs baseline: +4.8% ✅ ospathsplit_aspectTime: ✅ 484.949µs (SLO: <700.000µs 📉 -30.7%) vs baseline: -1.0% Memory: ✅ 42.487MB (SLO: <43.500MB -2.3%) vs baseline: +5.1% ✅ ospathsplit_noaspectTime: ✅ 492.574µs (SLO: <700.000µs 📉 -29.6%) vs baseline: -1.3% Memory: ✅ 42.428MB (SLO: <43.500MB -2.5%) vs baseline: +5.0% ✅ ospathsplitdrive_aspectTime: ✅ 369.535µs (SLO: <700.000µs 📉 -47.2%) vs baseline: -1.7% Memory: ✅ 42.271MB (SLO: <43.500MB -2.8%) vs baseline: +4.7% ✅ ospathsplitdrive_noaspectTime: ✅ 73.328µs (SLO: <700.000µs 📉 -89.5%) vs baseline: +0.4% Memory: ✅ 42.389MB (SLO: <43.500MB -2.6%) vs baseline: +5.0% ✅ ospathsplitext_aspectTime: ✅ 455.425µs (SLO: <700.000µs 📉 -34.9%) vs baseline: -1.2% Memory: ✅ 42.271MB (SLO: <43.500MB -2.8%) vs baseline: +4.7% ✅ ospathsplitext_noaspectTime: ✅ 460.799µs (SLO: <700.000µs 📉 -34.2%) vs baseline: -0.7% Memory: ✅ 42.408MB (SLO: <43.500MB -2.5%) vs baseline: +5.2% 📈 telemetryaddmetric - 30/30✅ 1-count-metric-1-timesTime: ✅ 3.387µs (SLO: <20.000µs 📉 -83.1%) vs baseline: 📈 +13.1% Memory: ✅ 34.977MB (SLO: <35.500MB 🟡 -1.5%) vs baseline: +5.2% ✅ 1-count-metrics-100-timesTime: ✅ 200.228µs (SLO: <220.000µs -9.0%) vs baseline: -0.3% Memory: ✅ 34.859MB (SLO: <35.500MB 🟡 -1.8%) vs baseline: +4.9% ✅ 1-distribution-metric-1-timesTime: ✅ 3.348µs (SLO: <20.000µs 📉 -83.3%) vs baseline: +0.4% Memory: ✅ 34.918MB (SLO: <35.500MB 🟡 -1.6%) vs baseline: +4.9% ✅ 1-distribution-metrics-100-timesTime: ✅ 214.697µs (SLO: <230.000µs -6.7%) vs baseline: +0.1% Memory: ✅ 34.937MB (SLO: <35.500MB 🟡 -1.6%) vs baseline: +5.0% ✅ 1-gauge-metric-1-timesTime: ✅ 2.214µs (SLO: <20.000µs 📉 -88.9%) vs baseline: -0.4% Memory: ✅ 34.898MB (SLO: <35.500MB 🟡 -1.7%) vs baseline: +5.0% ✅ 1-gauge-metrics-100-timesTime: ✅ 138.190µs (SLO: <150.000µs -7.9%) vs baseline: -0.5% Memory: ✅ 34.957MB (SLO: <35.500MB 🟡 -1.5%) vs baseline: +5.1% ✅ 1-rate-metric-1-timesTime: ✅ 3.164µs (SLO: <20.000µs 📉 -84.2%) vs baseline: +0.3% Memory: ✅ 34.839MB (SLO: <35.500MB 🟡 -1.9%) vs baseline: +4.9% ✅ 1-rate-metrics-100-timesTime: ✅ 213.298µs (SLO: <250.000µs 📉 -14.7%) vs baseline: +0.2% Memory: ✅ 34.937MB (SLO: <35.500MB 🟡 -1.6%) vs baseline: +5.2% ✅ 100-count-metrics-100-timesTime: ✅ 20.150ms (SLO: <22.000ms -8.4%) vs baseline: -0.3% Memory: ✅ 34.977MB (SLO: <35.500MB 🟡 -1.5%) vs baseline: +5.4% ✅ 100-distribution-metrics-100-timesTime: ✅ 2.215ms (SLO: <2.550ms 📉 -13.1%) vs baseline: -0.9% Memory: ✅ 34.996MB (SLO: <35.500MB 🟡 -1.4%) vs baseline: +5.3% ✅ 100-gauge-metrics-100-timesTime: ✅ 1.428ms (SLO: <1.550ms -7.9%) vs baseline: +0.4% Memory: ✅ 34.839MB (SLO: <35.500MB 🟡 -1.9%) vs baseline: +5.0% ✅ 100-rate-metrics-100-timesTime: ✅ 2.198ms (SLO: <2.550ms 📉 -13.8%) vs baseline: -0.5% Memory: ✅ 34.957MB (SLO: <35.500MB 🟡 -1.5%) vs baseline: +5.1% ✅ flush-1-metricTime: ✅ 4.519µs (SLO: <20.000µs 📉 -77.4%) vs baseline: -0.1% Memory: ✅ 35.252MB (SLO: <35.500MB 🟡 -0.7%) vs baseline: +5.0% ✅ flush-100-metricsTime: ✅ 173.573µs (SLO: <250.000µs 📉 -30.6%) vs baseline: -0.2% Memory: ✅ 35.311MB (SLO: <35.500MB 🟡 -0.5%) vs baseline: +5.0% ✅ flush-1000-metricsTime: ✅ 2.183ms (SLO: <2.500ms 📉 -12.7%) vs baseline: +0.2% Memory: ✅ 36.176MB (SLO: <36.500MB 🟡 -0.9%) vs baseline: +5.3% 🟡 Near SLO Breach (16 suites)🟡 coreapiscenario - 10/10 (1 unstable)
|
## Description * Adds support for multimodal prompts in the AI Guard API. * This is done via content parts, mimicking the Chat Completion API. ## Testing <!-- Describe your testing strategy or note what tests are included --> ## Risks <!-- Note any risks associated with this change, or "None" if no risks --> ## Additional Notes * [APPSEC-60266](https://datadoghq.atlassian.net/browse/APPSEC-60266) * [Internal docs](https://datadoghq.atlassian.net/wiki/x/04CxRAE) for content parts. [APPSEC-60266]: https://datadoghq.atlassian.net/browse/APPSEC-60266?atlOrigin=eyJpIjoiNWRkNTljNzYxNjVmNDY3MDlhMDU5Y2ZhYzA5YTRkZjUiLCJwIjoiZ2l0aHViLWNvbS1KU1cifQ
Description
Testing
Risks
Additional Notes