|
| 1 | +# Prometheus Metrics Configuration Guide |
| 2 | + |
| 3 | +This guide explains how to configure Prometheus metrics in GOModel. |
| 4 | + |
| 5 | +## Quick Start |
| 6 | + |
| 7 | +### Disabled by Default |
| 8 | + |
| 9 | +Metrics are **disabled by default**. To enable metrics collection, set `METRICS_ENABLED=true` and start GOModel: |
| 10 | + |
| 11 | +```bash |
| 12 | +export METRICS_ENABLED=true |
| 13 | +./bin/gomodel |
| 14 | +# Metrics available at http://localhost:8080/metrics |
| 15 | +``` |
| 16 | + |
| 17 | +### Disable Metrics |
| 18 | + |
| 19 | +**Option 1: Environment Variable** |
| 20 | + |
| 21 | +```bash |
| 22 | +export METRICS_ENABLED=false |
| 23 | +./bin/gomodel |
| 24 | +``` |
| 25 | + |
| 26 | +**Option 2: .env file** |
| 27 | + |
| 28 | +```bash |
| 29 | +echo "METRICS_ENABLED=false" >> .env |
| 30 | +./bin/gomodel |
| 31 | +``` |
| 32 | + |
| 33 | +**Option 3: config.yaml** |
| 34 | + |
| 35 | +```yaml |
| 36 | +metrics: |
| 37 | + enabled: false |
| 38 | +``` |
| 39 | +
|
| 40 | +### Custom Metrics Endpoint |
| 41 | +
|
| 42 | +Change the default `/metrics` path: |
| 43 | + |
| 44 | +```bash |
| 45 | +export METRICS_ENDPOINT=/internal/prometheus |
| 46 | +./bin/gomodel |
| 47 | +``` |
| 48 | + |
| 49 | +## Configuration Options |
| 50 | + |
| 51 | +### Via Environment Variables |
| 52 | + |
| 53 | +| Variable | Default | Description | |
| 54 | +| ------------------ | ---------- | --------------------------------- | |
| 55 | +| `METRICS_ENABLED` | `false` | Enable/disable metrics collection | |
| 56 | +| `METRICS_ENDPOINT` | `/metrics` | HTTP path for metrics endpoint | |
| 57 | + |
| 58 | +### Via config.yaml |
| 59 | + |
| 60 | +```yaml |
| 61 | +metrics: |
| 62 | + # Enable or disable Prometheus metrics collection |
| 63 | + # When disabled, no metrics are collected and endpoint returns 404 |
| 64 | + enabled: true |
| 65 | +
|
| 66 | + # HTTP endpoint path where metrics are exposed |
| 67 | + endpoint: "/metrics" |
| 68 | +``` |
| 69 | + |
| 70 | +## Examples |
| 71 | + |
| 72 | +### Production Setup (Metrics Enabled) |
| 73 | + |
| 74 | +**.env** |
| 75 | + |
| 76 | +```bash |
| 77 | +PORT=8080 |
| 78 | +GOMODEL_MASTER_KEY=your-secret-key |
| 79 | +METRICS_ENABLED=true |
| 80 | +METRICS_ENDPOINT=/metrics |
| 81 | +OPENAI_API_KEY=sk-... |
| 82 | +``` |
| 83 | + |
| 84 | +### Development Setup (Metrics Disabled) |
| 85 | + |
| 86 | +**.env** |
| 87 | + |
| 88 | +```bash |
| 89 | +PORT=8080 |
| 90 | +METRICS_ENABLED=false |
| 91 | +OPENAI_API_KEY=sk-... |
| 92 | +``` |
| 93 | + |
| 94 | +### Custom Endpoint for Internal Monitoring |
| 95 | + |
| 96 | +**config.yaml** |
| 97 | + |
| 98 | +```yaml |
| 99 | +server: |
| 100 | + port: "8080" |
| 101 | + master_key: "${GOMODEL_MASTER_KEY}" |
| 102 | +
|
| 103 | +metrics: |
| 104 | + enabled: true |
| 105 | + endpoint: "/internal/prometheus" # Custom path |
| 106 | +
|
| 107 | +providers: |
| 108 | + openai-primary: |
| 109 | + type: "openai" |
| 110 | + api_key: "${OPENAI_API_KEY}" |
| 111 | +``` |
| 112 | + |
| 113 | +## Verification |
| 114 | + |
| 115 | +### Check if Metrics are Enabled |
| 116 | + |
| 117 | +Start the server and look for log messages: |
| 118 | + |
| 119 | +**Metrics Enabled:** |
| 120 | + |
| 121 | +```json |
| 122 | +{ "level": "INFO", "msg": "prometheus metrics enabled", "endpoint": "/metrics" } |
| 123 | +``` |
| 124 | + |
| 125 | +**Metrics Disabled:** |
| 126 | + |
| 127 | +```json |
| 128 | +{ "level": "INFO", "msg": "prometheus metrics disabled" } |
| 129 | +``` |
| 130 | + |
| 131 | +### Test Metrics Endpoint |
| 132 | + |
| 133 | +**When Enabled:** |
| 134 | + |
| 135 | +```bash |
| 136 | +curl http://localhost:8080/metrics |
| 137 | +# Returns Prometheus metrics in text format |
| 138 | +``` |
| 139 | + |
| 140 | +**When Disabled:** |
| 141 | + |
| 142 | +```bash |
| 143 | +curl http://localhost:8080/metrics |
| 144 | +# Returns 404 Not Found |
| 145 | +``` |
| 146 | + |
| 147 | +## Performance Impact |
| 148 | + |
| 149 | +### Metrics Enabled |
| 150 | + |
| 151 | +- Minimal overhead: ~100ns per request for hook execution |
| 152 | +- Memory: ~1MB for metric storage (depends on cardinality) |
| 153 | +- CPU: Negligible impact (<0.1% in benchmarks) |
| 154 | + |
| 155 | +### Metrics Disabled |
| 156 | + |
| 157 | +- **Zero overhead**: No hooks registered, no collection |
| 158 | +- Metrics library is still linked but inactive |
| 159 | +- Recommended for maximum performance in non-production environments |
| 160 | + |
| 161 | +## Security Considerations |
| 162 | + |
| 163 | +### Exposing Metrics Endpoint |
| 164 | + |
| 165 | +The `/metrics` endpoint is protected by the master key authentication when a master key is configured, just like other HTTP endpoints. If no master key is configured, the endpoint is accessible without authentication, which allows Prometheus to scrape metrics without credentials. |
| 166 | + |
| 167 | +If you need to protect the metrics endpoint further: |
| 168 | + |
| 169 | +1. **Use a custom internal path:** |
| 170 | + |
| 171 | + ```yaml |
| 172 | + metrics: |
| 173 | + endpoint: "/internal/prometheus" # Harder to guess |
| 174 | + ``` |
| 175 | + |
| 176 | +2. **Use network-level security:** |
| 177 | + |
| 178 | + - Configure firewall rules to allow only Prometheus server |
| 179 | + - Use private network for metrics collection |
| 180 | + - Deploy Prometheus in the same VPC/network |
| 181 | + |
| 182 | +3. **Reverse proxy with authentication:** |
| 183 | + ```nginx |
| 184 | + location /metrics { |
| 185 | + auth_basic "Metrics"; |
| 186 | + auth_basic_user_file /etc/nginx/.htpasswd; |
| 187 | + proxy_pass http://gomodel:8080/metrics; |
| 188 | + } |
| 189 | + ``` |
| 190 | + |
| 191 | +## Prometheus Configuration |
| 192 | + |
| 193 | +### Scrape Config |
| 194 | + |
| 195 | +**prometheus.yml** |
| 196 | + |
| 197 | +```yaml |
| 198 | +scrape_configs: |
| 199 | + - job_name: "gomodel" |
| 200 | + static_configs: |
| 201 | + - targets: ["localhost:8080"] |
| 202 | + metrics_path: "/metrics" # Or your custom path |
| 203 | + scrape_interval: 15s |
| 204 | + scrape_timeout: 10s |
| 205 | +``` |
| 206 | + |
| 207 | +### With Custom Endpoint |
| 208 | + |
| 209 | +```yaml |
| 210 | +scrape_configs: |
| 211 | + - job_name: "gomodel" |
| 212 | + static_configs: |
| 213 | + - targets: ["localhost:8080"] |
| 214 | + metrics_path: "/internal/prometheus" # Custom path |
| 215 | + scrape_interval: 15s |
| 216 | +``` |
| 217 | + |
| 218 | +## Troubleshooting |
| 219 | + |
| 220 | +### Metrics Endpoint Returns 404 |
| 221 | + |
| 222 | +**Cause:** Metrics are disabled |
| 223 | + |
| 224 | +**Solution:** |
| 225 | + |
| 226 | +```bash |
| 227 | +# Check configuration |
| 228 | +echo $METRICS_ENABLED # Should be "true" or empty (defaults to true) |
| 229 | +
|
| 230 | +# Enable metrics |
| 231 | +export METRICS_ENABLED=true |
| 232 | +./bin/gomodel |
| 233 | +``` |
| 234 | + |
| 235 | +### No Metrics Data Appearing |
| 236 | + |
| 237 | +**Cause:** No requests have been made yet |
| 238 | + |
| 239 | +**Solution:** Make some requests to generate metrics: |
| 240 | + |
| 241 | +```bash |
| 242 | +curl -X POST http://localhost:8080/v1/chat/completions \ |
| 243 | + -H "Content-Type: application/json" \ |
| 244 | + -H "Authorization: Bearer your-master-key" \ |
| 245 | + -d '{"model": "gpt-4", "messages": [{"role": "user", "content": "Hi"}]}' |
| 246 | +
|
| 247 | +# Then check metrics |
| 248 | +curl http://localhost:8080/metrics | grep gomodel_requests_total |
| 249 | +``` |
| 250 | + |
| 251 | +### Custom Endpoint Not Working |
| 252 | + |
| 253 | +**Cause:** Endpoint must start with `/` |
| 254 | + |
| 255 | +**Incorrect:** |
| 256 | + |
| 257 | +```bash |
| 258 | +export METRICS_ENDPOINT=metrics # Missing leading slash |
| 259 | +``` |
| 260 | + |
| 261 | +**Correct:** |
| 262 | + |
| 263 | +```bash |
| 264 | +export METRICS_ENDPOINT=/metrics # Has leading slash |
| 265 | +``` |
| 266 | + |
| 267 | +## Best Practices |
| 268 | + |
| 269 | +### Development |
| 270 | + |
| 271 | +- **Disable metrics** for faster startup and reduced noise |
| 272 | +- Enable only when testing observability features |
| 273 | + |
| 274 | +### Staging |
| 275 | + |
| 276 | +- **Enable metrics** to test monitoring setup |
| 277 | +- Use custom endpoint if needed for security |
| 278 | + |
| 279 | +### Production |
| 280 | + |
| 281 | +- **Enable metrics** for full observability |
| 282 | +- Set up Prometheus alerting |
| 283 | +- Use Grafana dashboards for visualization |
| 284 | +- Consider custom endpoint for security |
| 285 | +- Monitor metric cardinality to avoid explosion |
| 286 | + |
| 287 | +## Migration Guide |
| 288 | + |
| 289 | +If you're upgrading from a version without configurable metrics: |
| 290 | + |
| 291 | +### Before (Always Enabled) |
| 292 | + |
| 293 | +```bash |
| 294 | +# Metrics were always enabled at /metrics |
| 295 | +./bin/gomodel |
| 296 | +``` |
| 297 | + |
| 298 | +### After (Configurable, Default Enabled) |
| 299 | + |
| 300 | +```bash |
| 301 | +# No change needed - metrics still enabled by default |
| 302 | +./bin/gomodel |
| 303 | +
|
| 304 | +# But now you can disable if needed |
| 305 | +export METRICS_ENABLED=false |
| 306 | +./bin/gomodel |
| 307 | +``` |
| 308 | + |
| 309 | +## See Also |
| 310 | + |
| 311 | +- [PROMETHEUS_IMPLEMENTATION.md](PROMETHEUS_IMPLEMENTATION.md) - Full implementation details |
| 312 | +- [config/config.yaml](config/config.yaml) - Complete configuration |
| 313 | +- [.env.template](.env.template) - Environment variable template |
0 commit comments