feat(api): Added error_sampler option#2456
Conversation
tests/test_client.py
Outdated
| # Baseline test with issues_sampler only, both floats and bools | ||
| (mock.MagicMock(return_value=1.0), None, 1), | ||
| (mock.MagicMock(return_value=0.0), None, 0), | ||
| (mock.MagicMock(return_value=True), None, 1), | ||
| (mock.MagicMock(return_value=False), None, 0), | ||
| # Baseline test with sample_rate only | ||
| (None, 0.0, 0), | ||
| (None, 1.0, 1), | ||
| # issues_sampler takes precedence over sample_rate | ||
| (mock.MagicMock(return_value=1.0), 0.0, 1), | ||
| (mock.MagicMock(return_value=0.0), 1.0, 0), | ||
| ), | ||
| ) |
There was a problem hiding this comment.
We should also test different rates for different errors. Like ZeroDivisionError: 1.0, RuntimeError: 0.5 and such.
| @@ -456,10 +456,11 @@ def _should_sample_error( | |||
| event, # type: Event | |||
There was a problem hiding this comment.
event contains information about the logged message via logging?
There is a hint parameter in the before_send that contains this information in log_record
There was a problem hiding this comment.
@saippuakauppias Could you please clarify your question? How exactly are you logging the message?
There was a problem hiding this comment.
If you are using the LoggingIntegration, your log message should appear in the event passed to this function.
There was a problem hiding this comment.
Yes, I am using LoggingIntegration and logging all warning messages to sentry. I tried to discard some messages that are not needed and in debug mode found that the path to the file that triggers the message is in hint['log_record'].pathname.
Actually, I need this as a minimum. I would like this feature to have a similar option to filter by module name.
There was a problem hiding this comment.
@saippuakauppias we decided to also pass the hint into the events_sampler, so you will be able to make your sampling decision based on information contained in the hint.
|
@szokeasaurusrex Is it only errors passing through If it's only errors, calling the option More importantly, are the events from the original issue errors (as opposed to e.g. messages) and would they pass through the |
|
An issue in Sentry represents N events, so calling it |
Co-authored-by: Ivana Kellyerova <ivana.kellyerova@sentry.io>
The
I am unsure given the information I see in the original issue. @saippuakauppias, could you please provide the code where these errors/messages originate from, or provide us more detail with how these are being generated, so we can answer this question? |
Yes, looks like it's everything but transactions and checkins: sentry-python/sentry_sdk/client.py Lines 556 to 560 in 4d10edf In that case
@cleptric Checkins and transactions also end up in |
|
Once we've settled on the API please don't forget to update the docs @szokeasaurusrex 📖 |
|
I discussed with @cleptric, and we now think we should pick a different name than The best idea we came up with was to just call the new sampler |
I'm using django, so it's going to be a bit of a specific example here, but I think it's reproducible. # settings/production.py
LOGGING = {
'version': 1,
'formatters': {
'simple': {
'format': '[%(asctime)s] %(levelname)s %(module)s %(message)s',
'datefmt': '%Y-%m-%d %H:%M:%S'
},
},
'handlers': {
'sentry': {
'level': 'WARNING',
'class': 'base.sentry.SentryHandler',
},
'streamlogger': {
'level': 'DEBUG',
'class': 'logging.StreamHandler',
'formatter': 'simple'
},
},
'loggers': {
'warn.logger': {
'handlers': ['sentry', 'streamlogger'],
'level': 'WARNING',
'propagate': False
},
},
'root': {
'level': 'INFO',
'handlers': ['streamlogger'],
},
}# base/sentry.py
from sentry_sdk.integrations.atexit import AtexitIntegration
from sentry_sdk.integrations.dedupe import DedupeIntegration
from sentry_sdk.integrations.django import DjangoIntegration
from sentry_sdk.integrations.excepthook import ExcepthookIntegration
from sentry_sdk.integrations.logging import EventHandler, LoggingIntegration
from sentry_sdk.integrations.modules import ModulesIntegration
from sentry_sdk.integrations.stdlib import StdlibIntegration
class SentryHandler(EventHandler):
...
def initialize(dsn, service, release, branch, debug: bool, breadcrumb_filters: list[dict] = None):
from django.conf import settings
sample_events_by_path = {
f'{settings.PROJECT_ROOT}/some/module.py': 0.0001,
}
def prepare_event(event, hint):
request_data = get_request_data()
tags = event.setdefault('tags', {})
# my hack for filtration here
is_sampled = False
if isinstance(hint, dict) and 'log_record' in hint:
pathname = getattr(hint['log_record'], 'pathname', '')
is_sampled = pathname in sample_events_by_path
sample_rate = sample_events_by_path.get(pathname, 1)
if sample_rate < 1.0 and random.random() < sample_rate:
return None
tags['is_sampled'] = is_sampled
return event
sentry_sdk.init(
dsn=dsn,
release=release or None,
before_send=prepare_event,
default_integrations=False,
integrations=[
StdlibIntegration(),
ExcepthookIntegration(),
DedupeIntegration(),
AtexitIntegration(),
ModulesIntegration(),
DjangoIntegration(),
LoggingIntegration(level=None, event_level=None),
],
)# some/module.py
import logging
import requests
logger = logging.getLogger('warn.logger')
def some_fun():
try:
resp = requsts.get('http://some.service.name')
resp.raise_for_status()
except Exception as e:
logger.warning('Error get value from service', exc_info=True) |
I'm not opposed to calling it FWIW I don't think there's an ideal solution here. But if I saw |
|
I discussed with @antonpirker this morning about naming the sampler as |
events_sampler option
sentrivana
left a comment
There was a problem hiding this comment.
LGTM, left one final suggestion.
sentry_sdk/client.py
Outdated
| else ("sample_rate", "contains") | ||
| ) | ||
| logger.warning( | ||
| "The provided %s %s an invalid value. The value should be a float or a bool. Defaulting to sampling the event." |
There was a problem hiding this comment.
Including the sample_rate in the log message might make debugging easier.
events_sampler optionerror_sampler option
|
Thank you so much for implementing this idea! Can you please tell me if I can hope that soon you will release a new version of the package so that I can install it from PyPI? |
|
@saippuakauppias We typically release new versions of the Python SDK every few weeks, so hopefully you will be able to install it from PyPI soon |
Now, it is possible for Python SDK users to dynamically sample errors using the
error_sampleroption, similar how they are able to sample transactions dynamically with thetraces_sampleroption.In other words, the new
error_sampleris to thesample_rateas the existingtraces_sampleris to thetraces_sample_rate.Fixes #2440