Skip to content

feat(py): Expose validation function for project configs#1860

Merged
jjbayer merged 23 commits intomasterfrom
feat/validate-project-config
Feb 20, 2023
Merged

feat(py): Expose validation function for project configs#1860
jjbayer merged 23 commits intomasterfrom
feat/validate-project-config

Conversation

@jjbayer
Copy link
Member

@jjbayer jjbayer commented Feb 17, 2023

Expose a function that validates arbitrary JSON against the ProjectConfig schema. This can be used in tests on sentry side, or possible even before writing configs to redis.

Because librelay releases are lagging behind relay master most of the time, adding new fields might pass validation even though the newest relay would reject them. To prevent this, the function has a strict mode which checks for stray additional fields.

Part of https://github.com/getsentry/team-ingest/issues/60.

#skip-changelog

@jjbayer jjbayer marked this pull request as ready for review February 17, 2023 13:38
@jjbayer jjbayer requested a review from a team February 17, 2023 13:38
struct RelayStr relay_validate_sampling_configuration(const struct RelayStr *value);

/**
* Validate entire project config. If `strict` is true, checks for unknown fields in the input.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

"checks" in the explanation of strict is confusing. I find more clear the sentence in the relay function itself:

If strict is true, verify that reserializing the instance results in the same fields.

If you want to have a not-so-defined sentence here to leave room for modifications on the other side, I'd suggest linking to the other function in relay-dynamic-config/src/utils.rs and removing the ambiguity.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good suggestion, changed in newest commit.

Base automatically changed from ref/project-config to master February 20, 2023 14:50

[dependencies]
anyhow = "1.0.66"
assert-json-diff = "2.0.2"
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I added this dependency to get more meaningful error messages, i.e. to enumerate any additional fields in the JSON. I used the library we also use for testing in relay-replays. If anyone has better ideas on how to compare JSON, let me know.

config["foobar"] = True
with pytest.raises(ValueError) as e:
sentry_relay.validate_project_config(json.dumps(config), strict=True)
assert str(e.value) == 'json atom at path ".foobar" is missing from rhs'
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The error message is a bit cryptic, but at least it contains the name of the problematic field.

struct RelayStr relay_validate_sampling_configuration(const struct RelayStr *value);

/**
* Validate entire project config. If `strict` is true, checks for unknown fields in the input.
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good suggestion, changed in newest commit.

Comment on lines 19 to 21
use relay_quotas::ReasonCode;
use relay_quotas::Scoping;
use relay_system::{Interface, NoResponse};
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please, sort the imports out. Move these to relay stuff

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks, I messed something up during merge. Reverted to master version now.

@jjbayer jjbayer merged commit 5d61524 into master Feb 20, 2023
@jjbayer jjbayer deleted the feat/validate-project-config branch February 20, 2023 16:45
jan-auer added a commit that referenced this pull request Feb 21, 2023
* master:
  chore(general): Fix typo with detailing (#1861)
  feat(py): Expose validation function for project configs (#1860)
jjbayer added a commit that referenced this pull request Mar 1, 2023
Some fields that are not being sent by Sentry when they are empty are
serialized by Relay. Align with Sentry and do not serialize these fields
if they have default values.

Reasons:

* The main reason is that #1860
introduced a validation function that can be used Sentry-side to check
if Sentry accidentally produces any fields that Relay cannot use. This
has already surfaced some inconsistencies (see
getsentry/sentry#45124), but it requires that
Sentry and Relay produce the same fields.
* Reduced payload size when sending configs to downstream Relays.
jjbayer added a commit to getsentry/sentry that referenced this pull request Mar 6, 2023
Make sure that the project configs that Sentry generates are consistent
with what Relay serializes for downstream usage:

* Sampling rules in Relay do not have an `active` field, so stop sending
it.
* `glob` sampling rules do not have options, so stop sending them.
* Sentry unnecessarily sends some empty/default project config fields
that Relay can easily derive. Stop sending them.

**Why?**

* getsentry/relay#1860 introduced a validation
function that can be used Sentry-side to check if Sentry accidentally
produces any fields that Relay cannot use. This has already surfaced
some inconsistencies, but it requires that Sentry and Relay produce the
same fields.
* Omitting default values reduces the payload size both in redis and on
the wire.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants