Skip to content

Pydantic plugins - entry-point support#6820

Merged
lig merged 62 commits intomainfrom
kludex/PYD-181
Sep 22, 2023
Merged

Pydantic plugins - entry-point support#6820
lig merged 62 commits intomainfrom
kludex/PYD-181

Conversation

@Kludex
Copy link
Copy Markdown
Member

@Kludex Kludex commented Jul 24, 2023 •
edited by samuelcolvin
Loading

Change Summary

Implement a plugin system using Python entry points.

Related issue number

Checklist

  • The pull request title is a good summary of the changes - it will be used in the changelog
  • Unit tests for the changes exist
  • Tests pass on CI
  • Documentation reflects the changes where applicable
  • My PR is ready to review, please add a comment including the phrase "please review" to assign reviewers

Missing

I'll add them if everything is fine with the current implementation.

  • Tests
  • Documentation
  • Other places where we can have PluggableSchemaValidator

Selected Reviewer: @samuelcolvin

@linear
Copy link
Copy Markdown

linear Bot commented Jul 24, 2023

PYD-181 Pydantic observability plugin

The idea is to support plugins from Pydantic, in a way we could "intercept", or "have hooks" on validation. For that, we intend to leverage the Entry Points Specification from Python.

As a reference, the two most known entry points are for flake8 and pytest.

The plugin needs to be agnostic i.e. it should just wrap the code we call the SchemaValidator.validate_*, which should allow pydantic_sdk to intercept the validation, and retrieve the data it wants.

The implementation will be "hidden" in a fork.

Questions

  1. Do we want to raise if the extra key doesn't have a plugin?

@cloudflare-workers-and-pages
Copy link
Copy Markdown

cloudflare-workers-and-pages Bot commented Jul 24, 2023 •
edited
Loading

Deploying with  Cloudflare Pages  Cloudflare Pages

Latest commit: e073168
Status: ✅  Deploy successful!
Preview URL: https://720a4185.pydantic-docs2.pages.dev
Branch Preview URL: https://kludex-pyd-181.pydantic-docs2.pages.dev

View logs

@Kludex Kludex requested a review from adriangb July 24, 2023 10:35
@Kludex
Copy link
Copy Markdown
Member Author

Kludex commented Jul 24, 2023

please review

@Kludex Kludex requested a review from samuelcolvin July 24, 2023 10:35
samuelcolvin
samuelcolvin requested changes Jul 24, 2023
View reviewed changes
Copy link
Copy Markdown
Member

@samuelcolvin samuelcolvin left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

otherwise I think this looks like a great start.

Obviously needs tests and documentation.

Comment thread pydantic/_internal/_dataclasses.py Outdated
Comment thread pydantic/plugin/loader.py Outdated
Comment thread pydantic/plugin/loader.py Outdated
Comment thread pydantic/plugin/schema_validator.py Outdated

def on_step(*args: Any, **kwargs: Any) -> None:
for plugin in plugins:
with contextlib.suppress(AttributeError, TypeError):
Copy link
Copy Markdown
Member

@samuelcolvin samuelcolvin Jul 24, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not sure we should be suppressing these errors?

Copy link
Copy Markdown
Member Author

@Kludex Kludex Jul 24, 2023 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Otherwise, I'll need more conditionals below to check if the steps are implemented, i.e. if on_event is not None and if step_func is not None. It's a bit more verbose.

What do you prefer?

Comment thread pydantic/plugin/schema_validator.py Outdated
@pydantic-hooky pydantic-hooky Bot added awaiting author revision awaiting changes from the PR author and removed ready for review labels Jul 24, 2023
@pydantic-hooky pydantic-hooky Bot assigned Kludex and unassigned davidhewitt Jul 24, 2023
@Kludex
Copy link
Copy Markdown
Member Author

Kludex commented Jul 24, 2023

@samuelcolvin Should I allow BaseModel.__init_subclass__ to receive keyword arguments on this PR?

@Kludex
Copy link
Copy Markdown
Member Author

Kludex commented Jul 25, 2023

@samuelcolvin Should I allow BaseModel.__init_subclass__ to receive keyword arguments on this PR?

I've talked to @davidhewitt. I'll accept other keyword arguments for __init_subclass__, and allow the plugin to customize itself depending on that.

@Kludex Kludex marked this pull request as draft July 25, 2023 09:43
@Kludex Kludex force-pushed the kludex/PYD-181 branch 2 times, most recently from 54a7612 to f7c32d1 Compare July 25, 2023 12:29
@Kludex Kludex mentioned this pull request Jul 25, 2023
Closed
4 tasks
@Kludex Kludex marked this pull request as ready for review July 26, 2023 10:16
@Kludex
Copy link
Copy Markdown
Member Author

Kludex commented Jul 26, 2023

Only some tests missing here.

@Kludex
Copy link
Copy Markdown
Member Author

Kludex commented Jul 26, 2023

please review

@pydantic-hooky pydantic-hooky Bot added ready for review and removed awaiting author revision awaiting changes from the PR author labels Jul 26, 2023
@pydantic-hooky pydantic-hooky Bot assigned davidhewitt and unassigned Kludex Jul 26, 2023
@samuelcolvin samuelcolvin changed the title Support pydantic entry point Pydantic plugin - entry point support Jul 26, 2023
lig added 14 commits September 18, 2023 10:09
@lig
🚨 Update usage docs link for plugins
0aac537
@lig
🏷 Fix event callbacks type
9f58041
@lig
🚧 Reproduce circular import in the plugin module
4c6c9e2
@lig
🐛 Suppress circular import errors while executing plugins
f21c926
@lig
💡 Add warning on import error when loading plugins
2581cdb
@lig
📝 Update docs and add experimental feature banners
abfffd8
@lig
✅ Skip randomly failing test on Python 3.9
df4b2a3
@lig
👌 Address most of code review comments
e43b40f
@lig
♻ Refactor PluggableSchemaValidator
e79ff4a
@lig
🔇 Suppress all exceptions while running a plugin
abe836b
@lig
🚑 Fix crash when only one of two handlers provided in a plugin
480b678
@lig
♻ Make Plugin class into PluginProtocol class
f3453d1
@lig
📝 Merge plugin docs into single document in the Usage section
e5802ba
@lig
🚑 Suppress ImportError only while running plugin callbacks
1e93da5
@samuelcolvin samuelcolvin mentioned this pull request Sep 22, 2023
Closed
1 task
@samuelcolvin samuelcolvin mentioned this pull request Sep 22, 2023
Merged
7 tasks
lig
lig suggested changes Sep 22, 2023
View reviewed changes
Comment thread docs/usage/plugins.md Outdated
Comment thread docs/usage/plugins.md Outdated
Comment thread pydantic/main.py
Comment thread tests/test_plugins.py
Comment thread tests/test_plugins.py
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@samuelcolvin samuelcolvin samuelcolvin approved these changes

@adriangb adriangb Awaiting requested review from adriangb

@hramezani hramezani Awaiting requested review from hramezani

+1 more reviewer

@lig lig lig approved these changes

Reviewers whose approvals may not affect merge requirements

Assignees

@Kludex Kludex

Labels

awaiting author revision awaiting changes from the PR author relnotes-feature

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

Support plugins

5 participants

@Kludex @lig @hramezani @samuelcolvin @davidhewitt