Skip to content

Add RFC for plugin connectors#49

Merged
majetideepak merged 4 commits into
prestodb:mainfrom
tdcmeehan:dyncon
Oct 23, 2025
Merged

Add RFC for plugin connectors#49
majetideepak merged 4 commits into
prestodb:mainfrom
tdcmeehan:dyncon

Conversation

@tdcmeehan

Copy link
Copy Markdown
Contributor

No description provided.

@majetideepak majetideepak left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The proposal looks good to me. Making the protocol layer pluggable is the missing piece for connector extensibility in Presto C++. Glad to see that happening here!

Comment thread RFC-0019-connector-plugins.md Outdated
void deserialize(const std::string& binary,
std::shared_ptr<ConnectorSplit>& split) const override {
// Deserialize binary into protocol-specific type
ArrowFlightSplit protocolSplit;

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This code block is not correct. Maybe copy a subset of the TpchConnectorProtocol code from your PR.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks, done!

@majetideepak majetideepak left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks.

@majetideepak majetideepak merged commit 5b10b34 into prestodb:main Oct 23, 2025
tdcmeehan added a commit to prestodb/presto that referenced this pull request Nov 5, 2025
…ion codecs (#26257)

## Description
Add support for custom connector-provided serialization codecs

## Motivation and Context
This will allow plugin connectors to be written in C++.

RFC: prestodb/rfcs#49
End to end changes migrating TPCH to the new framework:
#26026

## Impact
No immediate impact

## Test Plan
Included tests

## Contributor checklist

- [ ] Please make sure your submission complies with our [contributing
guide](https://github.com/prestodb/presto/blob/master/CONTRIBUTING.md),
in particular [code
style](https://github.com/prestodb/presto/blob/master/CONTRIBUTING.md#code-style)
and [commit
standards](https://github.com/prestodb/presto/blob/master/CONTRIBUTING.md#commit-standards).
- [ ] PR description addresses the issue accurately and concisely. If
the change is non-trivial, a GitHub Issue is referenced.
- [ ] Documented new properties (with its default value), SQL syntax,
functions, or other functionality.
- [ ] If release notes are required, they follow the [release notes
guidelines](https://github.com/prestodb/presto/wiki/Release-Notes-Guidelines).
- [ ] Adequate tests were added if applicable.
- [ ] CI passed.

## Release Notes

```
== NO RELEASE NOTE ==
```
20001020ycx added a commit to prestodb/presto that referenced this pull request Jun 9, 2026
…deserialization (#27652)

## Description

Add `customSerializedValue` routing in `from_json()` for all 10
connector handle types. When `customSerializedValue` is present in the
incoming JSON, the routing Base64-decodes it and dispatches to
`ConnectorProtocol::deserialize()` instead of the default JSON-to-struct
parsing.

## Motivation and Context

Enables C++ plugin connectors to use binary serialization codecs instead
of JSON, as proposed in prestodb/rfcs#49.

This is the C++ worker-side counterpart to the Java `ConnectorCodec` /
`CodecSerializer` pipeline (#26257).

## Test Plan

- [x] Build `presto_server` with the changes
- [x] End-to-end tested with a customized connector using binary
serialization codecs on the Java side, loaded as a native plugin via
`plugin-dir`

## Release Notes

```
== NO RELEASE NOTE ==
```

## Summary by Sourcery

Add support for binary-serialized connector handles via a new
customSerializedValue pathway in native protocol deserialization.

New Features:
- Allow connector handle JSON payloads to specify a
customSerializedValue that is Base64-decoded and deserialized by the
connector protocol instead of using standard JSON mapping.
- Introduce binary serialization hooks for ColumnHandle,
ConnectorPartitioningHandle, and ConnectorIndexHandle through new
serialize/deserialize APIs on ConnectorProtocol and corresponding handle
types.

Enhancements:
- Refine native protocol from_json routing for all connector handle
types to guard against use of customSerializedValue with internal handle
types.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

from:IBM PRs from IBM

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants