Skip to content

Add sender example for protobuf data#712

Merged
n3wscott merged 3 commits intocloudevents:mainfrom
kconwayinvision:protobuf/examples
Aug 31, 2021
Merged

Add sender example for protobuf data#712
n3wscott merged 3 commits intocloudevents:mainfrom
kconwayinvision:protobuf/examples

Conversation

@kconwayinvision
Copy link
Copy Markdown
Contributor

This example mirrors the existing HTTP sender example except that it
adds a protobuf object as the payload instead of a JSON map.

This example mirrors the existing HTTP sender example except that it
adds a protobuf object as the payload instead of a JSON map.

Signed-off-by: Kevin Conway <kevinconway@invisionapp.com>
@dan-j
Copy link
Copy Markdown
Contributor

dan-j commented Aug 31, 2021

LGTM, I've never contributed to the http aspects but since this is just adding a sample I guess there's nothing really to be worried about. @n3wscott?

@dan-j dan-j requested a review from n3wscott August 31, 2021 12:36
e.SetType("com.cloudevents.sample.sent")
e.SetSource("https://github.com/cloudevents/sdk-go/v2/samples/http/sender-protobuf")
e.SetDataSchema(string(data.ProtoReflect().Descriptor().FullName()))
_ = e.SetData(pbcloudevents.ContentTypeProtobuf, data)
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

oh! only the data is encoded in protobuf? I had assumed the entire cloudevent was being sent as a protobuf

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Haven't looked into the code, but maybe line 22 tells the encoder to use protobuf?

@n3wscott
Copy link
Copy Markdown
Member

LGTM, Can you clarify if it is possible to use proto for the entire message, or just the data? Thanks for adding a sample!!

Signed-off-by: Kevin Conway <kevinconway@invisionapp.com>
Signed-off-by: Kevin Conway <kevinconway@invisionapp.com>
@kconwayinvision
Copy link
Copy Markdown
Contributor Author

@n3wscott @dan-j I added a protobuf receiver that can be run together with the sender for a full end-to-end sample. Unfortunately, this only demonstrates the data encoding support and does not demonstrate the envelope format. The issue is that HTTP, and the other protocols, already implement a custom envelope format in the form of headers. For example, the protobuf sender makes requests like:

POST / HTTP/1.1
Host: localhost:8080
User-Agent: Go-http-client/1.1
Content-Length: 8
Ce-Dataschema: types://pb.Sample
Ce-Id: 67426e8d-f8b2-449f-aace-9007ac98ccc1
Ce-Source: https://github.com/cloudevents/sdk-go/v2/samples/http/sender-protobuf
Ce-Specversion: 1.0
Ce-Time: 2021-08-31T18:51:29.01318Z
Ce-Type: com.cloudevents.sample.sent
Content-Type: application/cloudevents+protobuf
Content-Type: application/protobuf
Accept-Encoding: gzip


sample

The content body is encoded with protobuf but the envelop is conveyed via the headers. This seems true for all protocols implemented for Go except nats (which requires JSON format according to the spec) and websockets (which has no server support with which I could create the demo). This makes it difficult to demo an end-to-end usage of the protobuf envelope format. I'm open to suggestions if folks can suggest how I might create the format sample.

For my usage of protobuf+CloudEvents, we've defined a protobuf service that accepts the CloudEvent structure. Example:

service SinkService {
  rpc Send(CloudEvent) returns (Empty);
  rpc Request(CloudEvent) returns (CloudEvent);
}

This seems like a potentially missing part of the protobuf spec that could then be used to generate gRCP (or other) implementations of the service/client wrapped in the sender and receiver interfaces. I think something like that would provide a path for a protobuf native suite of protocols.

I also addressed an issue raised by a comment I can't seem to find anymore about the data schema failing validation. I was including the type of the protobuf message as the data schema but the CloudEvent spec requires the field to be an absolute URI. I've adjusted the example to use an absolute URI and ran the sample to ensure it sends requests as expected.

Copy link
Copy Markdown
Member

@n3wscott n3wscott left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@n3wscott n3wscott merged commit 7e47d96 into cloudevents:main Aug 31, 2021
@kconwayinvision kconwayinvision deleted the protobuf/examples branch September 30, 2021 13:49
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants