Skip to content

draft: Common Service Client Mix-ins#694

Merged
noahdietz merged 35 commits intoaip-dev:masterfrom
noahdietz:mixin-aip
Aug 20, 2021
Merged

draft: Common Service Client Mix-ins#694
noahdietz merged 35 commits intoaip-dev:masterfrom
noahdietz:mixin-aip

Conversation

@noahdietz
Copy link
Collaborator

The following is a draft AIP for the forthcoming common service client mix-ins feature in client library generators.

@noahdietz noahdietz requested a review from a team as a code owner February 10, 2021 21:59
@google-cla google-cla bot added the cla: yes This human has signed the Contributor License Agreement. label Feb 10, 2021
Copy link
Contributor

@vchudnov-g vchudnov-g left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Some clarification comments, and some editorial suggestions.

Copy link
Contributor

@miraleung miraleung left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should we add a product-facing AIP, so that API teams can easily see a best-practices sample for using mixins?

@noahdietz
Copy link
Collaborator Author

Should we add a product-facing AIP, so that API teams can easily see a best-practices sample for using mixins?

I'm not 100% sure what you mean, but it shouldn't be something that an API team needs to go out of their way to use.

Often teams will include Locations, IAMPolicy, or Operations because the API uses them. A side-effect of them including the such a service is that the client libraries surface the RPCs on their clients alongside the product RPCs. These common services are already widely used in Service configurations today.

@noahdietz noahdietz requested a review from miraleung March 3, 2021 23:52
@noahdietz
Copy link
Collaborator Author

@lukesneeringer gentle ping.

@lukesneeringer
Copy link
Contributor

Looking!

Copy link
Contributor

@lukesneeringer lukesneeringer left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks mostly good. One significant question and a few nits.

### Generating mix-in methods

The mix-in API RPCs **should** be generated as methods on the surface of the
host API's service client library, alongside the host service's RPCs. This
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would like to dispute this. I assert it would be better to generate another service class with the mixin service's RPCs on it.

Rationale:

  • Removes ambiguity and avoids the potential for RPC name collision.
  • In the multiple host service case, generating the methods multiple times gives the potential impression that there is some kind of difference to the method. The basic principle here is: If your house has multiple front doors, guests will not know which one to use.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That's a good point, a separate client class may be better. Do you mean a single "mixin" client class with all of the configured mixin methods on it? Or do you mean individual OperationsClient, LocationsClient, IAMPolicy client classes? I think the former is better than the latter - it's clear these are utility services, vs. first class product services, and reduces confusion with any existing generated instances of these service clients.

I have a draft implementation of this for Go and while it does generate a bit of duplicate code when there are multiple services, I like that:

  • I don't need to instantiate another client to utilize any of these RPCs, they are right there on my existing client
  • I can use the GAPIC as an interface implementation for helpers, like with Operation wrappers that need a client reference to invoke Poll, Wait, etc. rather than needing to incorporate another client - (ping me if you'd like the internal one-pager I have for some Go GAPIC improvements)
  • These mix-in methods are closer to the other helpers, like response-type specific Operation wrappers, Resources a user wants to check the IAM Policy of, or the supported Locations for the Resource they are about to create
  • I assert that naming collisions will seldom happen (outside of the known GetIamPolicy, but this the same RPC, not a different one with the same name)

@miraleung you've implemented this for Java, WDYT?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Or, should we be less prescriptive and allow languages to do what they think is best?

Copy link
Contributor

@miraleung miraleung Mar 12, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

IMO the current approach has upsides, and I'd like to see what product teams or Yoshi think about the various options (wrt usability). Do we need to consider existing timelines of teams who are asking for this feature?

Backwards API compatibility would be one motivation for adding this to existing clients - PubSub and KMS currently use† the existing approach in Java and PHP. These APIs will require either this solution or a one-off among all microgenerators. Moreover, this might allow us to migrate other APIs that copy-pasted IAM RPCs into their protobufs, if a scenario arose where we'd be OK with breaking protobuf changes but none in the generated clients.

For Java and PHP, IMO some upsides of the current approach include a simpler generation logic, client library dev users include not having to figure out which new client to use (for option 1), and they won't be confused between many APIs' LocationsClient (option 2).

Or, should we be less prescriptive and allow languages to do what they think is best?

IMO staying consistent with the high-level structure would be ideal, unless there are core language motivations to do otherwise.

†It's complicated.... PubSub chose to configure the monolith's gapic.yaml to mix-in IAM into both of their Pub/Sub clients. However, they recently added a new Schema service without updating gapic.yaml, and it's hard to tell whether this was intended or due to loss of tribal knowledge.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Backwards API compatibility would be one motivation for adding this to existing clients - PubSub and KMS currently use† the existing approach in Java and PHP.

I think we should ignore this point. I am skeptical of the idea that we make sweeping decisions on a design based on 2 existing libraries in 2 languages.

Moreover, this might allow us to migrate other APIs that copy-pasted IAM RPCs into their protobufs

But those APIs themselves can not migrate for their own reasons, so whether you can migrate them in client libraries is not really relevant.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I am skeptical of the idea that we make sweeping decisions on a design based on 2 existing libraries in 2 languages.

Fair! It is a nice-to-have part of this approach.

But those APIs themselves can not migrate for their own reasons,

Ok. With a certain API framework being made the new default and precluding the need to redefine such RPCs at the Service level, should we not follow suit with the precedent already set by APIs today that is redefining the RPCs at the Service level? That approach was probably chosen for some technical reasons, but it also brought usability benefits - having the IAM rpcs defined directly on the client.

What say you to the other advantages we believe this approach brings?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@noahdietz @miraleung I totally forgot about this thread, and I apologize. I realize I am holding you up.

I suggest let's talk through this in a meeting. I am reasonably convinced in my own mind (😛 ) that my approach is better, but I might be missing something. That said, I think the right decision here is for me to try to convince you, and then defer to you on the final decision.

Seem reasonable?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sounds reasonable to me! I will schedule some time. Thanks Luke.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Circling back around after some time now that some blocking tasks have been resolved. I have updated this to allow for language idiomatic representations. Individual language actools+yoshi pairs have indicated a preference that is different from what Mira & I have proposed, and I believe it is reasonable to allow for language idiomatic flexibility - that is the spirit of the micro-generators and a goal of client libraries anyways.

@noahdietz
Copy link
Collaborator Author

@lukesneeringer gentle ping to review when you get a chance, please. :)

Copy link
Contributor

@vchudnov-g vchudnov-g left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I have some clarification questions/suggestions. Some are nits, but some are non-trivial.

```

Should the host API declare the `google.iam.v1.IAMPolicy` as a mixin service,
client library generators **must not** generate the resulting mixin methods for
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

s/the resulting mixin methods/the resulting mixin methods that match the names explicitly declared above, but they **must** generate the other methods for this mixin/ right?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hm interesting question. Since IAMPolicy is the only API we have that is redeclared by each individual API, we only really have the one use case to go off of. With IAMPolicy every RPC is redeclared, always, so omission of some RPCs but not others hasn't come up before. This is a question worth answering to make things future proof. Your interpretation certainly would work for this one use case we have today, because every RPC is always redeclared, I just don't know if that would be right for another such API. WDYT?

@noahdietz
Copy link
Collaborator Author

@lukesneeringer PTAL, we have already started on generator implementations and I'd like this to be submitted for reference.

@noahdietz
Copy link
Collaborator Author

@lukesneeringer can you give this another once-over?

@noahdietz noahdietz dismissed lukesneeringer’s stale review August 20, 2021 23:07

LGTM was given offline.

@noahdietz noahdietz merged commit 4971fdc into aip-dev:master Aug 20, 2021
@noahdietz noahdietz deleted the mixin-aip branch August 20, 2021 23:09
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

cla: yes This human has signed the Contributor License Agreement.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

6 participants