See also googleapis/google-cloud-rust#3239

1. Problem Statement

The google-cloud-rust libraries are being updated to add OpenTelemetry-compatible tracing for all outgoing HTTP requests. This requires propagating additional information from the generated client code to the underlying HTTP client (google-cloud-gax-internal):

1. Client-Level Info: Service name, client version, artifact name, and default host.
2. Request-Level Info: The unformatted HTTP path template (e.g., projects/{project}/topics/{topic}) corresponding to the API method being called. This is required for the url.template attribute in OpenTelemetry spans.

The librarian code generator must be modified to inject this information into the generated Rust code.

2. Chosen Alternative

The chosen approach is to modify the Mustache templates and the Go generator logic in librarian to define a static InstrumentationClientInfo in the generated lib.rs (within an info module) and use it within the generated transport.rs when constructing the ReqwestClient. Additionally, the path_template string will be added to RequestOptions at runtime within the transport layer.

Details:

1. InstrumentationClientInfo Definition and Usage:

   • File to Modify (Template): internal/sidekick/internal/rust/templates/crate/src/lib.rs.mustache

   • Change: Inside the pub(crate) mod info, define NAME, VERSION, and DEFAULT_HOST_SHORT constants. Then, define the pub(crate) static INSTRUMENTATION_CLIENT_INFO: google_cloud_gax_internal::options::InstrumentationClientInfo instance, initializing its fields using these constants.

     // In lib.rs.mustache
     // ... inside mod info
     const NAME: &str = env!("CARGO_PKG_NAME");
     const VERSION: &str = env!("CARGO_PKG_VERSION");
     const DEFAULT_HOST_SHORT: &str = "{{Codec.DefaultHostShort}}";
     
     lazy_static::lazy_static! {
         pub(crate) static ref X_GOOG_API_CLIENT_HEADER: String = {
             let ac = gaxi::api_header::XGoogApiClient{
                 name:          NAME,
                 version:       VERSION,
                 library_type:  gaxi::api_header::GAPIC,
             };
             ac.rest_header_value()
         };
     }
     
     pub(crate) static INSTRUMENTATION_CLIENT_INFO: google_cloud_gax_internal::options::InstrumentationClientInfo = google_cloud_gax_internal::options::InstrumentationClientInfo {
         service_name: "{{service.name}}", // e.g., showcase
         client_version: VERSION,
         client_artifact: NAME,
         default_host: DEFAULT_HOST_SHORT,
     };

   • File to Modify (Template): internal/sidekick/internal/rust/templates/crate/src/transport.rs.mustache

   • Change: In the new constructor of the transport struct, when google_cloud_gax_internal::http::ReqwestClient is created, conditionally call .with_instrumentation(Some(&crate::info::INSTRUMENTATION_CLIENT_INFO)) if tracing is enabled.

     let http_client = {
         let client = google_cloud_gax_internal::http::ReqwestClient::new(config.clone(), crate::DEFAULT_HOST).await?;
         if google_cloud_gax_internal::options::tracing_enabled(&config) {
             client.with_instrumentation(Some(&crate::info::INSTRUMENTATION_CLIENT_INFO))
         } else {
             client
         }
     };

2. path_template Population:

   • File to Modify (Go): internal/sidekick/internal/rust/annotate.go

   • Change: Add a method (b *pathBindingAnnotation) PathTemplate() string to construct the raw HTTP path template string from the binding's PathFmt and Substitutions. This method will be exposed to the Mustache template as {{Codec.PathTemplate}}.

   • File to Modify (Template): internal/sidekick/internal/rust/templates/crate/src/transport.rs.mustache

   • Change: Inside each {{#PathInfo.Bindings}} block in the .or_else(|| { ... }) chain, after the path variable is successfully determined, add a line to update the options variable using the internal setter:

     options = google_cloud_gax::options::internal::set_path_template(options, Some("{{Codec.PathTemplate}}".to_string()));

   • Rationale: The or_else chain ensures that this line is only executed for the chosen HTTP binding. {{Codec.PathTemplate}} will provide the raw, unformatted path template.

Infrastructure: No new infrastructure is required. Changes are confined to the librarian Go code and its Mustache templates.

3. Path Template Requirement & Examples

The url.template attribute for OpenTelemetry HTTP client spans requires a low-cardinality representation of the URI path. Path parameters from the google.api.http annotation are represented by curly braces containing the corresponding request message field name. This aligns with the OpenTelemetry Semantic Conventions for HTTP spans (see https://opentelemetry.io/docs/specs/semconv/registry/attributes/url/#url-template).

The (b *pathBindingAnnotation) PathTemplate() string method in annotate.go must produce the following mappings:

1. Simple Literal:
   • Proto: get: "/v1/things"
   • url.template: /v1/things
2. Single Variable:
   • Proto: get: "/v1/things/{thing_id}"
   • url.template: /v1/things/{thing_id}
3. Multiple Variables:
   • Proto: get: "/v1/projects/{project}/locations/{location}"
   • url.template: /v1/projects/{project}/locations/{location}
4. Variable with Complex Segment Match:
   • Proto: get: "/v1/{name=projects/*/locations/*}/databases"
   • url.template: /v1/{name}/databases
5. Variable Capturing Remaining Path:
   • Proto: get: "/v1/objects/{object=**}"
   • url.template: /v1/objects/{object}
6. Top-Level Single Wildcard:
   • Proto: get: "/{field=*}"
   • url.template: /{field}
7. Top-Level Double Wildcard:
   • Proto: get: "/{field=**}"
   • url.template: /{field}
8. Path with Custom Verb:
   • Proto: post: "/v1/things/{thing_id}:customVerb"
   • url.template: /v1/things/{thing_id}:customVerb

The (b *pathBindingAnnotation) PathTemplate() string method in annotate.go must implement the logic to achieve these transformations by replacing {} in b.PathFmt with { + sub.FieldName + } from the ordered b.Substitutions.

4. Test Plan

1. Unit Tests for Generator Changes (Go):

   • Add tests to annotate_test.go to verify the output of the new PathTemplate() method, covering all the examples listed in Section 3.
   • Test that the Go code correctly extracts the service name and default host.
   • Test that the correct data is added to the Mustache template context.

2. Golden Tests for Generated Code:

   • Update or add golden tests that compare the output of the generator with known good examples.
   • Ensure the new constants and the static InstrumentationClientInfo are present in the info module within lib.rs.
   • Ensure the call to set_path_template with the correct {{Codec.PathTemplate}} is present in transport.rs for each binding.

3. Integration Tests (in google-cloud-rust):

   • After regenerating the clients using the modified librarian, run the existing google-cloud-rust integration tests.
   • Specifically, the new tests in gax-internal that subscribe to tracing events should now receive the url.template attribute, populated with the value set by the generated code. Verify this attribute matches the expected path template from the proto definition.

5. Corpus of Relevant Information

• Files to Modify:

  • /usr/local/google/home/westarle/src/otel-rust/librarian/internal/sidekick/internal/rust/templates/crate/src/lib.rs.mustache
  • /usr/local/google/home/westarle/src/otel-rust/librarian/internal/sidekick/internal/rust/templates/crate/src/transport.rs.mustache
  • /usr/local/google/home/westarle/src/otel-rust/librarian/internal/sidekick/internal/rust/annotate.go
  • /usr/local/google/home/westarle/src/otel-rust/librarian/internal/sidekick/internal/rust/annotate_test.go

• Key Algorithms/Logic:

  • The PathTemplate() method in annotate.go is crucial for correctly formatting the path template string.
  • The core logic change in transport.rs.mustache involves inserting the set_path_template() call within each block of the or_else chain.

• Code Snippets: See Section 2 for the conceptual Mustache changes.

• Documentation:

  • google.api.http annotation documentation.
  • OpenTelemetry Semantic Conventions for url.template.

6. Discounted Alternatives

1. path_template in ClientConfig: Rejected as too coarse-grained.
2. Separate Function/Map for Template Selection: Rejected as more complex to generate than inlining.
3. Setting path_template Early (e.g., client_method_preamble.mustache): Rejected as the binding is not yet known.
4. Passing Client Info via ReqwestClient::new2: Rejected in favor of the existing with_instrumentation method in google-cloud-rust.

• Rationale: The chosen method integrates most cleanly with the existing template structure and the implemented features in google-cloud-rust.

7. Risks and Mitigation Strategies

1. Risk: Increased template complexity.
   • Mitigation: The changes are relatively small and localized. Golden tests will help catch regressions.
2. Risk: Incorrect PathTemplate() logic for complex bindings.
   • Mitigation: Thorough unit testing of PathTemplate() in annotate_test.go with diverse examples.

8. Next Steps

Implement the changes in the librarian Go generator and Mustache templates, followed by testing as outlined above. Regenerate clients in google-cloud-rust to confirm end-to-end functionality.