feat(java): migrate Java container from sdk-platform-java#2670
feat(java): migrate Java container from sdk-platform-java#2670
Conversation
…3926) This change introduces the initial scaffolding for `librariangen`, a containerized Go application that will serve as the Java-specific code generator within the Librarian pipeline. The primary goal of this change is to establish the foundational components for the Java language container, including: * A new `librariangen` CLI application: Written in Go, this will be the entry point for Java code generation tasks invoked by Librarian. It currently includes basic command dispatching and support for `--version`. * A multi-stage `Dockerfile`: This creates a minimal container image for librariangen. * Unit tests and CI: A new GitHub Actions workflow has been added to build and test the librariangen executable. This scaffolding is the first step in implementing the full code generation logic within the Librarian system, eventually replacing the Java hermetic code generation system. The code is largely adapted from the Go language container at https://github.com/googleapis/google-cloud-go/tree/main/internal/librariangen. Fixes: #2500
This should suppress the warning in b/450542318.
This will be used for executing commands like protoc. Copied from https://github.com/googleapis/google-cloud-go/tree/main/internal/librariangen/execv.
This will be used for parsing Librarian CLI requests. Copied from https://github.com/googleapis/google-cloud-go/tree/main/internal/librariangen/request.
Refactors the main function to be more testable by extracting the logic into a `runCLI` function. The logging setup is also extracted into a `parseLogLevel` function to allow for easier testing. This change increases the test coverage of main.go from 65.5% to 96.9%.
This will be used for constructing the protoc command for generating Java GAPICs. Based on https://github.com/googleapis/google-cloud-go/tree/main/internal/librariangen/protoc with the following important changes: * The `Build` function in `protoc.go` is updated to construct the `protoc` command with the correct arguments for Java GAPIC generation. * The tests in `protoc_test.go` are updated to reflect the changes in `protoc.go`. * The `gapicImportPath` is removed from the test configuration, as it is not relevant for Java.' * The testdata is included with modifications for Java.
Based on https://github.com/googleapis/google-cloud-go/tree/main/internal/librariangen/bazel with adaptation for Java.
Based on https://github.com/googleapis/google-cloud-go/tree/main/internal/librariangen/generate with adaptation for Java. Currently it's just the scaffolding and more work is needed to generate a usable GAPIC library. The `generate` package contains the core logic for the generation process, including: - Reading and parsing the `generate-request.json` from librarian. - Parsing `BUILD.bazel` files in the googleapis repository to extract GAPIC configuration. - Building and executing `protoc` with the `gapic-generator-java` plugin. - Unzipping and restructuring the generated files into the final library layout. A `run-generate-library.sh` script is included for local development and end-to-end testing of the generation process. Additionally, a `go.work` file has been added to the root of the repository to support the multi-module workspace structure.
…request (#3965) Introducing languagecontainer package. * **New languagecontainer package**: A new `languagecontainer` package has been introduced to encapsulate language-specific container operations and command execution logic, promoting modularity. This package parses the request JSON file and calls the corresponding implementation method in each language container. The languagecontainer package itself should not have language-specific implementation. * **release-init command parsing**: The `languagecontainer.Run` function now includes robust logic to parse `release-init` requests from JSON files, handle command-line flags, and invoke the `ReleaseInit` function. * Why isn't this providing an interface? It's because if `LanguageContainer` is an interface, then there would be package name conflict of `languagecontainer/release` package (language agostic) and `release` package (language-specific. In this case Java-specific). [Here is a piece of code](#2516 (comment)). * **Main command dispatch refactor**: The `librariangen` `main.go` has been refactored to delegate non-`generate` commands to the new `languagecontainer.Run` function, centralizing command execution and wiring up `release.Init` for the `release-init` command. * **message.Change struct update**: The `SourceCommitHash` field in the `message.Change` struct has been renamed to `CommitHash` for improved clarity and consistency. This is due to the recent renaming of the field and [google-cloud-go/internal/librariangen/request.Change](https://github.com/googleapis/google-cloud-go/blob/7a85df39319e3a4870d4ad413f714ae5edd78ac8/internal/librariangen/request/request.go#L60) already has the field. The user (Java's language container in this case) doesn't have to implement the parsing logic and the tests. I moved the argument parsing tests to languagecontainer/languagecontainer_test.go from main_test.go.
* Introduces the generation of gRPC stubs and resource helpers. * Modifies the `Generate` function to create output directories for GAPIC, gRPC, and proto files. * Updates the `invokeProtoc` function to pass an `OutputConfig` struct. * Updates the `restructureOutput` function to handle gRPC stubs and resource names. * Adds a `copyAndMerge` function to merge resource name files into the proto destination. * Updates the `cleanupIntermediateFiles` function to remove the GAPIC, gRPC, and proto directories. * Updates tests to reflect these changes. * Improvements in error handling.
|
(Probably you're familiar with this) Ensure you don't "squash" the commit. |
|
If you squash commits, the resulting commit is just the same as copying files over. You didn't need to preserve commit history. |
The reference to the PR will still exist, and I like having all of the history in the source PR. |
|
In that case, you could just copy-paste the output of Anyway, I'm fine with not preserving the history here. |
Good idea. added. |
Codecov Report❌ Patch coverage is Additional details and impacted files@@ Coverage Diff @@
## main #2670 +/- ##
==========================================
- Coverage 86.93% 86.75% -0.19%
==========================================
Files 108 117 +9
Lines 9194 9580 +386
==========================================
+ Hits 7993 8311 +318
- Misses 861 902 +41
- Partials 340 367 +27 ☔ View full report in Codecov by Sentry. 🚀 New features to boost your workflow:
|
PR created by the Librarian CLI to initialize a release. Merging this PR will auto trigger a release. Librarian Version: not available Language Image: us-central1-docker.pkg.dev/cloud-sdk-librarian-prod/images-prod/librarian-release-container:latest <details><summary>librarian: 1.0.0</summary> ## [1.0.0](v0.5.0...v1.0.0) (2025-11-06) ### Features * Remove `..._gax` dependency (#2713) ([01675b7](01675b72)) * add context to Librarian PRs (#2698) ([0523329](0523329a)) * add a package comment linter (#2712) ([1bd3e32](1bd3e32e)) * generate pom.xml files (#2682) ([50b95f2](50b95f20)) * add test-container test logic (#2656) ([514cf7e](514cf7e5)) * improve rust-publish logging (#2671) ([585ed50](585ed50b)) * write a timing log at the end of update-image (#2771) ([5fc9e3b](5fc9e3b6)) * migrate Java container from sdk-platform-java (#2670) ([69ac47f](69ac47fd)) * switch to original head after running update-image (#2696) ([7a3e404](7a3e404a)) * format bulk commit from other sources in release notes (#2665) ([7c52da2](7c52da2b)) * enable multi-version API generation (#2699) ([86c5250](86c52507)) * release stage to update pom.xml files (#2772) ([be56755](be567550)) * use proto presence info to generate null-safe code (#2726) ([e36fb81](e36fb81a)) * Base messages in google_cloud_protobuf (#2660) ([e607ea6](e607ea63)) ### Bug Fixes * retry Github 503 requests up to 3 times (#2650) ([09468fa](09468faf)) * Changed docs to ConfigurationException (#2697) ([0950c1e](0950c1e3)) * fix the reference to the old "librariangen" folder (#2677) ([09dc53f](09dc53fe)) * run godoclint via golangci-lint (#2751) ([264a6a0](264a6a0e)) * pass context as the first argument (#2769) ([298a3bd](298a3bd8)) * Move HTTP client creation to `_gax` (#2672) ([4968d63](4968d63d)) * deduplicate bulk commits (#2758) ([4dfae9a](4dfae9ae)) * Support additional api paths for an existing library (#2666) ([50046f5](50046f55)) * remove stray TODO (#2748) ([5072f0e](5072f0e0)) * omit status field when empty (#2654) ([572ae4f](572ae4f3)) * use T.Context in tests (#2768) ([7e7cd2d](7e7cd2dd)) * wrap error to provide more context for commitAndPush (#2767) ([a2a41a4](a2a41a4a)) * use t.Fatal in tests for proper failure handling (#2759) ([cdabb28](cdabb287)) * enforce removal before copying library files (#2765) ([d5ac6c8](d5ac6c87)) * change log level to debug (#2798) ([f042d0b](f042d0bd)) * change path to `doc.go` in docgen test (#2700) ([fd6bae4](fd6bae40)) ### Documentation * use consistent library id in flags and examples (#2770) ([87a1005](87a10056)) </details>
This PR moves Java Librarian container implementation WIP from
github.com/googleapis/sdk-platform-java/internal/librariangentogithub.com/googleapis/librarian/internal/container/javawhile preserving the git history.Here are the steps that were followed:
Since we are still in early stages of implementing the Java container, the plan is to squash-and-merge this PR.
original commits: