OpenAPI Export for API Definition, Swagger UI Binding by Nanashi-lab · Pull Request #1454 · golemcloud/golem

Nanashi-lab · 2025-03-28T03:43:26Z

PR: OpenAPI Export and Swagger UI Support for Golem API Definitions Closes #1178
/claim #1178

This PR provides OpenAPI export functionality and Swagger UI (Binding and CLI) support. Specifically, this PR implements:

Key Features:
- OpenAPI Export Endpoint: Added /v1/api-definitions/{id}/{version}/export endpoint to export API Definitions to OpenAPI Specifications.
- Full and Lossless Conversion: Implemented conversion logic for request and response types, accurate mapping of type to OpenAPI schema.
- Swagger UI Binding Type: A new "Swagger UI" binding type for routes in the Worker Gateway, allowing for serving of Swagger UI interfaces from API definitions when deployed.
- Swagger UI Integration: Code to render the exported OpenAPI schema as a Swagger UI for user-friendly API testing.
- Testing: Added basic unit for API to OpenAPI spec, tests include, verifying CORS, secuirty, parameter, input and output maping, I have also added end-to-end tests to validate Swagger UI.
- Golem CLI Updates: golem-cli needs to be updated with:
  - golem-cli api-definition swagger subcommand for launching a browser to a generated Swagger UI.
  - golem-cli api-definition export subcommand to export an API definition to an OpenAPI schema.
- Documentation: Documentation is required to explain these new features and will be provided in a separate PR to the docs repository.
- System/Integration Tests:
  - System tests to OpenAPI Spec from API, and attempting full circle by importing the OpenApi Schema,

System test and verify they interact correctly with the SWAGGER UI (Might not be possible currently all deployment tests fail.)

You can try this out, after building, by creating a new route with bindingtype swagger-ui, with path without any varriables, example /swagger . If deployed to localhost:9006, you will find your swagger ui there.

Your feedback is appreciated! While I work on the remaining task.

golem-worker-service-base/src/gateway_execution/gateway_http_input_executor.rs

Nanashi-lab · 2025-03-28T11:18:44Z

PR for CLI Part of the task is here
There are two new command under golem-cli api definition
swagger and export

golem-cli api definition export --id <id> --version <version> will download the open api spec in either json or yaml format, this will be the whole open api schema including input and output type

golem-cli api definition export --id <id> --version <version> --host <host> will open the webbrowser and navigate to the host (usually localhost:), if the api has deployments it will add the deployments in server, to test. else it will show the open api without any server information

example of the swagger ui usage in localhost:9900, it rightly shows 9006 (local deployment) as server for testing, it also doesnt show cors-binding or swagger ui binding. It only shows binding-type -> default

golem-worker-service-base/src/gateway_api_definition/http/api_oas_convert.rs

golem-worker-service-base/tests/api_oas_convert_tests.rs

Nanashi-lab · 2025-04-04T07:33:43Z

This is a link to repository containing, some wasm-files, api json files and the equivalent yaml output files.

Nanashi-lab · 2025-04-04T15:36:04Z

All Checks have passed, that's really good news.

golem-worker-service-base/tests/api_gateway_end_to_end_tests.rs

Nanashi-lab · 2025-04-11T08:36:57Z

I matched HEAD with current golem,
Few other changes are

Removed extract_path_parameters, we get path details directly input_mapping
Added test for first class worker
Added test for variant and query
Cleaned up the repeated test.

Nanashi-lab · 2025-04-23T00:30:45Z

One of my tests failed

ERROR test_export_import_api_definition{deps=EnvBasedTestDependencies}: golem_common::tracing: panic panic_info=panicked at integration-tests/tests/api/api_definition.rs:587:10:
called `Result::unwrap()` on an `Err` value: ApiDefinitionError { error: Some(BadRequest(ErrorsBody { errors: ["Invalid JSON"] })) }"

I will figure out why this happening. Couldnt replicate this locally. I split the test into one simple test for export endpoint and another for roundtrip test

Nanashi-lab · 2025-04-30T12:13:05Z

@jdegoes @vigoo Gentle Reminder -

Please review & offer feedback on this PR, which I submitted about a month ago. To ensure it's as easy as possible to review, I've rebased both golem and golem-cli with the latest main branch, so it is up-to-date.

Could you also let me know the current state of the ticket? Is the PR/Issue still something the team is actively interested in? Knowing this will help me prioritize accordingly.

I want to ensure I'm providing everything you need to make a decision on this PR. Please let me know what I can do to help move it forward.

To make this process smoother, I'm happy to:

Set up a live demo to walk you through the changes.
Provide any additional technical information or documentation you need.
Answer any questions you may have.

P.S. - The ignored roundtrip should be possible after #1569

vigoo · 2025-04-30T13:54:54Z

Is the PR/Issue still something the team is actively interested in?

Yes we definitely want this feature!

afsalthaj · 2025-05-15T22:14:56Z

@Nanashi-lab I am currently taking a look at this PR

Clean testframework, and swagger-ui handler Remove string scanning in api_oas_convert.rs Change add new more comprehensive test in api_oas_convert_tests Cargo fix

Test for default response

Remove extract_path_parameters Add support for query parameters Add test for first class workers Add test for query parameters and variant Cargo make fix

Add Getters for Component Name and Version Change ExportApiDefinition to ExportOpenapiSpec

Cargo Make Fix

golem-worker-service-base/src/gateway_api_definition/http/api_oas_convert.rs

afsalthaj · 2025-05-16T20:13:20Z

@Nanashi-lab I would request you to commit the fixes after I finish the review - I will let you know when, otherwise hard for me to review :D

It's looking good already, but just need a couple of more hours.

golem-worker-service-base/src/gateway_api_definition/http/api_oas_convert.rs

afsalthaj · 2025-05-16T20:26:58Z

golem-worker-service-base/src/gateway_api_definition/http/api_oas_convert.rs

+    }
+
+    // Fallback to default object schema if no body is found
+    create_default_object_schema()


I hope this is not by any accident :)
What is called as default_object_schema? That it expects empty object in the request body?
Couldn't we completely avoid a schema if there is none in the request.body?

afsalthaj · 2025-05-16T20:39:39Z

golem-worker-service-base/src/gateway_api_definition/http/api_oas_convert.rs

+            ))),
+            ..Default::default()
+        };
+        content.insert("application/json".to_string(), media);


I don't think it is application/json all the time

We do support application/text as well, and you can try it out as well.
If our rib script is as follows, and then deploy it by changing the content types

- method: GET path: /v1/{user}/contents binding: type: default componentName: "amazon:shopping-cart" response: | let user: u32 = request.path.user; let worker = instance("cart-${user}"); {headers: { Content-Type: "application/text" }, body: worker.get-string()}

The worker function merely returns String.

curl -X GET http://localhost:9006/v1/100/contents this is a string

And if it's json

{headers: { Content-Type: "application/json" }, body: worker.get-string()}

then,

curl -X GET http://localhost:9006/v1/100/contents "this is a string"

golem-worker-service-base/src/gateway_api_definition/http/api_oas_convert.rs

golem-worker-service-base/src/gateway_execution/swagger_binding_handler.rs

golem-worker-service-base/tests/api_oas_convert_tests.rs

afsalthaj

I have made some comments. There are a few things to fix before we could merge. I can see you are getting into various internals to reach this point, and pretty sure we can make it all the way if you could fix these comments soon :)

jdegoes · 2025-05-22T18:17:31Z

@Nanashi-lab Please re-open when 100% of all feedback has been addressed in the way indicated and you are ready for a fresh review. Thank you!

Nanashi-lab changed the title ~~OpenAPI Export for API Definition, Swagger UI Binding - Closes #1234~~ OpenAPI Export for API Definition, Swagger UI Binding - Closes #1178 Mar 28, 2025

Nanashi-lab closed this Mar 28, 2025

Nanashi-lab reopened this Mar 28, 2025

Nanashi-lab mentioned this pull request Mar 28, 2025

Develop OpenAPI Export for API Definition #1178

Closed

8 tasks

Nanashi-lab commented Mar 28, 2025

View reviewed changes

golem-worker-service-base/src/gateway_execution/gateway_http_input_executor.rs Show resolved Hide resolved

Nanashi-lab marked this pull request as ready for review March 28, 2025 04:21

Nanashi-lab mentioned this pull request Mar 28, 2025

Openapi export and swaggerui command golemcloud/golem-cli#159

Closed

jdegoes reviewed Mar 30, 2025

View reviewed changes

golem-worker-service-base/src/gateway_api_definition/http/api_oas_convert.rs Outdated Show resolved Hide resolved

jdegoes reviewed Mar 30, 2025

View reviewed changes

golem-worker-service-base/src/gateway_api_definition/http/api_oas_convert.rs Outdated Show resolved Hide resolved

jdegoes reviewed Mar 30, 2025

View reviewed changes

golem-worker-service-base/src/gateway_api_definition/http/api_oas_convert.rs Show resolved Hide resolved

jdegoes reviewed Mar 30, 2025

View reviewed changes

golem-worker-service-base/tests/api_oas_convert_tests.rs Outdated Show resolved Hide resolved

jdegoes reviewed Mar 30, 2025

View reviewed changes

golem-worker-service-base/tests/api_oas_convert_tests.rs Outdated Show resolved Hide resolved

jdegoes reviewed Mar 30, 2025

View reviewed changes

golem-worker-service-base/tests/api_oas_convert_tests.rs Outdated Show resolved Hide resolved

jdegoes reviewed Mar 30, 2025

View reviewed changes

golem-worker-service-base/tests/api_oas_convert_tests.rs Outdated Show resolved Hide resolved

Nanashi-lab commented Apr 4, 2025

View reviewed changes

golem-worker-service-base/tests/api_gateway_end_to_end_tests.rs Outdated Show resolved Hide resolved

algora-pbc bot added the 🙋 Bounty claim label Apr 10, 2025

Nanashi-lab force-pushed the openapi branch from dd07095 to a3d4897 Compare April 11, 2025 08:33

Nanashi-lab changed the title ~~OpenAPI Export for API Definition, Swagger UI Binding - Closes #1178~~ OpenAPI Export for API Definition, Swagger UI Binding Apr 12, 2025

Nanashi-lab force-pushed the openapi branch from a3d4897 to 4331622 Compare April 23, 2025 00:19

Nanashi-lab requested a review from jdegoes April 23, 2025 00:19

Nanashi-lab force-pushed the openapi branch from 634e201 to 4b29ba8 Compare April 24, 2025 10:48

Nanashi-lab mentioned this pull request Apr 24, 2025

Figure why Create Api Definition by Importing OpenAPI yaml only works in http-mode #1559

Closed

Nanashi-lab added 9 commits May 16, 2025 12:09

Refactor the code for api_oas_convert to make it more readable

6e1e3d2

Clean testframework, and swagger-ui handler Remove string scanning in api_oas_convert.rs Change add new more comprehensive test in api_oas_convert_tests Cargo fix

Add default response to OpenAPI

f727bf0

Test for default response

Match Head

dad41e9

Remove extract_path_parameters Add support for query parameters Add test for first class workers Add test for query parameters and variant Cargo make fix

Rebase & Improve Roundtrip Test

795e789

Add Simple Export test, Improve Roundtrip test

97b7cb2

Cargo make fix

9194ab7

Ignore Roundtrip test

9c3ffbd

Remove Custom Deserialize for GatewayBindingType

048b3ba

Add Getters for Component Name and Version Change ExportApiDefinition to ExportOpenapiSpec

Rebase & Cargo Make Fix

fa325c8

Nanashi-lab force-pushed the openapi branch from b88dc0e to fa325c8 Compare May 16, 2025 06:54

Nanashi-lab added 3 commits May 16, 2025 14:28

Minor Fixes

d4a2858

Fix Tests

c462dd4

Cargo Make Fix

Clean up Code

82fd56f