Migrate to libopenapi #1388

yozel · 2023-12-18T12:04:03Z

This pull request is a complete migration to libopenapi. Almost the tests and examples generates the same code, except embedded swaggerSpec files. While semantically its the same, due to change of order of object keys in the Json, it generates a different gzipped base64.

There is only one tests that fails, which is internal/test/issues/issue-removed-external-ref. And I believe it's related to this: pb33f/libopenapi#219. I'm following/contributing to the thread to solve the issue.

The generated code continues to use kin-api since validation logic depends on middleware repos, which still uses kin-api.

Test results:

$ go test ./...
?   	github.com/deepmap/oapi-codegen/v2/pkg/ecdsafile	[no test files]
?   	github.com/deepmap/oapi-codegen/v2/pkg/securityprovider	[no test files]
ok  	github.com/deepmap/oapi-codegen/v2/cmd/oapi-codegen	0.733s
ok  	github.com/deepmap/oapi-codegen/v2/pkg/codegen	6.210s
ok  	github.com/deepmap/oapi-codegen/v2/pkg/util	(cached)

The MR also includes regenerated *.gen.go files, so you can compare the difference.

$ cd ./internal/test
$ go generate -v ./...
all_of/doc.go
all_of/v1/openapi.gen.go
all_of/v2/openapi.gen.go
any_of/param/doc.go
any_of/param/param.gen.go
any_of/param/param_test.go
client/client.gen.go
client/client_test.go
client/doc.go
components/components.gen.go
components/components_test.go
components/doc.go
externalref/doc.go
externalref/externalref.gen.go
externalref/imports_test.go
externalref/packageA/doc.go
externalref/packageA/externalref.gen.go
externalref/packageB/doc.go
externalref/packageB/externalref.gen.go
issues/issue-1087/api.gen.go
issues/issue-1087/doc.go
issues/issue-1087/deps/deps.gen.go
issues/issue-1087/deps/doc.go
issues/issue-1093/doc.go
issues/issue-1093/issue_test.go
issues/issue-1093/api/child/child.gen.go
issues/issue-1093/api/parent/parent.gen.go
issues/issue-1127/api.gen.go
issues/issue-1127/doc.go
issues/issue-1168/api.gen.go
issues/issue-1168/doc.go
issues/issue-1180/doc.go
issues/issue-1180/issue.gen.go
issues/issue-1180/issue_test.go
issues/issue-1182/pkg1/doc.go
issues/issue-1182/pkg1/pkg1.gen.go
issues/issue-1182/pkg2/doc.go
issues/issue-1182/pkg2/pkg2.gen.go
issues/issue-1189/doc.go
issues/issue-1189/issue1189.gen.go
issues/issue-1208-1209/doc.go
issues/issue-1208-1209/issue-multi-json.gen.go
issues/issue-1208-1209/issue-multi-json_test.go
issues/issue-1212/pkg1/doc.go
issues/issue-1212/pkg1/pkg1.gen.go
issues/issue-1212/pkg2/doc.go
issues/issue-1212/pkg2/pkg2.gen.go
issues/issue-1219/doc.go
issues/issue-1219/issue.gen.go
issues/issue-1219/issue_test.go
issues/issue-1298/doc.go
issues/issue-1298/issue1298.gen.go
issues/issue-1298/issue1298_test.go
issues/issue-312/doc.go
issues/issue-312/issue.gen.go
issues/issue-312/issue_test.go
issues/issue-52/doc.go
issues/issue-52/issue.gen.go
issues/issue-52/issue_test.go
issues/issue-579/gen.go
issues/issue-579/issue.gen.go
issues/issue-579/issue_test.go
issues/issue-832/generate.go
issues/issue-832/issue.gen.go
issues/issue-936/api.gen.go
issues/issue-936/gen.go
issues/issue-grab_import_names/doc.go
issues/issue-grab_import_names/issue.gen.go
issues/issue-grab_import_names/issue_test.go
issues/issue-head-digit-of-httpheader/doc.go
issues/issue-head-digit-of-httpheader/issue.gen.go
issues/issue-head-digit-of-operation-id/doc.go
issues/issue-head-digit-of-operation-id/issue.gen.go
issues/issue-illegal_enum_names/doc.go
issues/issue-illegal_enum_names/issue.gen.go
issues/issue-illegal_enum_names/issue_test.go
issues/issue-removed-external-ref/doc.go
issues/issue-removed-external-ref/gen/spec_base/issue.gen.go
issues/issue-removed-external-ref/gen/spec_ext/issue.gen.go
parameters/doc.go
parameters/parameters.gen.go
parameters/parameters_test.go
schemas/doc.go
schemas/schemas.gen.go
server/doc.go
server/server.gen.go
server/server_test.go
strict-server/strict_test.go
strict-server/chi/server.gen.go
strict-server/chi/server.go
strict-server/chi/types.gen.go
strict-server/client/client.gen.go
strict-server/client/client.go
strict-server/echo/server.gen.go
strict-server/echo/server.go
strict-server/echo/types.gen.go
strict-server/fiber/server.gen.go
strict-server/fiber/server.go
strict-server/fiber/types.gen.go
strict-server/gin/server.gen.go
strict-server/gin/server.go
strict-server/gin/types.gen.go
strict-server/gorilla/server.gen.go
strict-server/gorilla/server.go
strict-server/gorilla/types.gen.go
strict-server/iris/server.gen.go
strict-server/iris/server.go
strict-server/iris/types.gen.go

$ cd examples
$ go generate -v ./...
authenticated-api/echo/main.go
authenticated-api/echo/api/api.gen.go
authenticated-api/echo/api/doc.go
authenticated-api/echo/server/fake_jws.go
authenticated-api/echo/server/jwt_authenticator.go
authenticated-api/echo/server/server.go
authenticated-api/echo/server/server_test.go
custom-client-type/custom-client-type.gen.go
custom-client-type/doc.go
no-vcs-version-override/echo/api/api.gen.go
no-vcs-version-override/echo/api/doc.go
petstore-expanded/petstore-client.gen.go
petstore-expanded/chi/petstore.go
petstore-expanded/chi/petstore_test.go
petstore-expanded/chi/api/petstore.gen.go
petstore-expanded/chi/api/petstore.go
petstore-expanded/echo/petstore.go
petstore-expanded/echo/petstore_test.go
petstore-expanded/echo/pkg_codegen_petstore_test.go
petstore-expanded/echo/api/petstore-server.gen.go
petstore-expanded/echo/api/petstore.go
petstore-expanded/echo/api/models/models.gen.go
petstore-expanded/fiber/petstore.go
petstore-expanded/fiber/petstore_test.go
petstore-expanded/fiber/api/petstore-server.gen.go
petstore-expanded/fiber/api/petstore-types.gen.go
petstore-expanded/fiber/api/petstore.go
petstore-expanded/gin/petstore.go
petstore-expanded/gin/petstore_test.go
petstore-expanded/gin/api/petstore-server.gen.go
petstore-expanded/gin/api/petstore-types.gen.go
petstore-expanded/gin/api/petstore.go
petstore-expanded/gorilla/petstore.go
petstore-expanded/gorilla/petstore_test.go
petstore-expanded/gorilla/api/petstore.gen.go
petstore-expanded/gorilla/api/petstore.go
petstore-expanded/internal/doc.go
petstore-expanded/iris/petstore.go
petstore-expanded/iris/petstore_test.go
petstore-expanded/iris/api/petstore-server.gen.go
petstore-expanded/iris/api/petstore-types.gen.go
petstore-expanded/iris/api/petstore.go
petstore-expanded/strict/petstore.go
petstore-expanded/strict/petstore_test.go
petstore-expanded/strict/api/petstore-server.gen.go
petstore-expanded/strict/api/petstore-types.gen.go
petstore-expanded/strict/api/petstore.go

jamietanna · 2023-12-18T16:41:38Z

Hey, thanks very much for this PR! This is something I'd started looking at in #1211 but put on hold.

I'll try and have a look at it in a couple of weeks, as I'll need a bit of time to go through all the changes. Appreciate the hard work 🙌

jamietanna · 2023-12-18T16:42:14Z

I notice your email address isn't associated with your GitHub account - mind doing that, so it shows correctly in GitHub? (Right now 6860428 doesn't have your avatar next to it)

yozel · 2023-12-18T16:44:53Z

@jamietanna thanks for noticing. I was not aware of that, now I added my work email into my account and it shows my avatar.

jamietanna · 2023-12-18T16:46:10Z

pkg/codegen/codegen.go

 // the descriptions we've built up above from the schema objects.
 // opts defines
-func Generate(spec *openapi3.T, opts Configuration) (string, error) {
+func Generate(spec *libopenapi.DocumentModel[v3.Document], opts Configuration) (string, error) {


The codegen package is used by external users, so this would be a breaking change - thoughts on how we could look at making this a non-breaking change? Perhaps introducing a new package i.e. pkg/codegen/openapi?

I changed the code to not to modify anything under pkg/* directories, instead created a new package under pkg/codegen/openapi

jamietanna · 2023-12-18T16:46:48Z

Labelling the implementation right now as breaking - we'll look at what we can do to make it non-breaking if possible 🤞

yozel · 2023-12-18T17:48:25Z

There's a failing test:

--- FAIL: TestGetSwagger (0.00s)
    imports_test.go:27: 
                Error Trace:    /Users/yozel/Projects/oapi-codegen/internal/test/externalref/imports_test.go:27
                Error:          Expected nil, but got: &errors.errorString{s:"path not found: object_c.json"}
                Test:           TestGetSwagger

libopenapi doesn't have InternalizeRefs() which results with externally referencing spec embeddings. I'm also working on that issue (pb33f/libopenapi#222)

zekth · 2024-02-22T12:08:47Z

Any update on this?

SimonAM · 2024-04-21T03:14:16Z

There's a failing test:
--- FAIL: TestGetSwagger (0.00s)
    imports_test.go:27: 
                Error Trace:    /Users/yozel/Projects/oapi-codegen/internal/test/externalref/imports_test.go:27
                Error:          Expected nil, but got: &errors.errorString{s:"path not found: object_c.json"}
                Test:           TestGetSwagger
libopenapi doesn't have InternalizeRefs() which results with externally referencing spec embeddings. I'm also working on that issue (pb33f/libopenapi#222)

The issue is resolved now

amahdhaoui-ionos · 2024-05-02T09:17:26Z

Hi! Is this PR still active? I was considering testing the branch as suggested by #373 (comment)

TerminalFi · 2024-05-25T14:16:54Z

Testing this branch, I run into this, however when doing 3.1.0 validation it shows these is a valid spec

error generating code: error generating type definitions: error generating code for type definitions: duplicate typename 'DataStoreExtended_ConnectorEnvironment' detected, can't auto-rename, please use x-go-name to specify your own name for one of them

It is basically error on these and saying these are the same? Even though one has a single _ and the other has two __

But then even removing these,

It errors out again

: api.go:108456:28: '_' must separate successive digits (and 3 more errors)

Which seems like its because that struct type never got defined

mattoni · 2024-09-02T01:18:15Z

What's the best way to test this? I'm getting a lot of errors with compilation when overriding my go.mod file with a require and the local repo set to the correct branch:

replace github.com/deepmap/oapi-codegen/v2 v2.2.0 => ../../../../../Github/oapi-codegen

gives a bunch of errors such as

deepmap/oapi-codegen/v2@v2.2.0/pkg/codegen/merge_schemas.go:123:13: s1.Type.Slice undefined (type string has no field or method Slice)

Penthious · 2024-11-22T05:33:38Z

Any plans to make a v3 tag with this breaking change?

jamietanna · 2024-11-22T08:53:27Z

Any plans to make a v3 tag with this breaking change?

Answered in more depth in #373 (comment)

yozel force-pushed the migrate-to-libopenapi branch 7 times, most recently from 125b654 to 6860428 Compare December 18, 2023 16:31

yozel mentioned this pull request Dec 18, 2023

lib doesn't resolve multi file spec pb33f/libopenapi#219

Closed

Migrate to libopenapi

09ae9ad

yozel force-pushed the migrate-to-libopenapi branch from 6860428 to 09ae9ad Compare December 18, 2023 16:42

jamietanna reviewed Dec 18, 2023

View reviewed changes

jamietanna added the ☢️ breaking change This change would break existing users' code label Dec 18, 2023

jamietanna added the enhancement New feature or request label Dec 18, 2023

yozel added 2 commits December 18, 2023 18:06

Do not touch existing packages instead use new package

7bb8bdc

tmp

848b848

jamietanna mentioned this pull request Feb 28, 2024

OpenAPI 3.1 support? #373

Open

jamietanna mentioned this pull request May 5, 2024

feat: add an engine or openapi-library configuration item #1593

Open

diamondburned mentioned this pull request Aug 12, 2024

Generate schema properties in the original order #1722

Draft

jamietanna mentioned this pull request Nov 3, 2024

Organise code in packages #1820

Closed

Uh oh!

Migrate to libopenapi #1388

Are you sure you want to change the base?

Migrate to libopenapi #1388

Uh oh!

Conversation

yozel commented Dec 18, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

jamietanna commented Dec 18, 2023

Uh oh!

jamietanna commented Dec 18, 2023

Uh oh!

yozel commented Dec 18, 2023

Uh oh!

jamietanna Dec 18, 2023

Choose a reason for hiding this comment

Uh oh!

yozel Dec 18, 2023

Choose a reason for hiding this comment

Uh oh!

jamietanna commented Dec 18, 2023

Uh oh!

yozel commented Dec 18, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

zekth commented Feb 22, 2024

Uh oh!

SimonAM commented Apr 21, 2024

Uh oh!

amahdhaoui-ionos commented May 2, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

TerminalFi commented May 25, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

mattoni commented Sep 2, 2024

Uh oh!

Penthious commented Nov 22, 2024

Uh oh!

jamietanna commented Nov 22, 2024

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

8 participants

yozel commented Dec 18, 2023 •

edited

Loading

yozel commented Dec 18, 2023 •

edited

Loading

amahdhaoui-ionos commented May 2, 2024 •

edited

Loading

TerminalFi commented May 25, 2024 •

edited

Loading