Skip to content

[Streams][Streamlang][API] Fully use meta({id}) to reuse schema partials in OAS output#259275

Merged
CoenWarmer merged 5 commits intoelastic:mainfrom
CoenWarmer:streams-api-schema-tweaks
Mar 24, 2026
Merged

[Streams][Streamlang][API] Fully use meta({id}) to reuse schema partials in OAS output#259275
CoenWarmer merged 5 commits intoelastic:mainfrom
CoenWarmer:streams-api-schema-tweaks

Conversation

@CoenWarmer
Copy link
Copy Markdown
Contributor

@CoenWarmer CoenWarmer commented Mar 24, 2026

Resolves #259277

Summary

Improves the Streams and Streamlang OAS output by eliminating deeply nested anyOf > anyOf > allOf structures in the generated spec. The root cause was that .meta({ id }) calls for most schemas lived only in the oas_definitions.ts registry files rather than at definition time. Because Zod v4's .meta() is immutable (it returns a new schema without modifying the original), the exported schema variables themselves were never registered — so every consumer inlined the full schema instead of emitting a $ref.

Root cause

// oas_definitions.ts — old pattern: .meta() result is stored but the original
// conditionSchema variable remains unregistered
Condition: conditionSchema.meta({ id: 'Condition' }),

When conditionSchema was used in routingDefinitionSchema.where, the OAS generator had no id to look up, so it inlined the full recursive union. The fix is to call .meta({ id }) at definition time so the exported variable itself is the registered schema.

Before / after

PUT /api/streams/{name} request body

# Before
schema:
  anyOf:
    - anyOf:
        - allOf: ...

# After
schema:
  $ref: '#/components/schemas/StreamUpsertRequest'

GET /api/streams/{name} response

# Before: three large inline objects
anyOf:
  - type: object
    properties: { ... hundreds of lines ... }
# After
$ref: '#/components/schemas/StreamGetResponse'

@CoenWarmer CoenWarmer requested review from a team as code owners March 24, 2026 09:17
@CoenWarmer CoenWarmer added release_note:skip Skip the PR/issue when compiling release notes backport:skip This PR does not require backporting labels Mar 24, 2026
@rStelmach rStelmach self-assigned this Mar 24, 2026
@elastic-vault-github-plugin-prod elastic-vault-github-plugin-prod bot requested a review from a team as a code owner March 24, 2026 09:47
Copy link
Copy Markdown
Contributor

@tonyghiani tonyghiani left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Great reduction of the OAS output size 👏

Copy link
Copy Markdown
Contributor

@rStelmach rStelmach left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, Thank you for this change !

@elasticmachine
Copy link
Copy Markdown
Contributor

💔 Build Failed

Failed CI Steps

Metrics [docs]

Async chunks

Total size of all lazy-loaded chunks that will be downloaded as the user navigates the app

id before after diff
datasetQuality 570.1KB 571.0KB +971.0B
streamsApp 1.8MB 1.8MB +1.0KB
total +2.0KB

cc @rStelmach

@CoenWarmer CoenWarmer requested a review from a team as a code owner March 24, 2026 11:05
Copy link
Copy Markdown
Member

@florent-leborgne florent-leborgne left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM for docs

Copy link
Copy Markdown
Contributor

@Bamieh Bamieh left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

core change in the openapispec zod converter LGTM, deleting the id as it is not allowed in the json schema node. i've read through the original bug issue and this seems to be the fix for it

@CoenWarmer CoenWarmer enabled auto-merge (squash) March 24, 2026 12:00
@CoenWarmer CoenWarmer merged commit b99a7ed into elastic:main Mar 24, 2026
18 checks passed
mbondyra added a commit to mbondyra/kibana that referenced this pull request Mar 24, 2026
…ra/kibana into dashboard_align_attachment_to_api

* 'dashboard_align_attachment_to_api' of github.com:mbondyra/kibana: (45 commits)
  [OTel Tracing] HTTP instrumentation (elastic#258663)
  Replace deprecated EUI icons in files owned by @elastic/ml-ui (elastic#255624)
  [Codeowners] add missing codeowners for security_solution_api_integration tests (elastic#259223)
  [CI] fix bad imports that came from a merge-race (elastic#259383)
  Add `.claude/worktrees/` to `.gitignore` (elastic#259192)
  Improve unknown-key validation error message in @kbn/config-schema (elastic#258633)
  [ML] Update Security ML jobs to use entity analytics fields for host and user fields (elastic#255339)
  [Table sweep] Update table columns responsiveness in Index Management and Dashboards (elastic#259340)
  skip failing test suite (elastic#258790)
  skip failing test suite (elastic#259261)
  chore: util to clean cached images (elastic#259335)
  [Entity Store] Use last_seen for automated resolution watermark (elastic#258574)
  [One Workflow] Fix flaky alert trigger Scout test by removing order-dependent assertions (elastic#259299)
  Skip serverless Discover request counts tests for MKI (elastic#259333)
  [Security Solution] render header title in new document flyout in Security Solution and Discover (elastic#258166)
  [Agent Builder] register inference endpoint feature (elastic#259259)
  [Agent Builder] Skills Command Menu - Add descriptions and scope options to agent (elastic#258964)
  [Streams][Streamlang][API] Fully use meta({id}) to reuse schema partials in OAS output (elastic#259275)
  fix(files_example): add tableCaption to EuiInMemoryTable for a11y (elastic#258289)
  [Entity Store] Adding list endpoint with query filter (elastic#258320)
  ...
jeramysoucy pushed a commit to jeramysoucy/kibana that referenced this pull request Mar 26, 2026
…als in OAS output (elastic#259275)

Resolves elastic#259277

## Summary

Improves the Streams and Streamlang OAS output by eliminating deeply
nested `anyOf > anyOf > allOf` structures in the generated spec. The
root cause was that `.meta({ id })` calls for most schemas lived only in
the `oas_definitions.ts` registry files rather than at definition time.
Because Zod v4's `.meta()` is immutable (it returns a new schema without
modifying the original), the exported schema variables themselves were
never registered — so every consumer inlined the full schema instead of
emitting a `$ref`.

## Root cause

```typescript
// oas_definitions.ts — old pattern: .meta() result is stored but the original
// conditionSchema variable remains unregistered
Condition: conditionSchema.meta({ id: 'Condition' }),
```

When conditionSchema was used in `routingDefinitionSchema.where`, the
OAS generator had no id to look up, so it inlined the full recursive
union. The fix is to call `.meta({ id })` at definition time so the
exported variable itself is the registered schema.

## Before / after

`PUT /api/streams/{name}` request body

```
# Before
schema:
  anyOf:
    - anyOf:
        - allOf: ...

# After
schema:
  $ref: '#/components/schemas/StreamUpsertRequest'
```

`GET /api/streams/{name}` response

```
# Before: three large inline objects
anyOf:
  - type: object
    properties: { ... hundreds of lines ... }
# After
$ref: '#/components/schemas/StreamGetResponse'
```

---------

Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

backport:skip This PR does not require backporting release_note:skip Skip the PR/issue when compiling release notes v9.4.0

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Streams] OAS spec for Streams API has deeply nested anyOf > anyOf > allOf structures

7 participants