[Streams 🌊] Enable `/api/streams/{id}/_group` endpoints for GroupStreams by simianhacker · Pull Request #210114 · elastic/kibana

simianhacker · 2025-02-06T22:17:28Z

Summary

This PR implements 2 endpoints as a follow up to #208126 for working directly with the group object for GroupStreamDefinition:

PUT /api/streams/{id}/_group
GET /api/streams/{id}/_group

x-pack/solutions/observability/plugins/streams/server/routes/streams/group/route.ts

flash1293

The _group endpoint allows to overwrite any stream to a group stream:

We should have some validation in there to make sure the type isn't changed.

simianhacker · 2025-02-09T07:09:53Z

Oops... good catch. Did we miss that before?

flash1293 · 2025-02-10T11:08:24Z

There is a check that prevents wired/unwired switches somewhere, but I think we did for group<->ingest streams. We should probaly also update the _ingest endpoint in a similar way. This also goes back a bit to https://github.com/elastic/streams-program/discussions/147 , but let's not block on a decision on that :)

…endpoint

flash1293

The particular case that's mentioned is fixed, but I found a new one... if there is a classic stream occupying a name already, it will still save the group stream:

POST metrics-test-test/_doc
{
  "@timestamp": "2025"
}

# unwired stream
GET kbn:/api/streams/metrics-test-test

# create as group stream
PUT kbn:/api/streams/metrics-test-test
{
  "stream": {
    "group": {
      "members": ["logs.child1"]
    }
  },
  "dashboards": []
}

# returns group stream, unwired stream unaccessible
GET kbn:/api/streams/metrics-test-test

…endpoint

elasticmachine · 2025-02-19T20:38:19Z

💛 Build succeeded, but was flaky

Buildkite Build
Commit: 6648a8f

Failed CI Steps

Jest Integration Tests #3

Test Failures

[job] [logs] Jest Integration Tests #3 / unrecognized task types should be no workload aggregator errors when there are removed task types

Metrics [docs]

Public APIs missing comments

Total count of every public API that lacks a comment. Target amount is 0. Run node scripts/build_api_docs --plugin [yourplugin] --stats comments for more detailed information.

id	before	after	diff
`@kbn/streams-schema`	266	269	+3

Async chunks

Total size of all lazy-loaded chunks that will be downloaded as the user navigates the app

id	before	after	diff
`streamsApp`	287.4KB	287.4KB	+24.0B

Unknown metric groups

API count

id	before	after	diff
`@kbn/streams-schema`	269	273	+4

History

💚 Build #277797 succeeded af87994
💚 Build #277477 succeeded 9196ac1
💔 Build #277402 failed 25d324e
💚 Build #276504 succeeded 0dcf0b4
💛 Build #275556 was flaky 3b998fc
💛 Build #274157 was flaky 186c764

flash1293

LGTM

flash1293 · 2025-02-20T13:54:42Z

x-pack/solutions/observability/plugins/streams/server/routes/streams/crud/route.ts

@@ -166,11 +166,23 @@ export const editStreamRoute = createServerRoute({

    if (
      isGroupStreamDefinition({ ...params.body.stream, name: params.path.name }) &&


To make sure I get it right - this branch is just to make typescript happy, right?

kibanamachine · 2025-02-20T16:03:23Z

Starting backport for target branches: 8.x, 9.0

https://github.com/elastic/kibana/actions/runs/13439488930

kibanamachine · 2025-02-20T16:08:52Z

💔 All backports failed

Status	Branch	Result
❌	8.x	Backport failed because of merge conflicts You might need to backport the following PRs to 8.x: - [Streams 🌊] Ensure the members array is unique for GroupStreamDefinitions (#210089)
❌	9.0	Backport failed because of merge conflicts You might need to backport the following PRs to 9.0: - [Streams 🌊] Ensure the members array is unique for GroupStreamDefinitions (#210089)

Manual backport

To create the backport manually run:

node scripts/backport --pr 210114

Questions ?

Please refer to the Backport tool documentation

simianhacker · 2025-02-21T16:11:58Z

💚 All backports created successfully

Status	Branch	Result
✅	8.x

Note: Successful backport PRs will be merged automatically after passing CI.

Questions ?

Please refer to the Backport tool documentation

…ams (elastic#210114) ## Summary This PR implements 2 endpoints as a follow up to elastic#208126 for working directly with the `group` object for `GroupStreamDefinition`: - `PUT /api/streams/{id}/_group` - `GET /api/streams/{id}/_group` --------- Co-authored-by: Joe Reuter <johannes.reuter@elastic.co> (cherry picked from commit 8c90076)

kibanamachine · 2025-02-21T16:49:00Z

Looks like this PR has a backport PR but it still hasn't been merged. Please merge it ASAP to keep the branches relatively in sync.

kibanamachine · 2025-02-24T16:49:11Z

Looks like this PR has a backport PR but it still hasn't been merged. Please merge it ASAP to keep the branches relatively in sync.

…upStreams (#210114) (#212114) # Backport This will backport the following commits from `main` to `8.x`: - [[Streams 🌊] Enable `/api/streams/{id}/_group` endpoints for GroupStreams (#210114)](#210114)  ### Questions ? Please refer to the [Backport tool documentation](https://github.com/sorenlouv/backport)

…upStreams (elastic#210114) (elastic#212114) # Backport This will backport the following commits from `main` to `8.x`: - [[Streams 🌊] Enable `/api/streams/{id}/_group` endpoints for GroupStreams (elastic#210114)](elastic#210114)  ### Questions ? Please refer to the [Backport tool documentation](https://github.com/sorenlouv/backport)

…ams (elastic#210114) ## Summary This PR implements 2 endpoints as a follow up to elastic#208126 for working directly with the `group` object for `GroupStreamDefinition`: - `PUT /api/streams/{id}/_group` - `GET /api/streams/{id}/_group` --------- Co-authored-by: Joe Reuter <johannes.reuter@elastic.co>

[Streams] Enable _group endpoint for GroupStreams

a8ef280

simianhacker added release_note:enhancement v9.0.0 backport:version Backport to applied version labels Feature:Streams This is the label for the Streams Project v9.1.0 v8.19.0 labels Feb 6, 2025

removing extraneous types

186c764

simianhacker marked this pull request as ready for review February 6, 2025 22:20

simianhacker requested a review from a team as a code owner February 6, 2025 22:20

simianhacker changed the title ~~[Streams 🌊] Enable _group endpoint for GroupStreams~~ [Streams 🌊] Enable /api/streams/{id}/_group endpoints for GroupStreams Feb 6, 2025

flash1293 reviewed Feb 7, 2025

View reviewed changes

x-pack/solutions/observability/plugins/streams/server/routes/streams/group/route.ts Outdated Show resolved Hide resolved

flash1293 reviewed Feb 7, 2025

View reviewed changes

simianhacker added 3 commits February 10, 2025 08:15

Merge branch 'main' of github.com:elastic/kibana into streams/_group-…

0680df8

…endpoint

Ensure group streams do not start with logs

ccb0b5b

Merge branch 'main' of github.com:elastic/kibana into streams/_group-…

3b998fc

…endpoint

simianhacker requested a review from flash1293 February 12, 2025 13:33

Merge branch 'main' into streams/_group-endpoint

0dcf0b4

flash1293 reviewed Feb 14, 2025

View reviewed changes

simianhacker added 4 commits February 18, 2025 11:26

Fix the ingestToGroup check

9c0d960

Updating tests with validation scenarios

dc13e14

Merge branch 'main' of github.com:elastic/kibana into streams/_group-…

25d324e

…endpoint

Merge branch 'main' of github.com:elastic/kibana into streams/_group-…

9196ac1

…endpoint

simianhacker requested a review from flash1293 February 18, 2025 22:33

simianhacker added 4 commits February 19, 2025 09:14

Merge branch 'main' of github.com:elastic/kibana into streams/_group-…

a2f021a

…endpoint

Merge branch 'main' of github.com:elastic/kibana into streams/_group-…

af87994

…endpoint

Adding name check to _group endpoint; changing id to name

a0fe64e

Merge branch 'main' of github.com:elastic/kibana into streams/_group-…

6648a8f

…endpoint

flash1293 approved these changes Feb 20, 2025

View reviewed changes

simianhacker merged commit 8c90076 into elastic:main Feb 20, 2025

simianhacker removed the v9.0.0 label Feb 20, 2025

kibanamachine mentioned this pull request Feb 21, 2025

[Security Solution] Extend rule upgrade integration tests for prebuilt rule customization feature #209260

Merged

simianhacker mentioned this pull request Feb 21, 2025

[8.x] [Streams 🌊] Enable /api/streams/{id}/_group endpoints for GroupStreams (#210114) #212114

Merged

kibanamachine added the backport missing Added to PRs automatically when the are determined to be missing a backport. label Feb 21, 2025

kibanamachine removed the backport missing Added to PRs automatically when the are determined to be missing a backport. label Feb 25, 2025

		@@ -166,11 +166,23 @@ export const editStreamRoute = createServerRoute({

		if (
		isGroupStreamDefinition({ ...params.body.stream, name: params.path.name }) &&

Conversation

simianhacker commented Feb 6, 2025 • edited by kibanamachine Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Uh oh!

Uh oh!

flash1293 left a comment

Choose a reason for hiding this comment

Uh oh!

simianhacker commented Feb 9, 2025

Uh oh!

flash1293 commented Feb 10, 2025

Uh oh!

flash1293 left a comment

Choose a reason for hiding this comment

Uh oh!

elasticmachine commented Feb 19, 2025

💛 Build succeeded, but was flaky

Failed CI Steps

Test Failures

Metrics [docs]

Public APIs missing comments

Async chunks

API count

History

Uh oh!

flash1293 left a comment

Choose a reason for hiding this comment

Uh oh!

flash1293 Feb 20, 2025

Choose a reason for hiding this comment

Uh oh!

simianhacker Feb 20, 2025

Choose a reason for hiding this comment

Uh oh!

kibanamachine commented Feb 20, 2025

Uh oh!

kibanamachine commented Feb 20, 2025

💔 All backports failed

Manual backport

Questions ?

Uh oh!

simianhacker commented Feb 21, 2025

💚 All backports created successfully

Questions ?

Uh oh!

kibanamachine commented Feb 21, 2025

Uh oh!

kibanamachine commented Feb 24, 2025

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

simianhacker commented Feb 6, 2025 •

edited by kibanamachine

Loading