Skip to content

feat: create a migration infrastructure for context and configuration v1=>v2#1014

Merged
Skarlso merged 21 commits into
open-component-model:mainfrom
Skarlso:migrate-config-and-context
Oct 15, 2025
Merged

feat: create a migration infrastructure for context and configuration v1=>v2#1014
Skarlso merged 21 commits into
open-component-model:mainfrom
Skarlso:migrate-config-and-context

Conversation

@Skarlso

@Skarlso Skarlso commented Oct 7, 2025

Copy link
Copy Markdown
Contributor

What this PR does / why we need it

Towards open-component-model/ocm-project#626

This is the first step in migrating the configuration from v1=>v2. This adds the infrastructure to begin to convert to the configuration in v2.

Which issue(s) this PR fixes

@Skarlso Skarlso requested a review from a team as a code owner October 7, 2025 12:52
@github-actions github-actions Bot added kind/feature new feature, enhancement, improvement, extension size/l Large labels Oct 7, 2025
@Skarlso Skarlso force-pushed the migrate-config-and-context branch from 0f21707 to 74b1d95 Compare October 7, 2025 12:57
… v1=>v2

Signed-off-by: Gergely Brautigam <182850+Skarlso@users.noreply.github.com>

On-behalf-of: Gergely Brautigam <gergely.brautigam@sap.com>
@Skarlso Skarlso force-pushed the migrate-config-and-context branch from 74b1d95 to 5783c74 Compare October 7, 2025 13:17
jakobmoellerdev and others added 3 commits October 7, 2025 15:30
On-behalf-of: Gergely Brautigam <gergely.brautigam@sap.com>

Signed-off-by: Gergely Brautigam <182850+Skarlso@users.noreply.github.com>
…component-model into migrate-config-and-context

@jakobmoellerdev jakobmoellerdev left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Im currently missing a hash method for the configs to accomodate our phase 2 which I would like to include in this PR as it fits together. Apart from smaller nits, conceptually this is looking great!

I did wonder about efficient sorting of cfgs though, so maybe thats worth checking again.

Also: Is flatmap stable?

Comment thread kubernetes/controller/go.mod
Comment thread kubernetes/controller/internal/configuration/config.go
Comment thread kubernetes/controller/internal/configuration/config.go Outdated
Comment thread kubernetes/controller/internal/configuration/config.go Outdated
Comment thread kubernetes/controller/internal/configuration/config.go Outdated
Comment thread kubernetes/controller/internal/setup/repository.go
Comment thread kubernetes/controller/internal/setup/repository.go Outdated
@Skarlso

Skarlso commented Oct 8, 2025

Copy link
Copy Markdown
Contributor Author

Also: Is flatmap stable?

That's a good question. :D

On-behalf-of: Gergely Brautigam <gergely.brautigam@sap.com>

Signed-off-by: Gergely Brautigam <182850+Skarlso@users.noreply.github.com>
Comment thread kubernetes/controller/internal/configuration/config.go Outdated
Skarlso and others added 2 commits October 8, 2025 12:58
Signed-off-by: Gergely Brautigam <gergely.brautigam@sap.com>
On-behalf-of: Gergely Brautigam <gergely.brautigam@sap.com>

Signed-off-by: Gergely Brautigam <182850+Skarlso@users.noreply.github.com>
@Skarlso

Skarlso commented Oct 8, 2025

Copy link
Copy Markdown
Contributor Author

I know I need to do some refactoring that method is huge. I'm going to do it in a bit.

On-behalf-of: Gergely Brautigam <gergely.brautigam@sap.com>

Signed-off-by: Gergely Brautigam <182850+Skarlso@users.noreply.github.com>
@Skarlso Skarlso force-pushed the migrate-config-and-context branch from 515506d to d721488 Compare October 8, 2025 12:34
On-behalf-of: Gergely Brautigam <gergely.brautigam@sap.com>

Signed-off-by: Gergely Brautigam <182850+Skarlso@users.noreply.github.com>
On-behalf-of: Gergely Brautigam <gergely.brautigam@sap.com>

Signed-off-by: Gergely Brautigam <182850+Skarlso@users.noreply.github.com>
@Skarlso

Skarlso commented Oct 8, 2025

Copy link
Copy Markdown
Contributor Author

@jakobmoellerdev So now I added hashing to the sorting, but that's not what you mean here, right:

Im currently missing a hash method for the configs to accomodate our phase 2

But this exists, what other kind of hashing did you have in mind?

@jakobmoellerdev

Copy link
Copy Markdown
Member

Im referring to the hash that can be used to uniquely key config content so that we can use it in the resolution service to determine if we need to initialize a new repository

@Skarlso

Skarlso commented Oct 9, 2025

Copy link
Copy Markdown
Contributor Author

Im referring to the hash that can be used to uniquely key config content so that we can use it in the resolution service to determine if we need to initialize a new repository

I'm sorry, I might be stupid, but I don't understand how what you are saying here is different from what exists right now that I'm already using to produce a hash for the config data. :D Could you please elaborate the difference?

@jakobmoellerdev

Copy link
Copy Markdown
Member

I believe that we should epxose the hash you are already generating here: https://github.com/open-component-model/open-component-model/pull/1014/files#diff-a6095938c6975e7550d720d60b23d93bdd2cbbe3155b9d72b79431a4de1037d8R72

This will be needed because otherwise consumers like the resolution service will have to compute it again from scratch. To be exact, I dont really understand why you use the hash to sort. In the end the input needs to be sorted and the output needs to be a hash. You are currently hashing all objects, and then sorting the objects by hash. That is not really the same though for me (maybe Im stupid as well :D )

…ration order for objects

On-behalf-of: Gergely Brautigam <gergely.brautigam@sap.com>

Signed-off-by: Gergely Brautigam <182850+Skarlso@users.noreply.github.com>
On-behalf-of: Gergely Brautigam <gergely.brautigam@sap.com>

Signed-off-by: Gergely Brautigam <182850+Skarlso@users.noreply.github.com>
Comment thread kubernetes/controller/internal/configuration/config_test.go Outdated
@Skarlso Skarlso force-pushed the migrate-config-and-context branch from d564702 to a3e5576 Compare October 9, 2025 15:26
@Skarlso

Skarlso commented Oct 9, 2025

Copy link
Copy Markdown
Contributor Author

For the reference on the above comment. We talked about this on Slack, and now I understand what Jakob is saying. :D

Basically, since the original Hasher hashes all objects and all their Data segment including any random key they might provide, this is insufficient.

However, now, given the flatmap that is basically a stable representation of the config compacted. This is now easy to hash. We can just marshal it, and run it through sha256 and done. That's what we are doing now. The configuration order is based on the reference declaration in the original object spec.

We also parallelize the Get calls to make it a bit more performant since this is a very critical code section.

On-behalf-of: Gergely Brautigam <gergely.brautigam@sap.com>

Signed-off-by: Gergely Brautigam <182850+Skarlso@users.noreply.github.com>
Comment thread kubernetes/controller/internal/configuration/config.go Outdated
Comment thread kubernetes/controller/internal/configuration/config.go Outdated
Comment thread kubernetes/controller/internal/configuration/config.go Outdated
Comment thread kubernetes/controller/internal/configuration/config.go Outdated
Comment thread kubernetes/controller/internal/configuration/config.go
Comment thread kubernetes/controller/internal/configuration/config.go Outdated
Co-authored-by: Fabian Burth <fabian.burth@sap.com>
Signed-off-by: Gergely Brautigam <skarlso777@gmail.com>
@Skarlso Skarlso force-pushed the migrate-config-and-context branch from b039f00 to ed25c37 Compare October 13, 2025 10:42
Skarlso and others added 5 commits October 13, 2025 13:26
On-behalf-of: Gergely Brautigam <gergely.brautigam@sap.com>

Signed-off-by: Gergely Brautigam <182850+Skarlso@users.noreply.github.com>
On-behalf-of: Gergely Brautigam <gergely.brautigam@sap.com>

Signed-off-by: Gergely Brautigam <182850+Skarlso@users.noreply.github.com>
On-behalf-of: Gergely Brautigam <gergely.brautigam@sap.com>

Signed-off-by: Gergely Brautigam <182850+Skarlso@users.noreply.github.com>
@Skarlso Skarlso merged commit e31d223 into open-component-model:main Oct 15, 2025
18 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

kind/feature new feature, enhancement, improvement, extension size/l Large

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants