Skip to content

feat: add root component for plugin registry#1229

Merged
jakobmoellerdev merged 15 commits into
open-component-model:mainfrom
Skarlso:create-root-plugin-registry-component
Nov 27, 2025
Merged

feat: add root component for plugin registry#1229
jakobmoellerdev merged 15 commits into
open-component-model:mainfrom
Skarlso:create-root-plugin-registry-component

Conversation

@Skarlso

@Skarlso Skarlso commented Nov 18, 2025

Copy link
Copy Markdown
Contributor

What this PR does / why we need it

Fixes open-component-model/ocm-project#598.

This is the current flow:

  • either a manual run triggers the workflow or a plugin update workflow once it finishes remote triggers this workflow with its newly published version and name.
  • once run, it will check if the root registry component exists or not.
    • if it doesn't exist it will create it current using the template under .github/config/plugin-registry-constructor.yaml
    • ( why template? because it's easy and visible and version controlled what kind of plugins there are, I didn't want to construct it dynamically, because I'm afraid that we will have no idea what plugins there are ( other then doing a component get and checking. but I'm up for a debate on this )
  • based on if a new plugin is added or an existing plugin is being modified it will automatically calculate the new version and bump it accordingly. new plugin -> minor update, existing plugin -> patch update. Major updates should be performed by a person pushing a new style of the plugin component or some such
  • the new version is constructed and pushed

Right now, there is no hashing of the component reference or the plugin registry. This will be added in a follow-up.

Flow:
Release new Go Submodule for Helm -> Triggers a publish on the plugin -> Triggers an update of the root component update.

TODO/Questions:

  • Right now, the plugin checking itself and the updates and such are agnostic of the plugin's name and such that is being pushed. However, this presents and interesting problem. Do we just blindly add plugins all the time and the template is literally just the main part with an empty componentReference? Or do we restrict what plugins we know about by templating that part like it is right now, we DON'T allow pushing a plugin that does NOT exist like the above flow thus currently

Which issue(s) this PR fixes

@github-actions github-actions Bot added kind/feature new feature, enhancement, improvement, extension component/github-actions Changes on GitHub Actions or within `.github/` directory size/l Large labels Nov 18, 2025
Comment thread .github/scripts/bump-semver.js Outdated
Comment thread .github/workflows/update-plugin-registry.yaml Outdated
Signed-off-by: Gergely Brautigam <182850+Skarlso@users.noreply.github.com>

On-behalf-of: Gergely Brautigam <gergely.brautigam@sap.com>
@Skarlso Skarlso force-pushed the create-root-plugin-registry-component branch from f35c000 to 1bc2925 Compare November 18, 2025 16:44
Comment thread .github/workflows/update-plugin-registry.yaml Outdated
Comment thread .github/workflows/update-plugin-registry.yaml Outdated
Comment thread .github/workflows/publish-helminput-plugin-component.yaml
On-behalf-of: Gergely Brautigam <gergely.brautigam@sap.com>

Signed-off-by: Gergely Brautigam <182850+Skarlso@users.noreply.github.com>
On-behalf-of: Gergely Brautigam <gergely.brautigam@sap.com>

Signed-off-by: Gergely Brautigam <182850+Skarlso@users.noreply.github.com>
@frewilhelm

Copy link
Copy Markdown
Contributor

Major updates should be performed by a person pushing a new style of the plugin component or some such

What major updates are you thinking about?

What about deletions of plugins from a registry - How would that work?

@frewilhelm

Copy link
Copy Markdown
Contributor

Or do we restrict what plugins we know about by templating that part like it is right now, we DON'T allow pushing a plugin that does NOT exist like the above flow thus currently

I think this should be the way. If a new plugin is introduced, it should also be added to the template. Otherwise it feels like an exploitable vulnerability.

@frewilhelm frewilhelm left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

great work! just some questions :)

Comment thread .github/scripts/bump-semver.js Outdated
Comment thread .github/workflows/update-plugin-registry.yaml
Comment thread .github/workflows/update-plugin-registry.yaml Outdated
Comment thread .github/workflows/update-plugin-registry.yaml Outdated
@Skarlso

Skarlso commented Nov 20, 2025

Copy link
Copy Markdown
Contributor Author

Going to make a decision here, and generate the plugins list dynamically.

Skarlso and others added 3 commits November 20, 2025 20:45
Signed-off-by: Gergely Brautigam <182850+Skarlso@users.noreply.github.com>
On-behalf-of: Gergely Brautigam <gergely.brautigam@sap.com>

Signed-off-by: Gergely Brautigam <182850+Skarlso@users.noreply.github.com>
@Skarlso

Skarlso commented Nov 21, 2025

Copy link
Copy Markdown
Contributor Author

So... flow is as follows:

  • check if the pushed plugin already exist with the same version, if yes, we error. we never overwrite a version
  • load the template
  • if registry exists
    • update the template's componentReferences list from the registry
    • it doesn't, create an empty list
  • add the new plugin to the list of componentReferences

Done. Much simpler flow.

@Skarlso

Skarlso commented Nov 21, 2025

Copy link
Copy Markdown
Contributor Author

This is how it looks like when a new plugin is released and then another plugin is released with a new version:

- component:
    componentReferences:
    - componentName: ocm.software/plugins/helminput
      digest:
        hashAlgorithm: SHA-256
        normalisationAlgorithm: jsonNormalisation/v4alpha1
        value: 4a3c1303bfbbc01090375e3ffc73bc8b7bbfcf6e3d4f682eaa8634df3462375b
      name: helminput
      version: 3.0.8
    - componentName: ocm.software/plugins/helminput
      digest:
        hashAlgorithm: SHA-256
        normalisationAlgorithm: jsonNormalisation/v4alpha1
        value: bfc01fc6eccf571864ef6ebe24ff476ae3019d42178ab5e7998a22bd848139a6
      name: helminput
      version: 3.1.0
    name: ocm.software/plugin-registry
    provider: ocm.software
    repositoryContexts: null
    resources: null
    sources: null
    version: v0.1.1
  meta:
    schemaVersion: v2
- component:
    componentReferences:
    - componentName: ocm.software/plugins/helminput
      digest:
        hashAlgorithm: SHA-256
        normalisationAlgorithm: jsonNormalisation/v4alpha1
        value: 4a3c1303bfbbc01090375e3ffc73bc8b7bbfcf6e3d4f682eaa8634df3462375b
      name: helminput
      version: 3.0.8
    name: ocm.software/plugin-registry
    provider: ocm.software
    repositoryContexts: null
    resources: null
    sources: null
    version: v0.1.0
  meta:
    schemaVersion: v2

@Skarlso Skarlso marked this pull request as ready for review November 21, 2025 12:28
@Skarlso Skarlso requested a review from a team as a code owner November 21, 2025 12:28
@frewilhelm

frewilhelm commented Nov 21, 2025

Copy link
Copy Markdown
Contributor

This is how it looks like when a new plugin is released and then another plugin is released with a new version:

Puh, this will be endlessly redundant. Not sure if this is maintainable after v0.123.0. But I don't have any idea how to do that better right now :D

@Skarlso

Skarlso commented Nov 21, 2025

Copy link
Copy Markdown
Contributor Author

Yeah. 🤔 I don't know. We don't want to overwrite versions... So... I don't know. :D

@jakobmoellerdev

Copy link
Copy Markdown
Member

The plugin registry file will be large. very large in fact. but thats okay :)

@matthiasbruns

Copy link
Copy Markdown
Contributor

The plugin registry file will be large. very large in fact. but thats okay :)

maybe in a later iteration we find a way to split it more - like nexus handles dependencies (folder structure, etc)

Comment thread .github/config/plugin-registry-constructor.yaml Outdated
Comment thread .github/workflows/release-go-submodule.yaml
Comment thread .github/workflows/update-plugin-registry.yaml Outdated
Comment thread .github/workflows/update-plugin-registry.yaml Outdated
Comment thread .github/workflows/update-plugin-registry.yaml Outdated
Comment thread .github/workflows/update-plugin-registry.yaml
On-behalf-of: Gergely Brautigam <gergely.brautigam@sap.com>

Signed-off-by: Gergely Brautigam <182850+Skarlso@users.noreply.github.com>
@Skarlso Skarlso force-pushed the create-root-plugin-registry-component branch from 2e72104 to bfcc1c8 Compare November 25, 2025 16:01
On-behalf-of: Gergely Brautigam <gergely.brautigam@sap.com>

Signed-off-by: Gergely Brautigam <182850+Skarlso@users.noreply.github.com>
@Skarlso Skarlso force-pushed the create-root-plugin-registry-component branch from bfcc1c8 to dd58335 Compare November 25, 2025 16:16
@Skarlso

Skarlso commented Nov 25, 2025

Copy link
Copy Markdown
Contributor Author

I'm simplifying this even further.

@Skarlso Skarlso force-pushed the create-root-plugin-registry-component branch 2 times, most recently from a87c864 to 7c57e17 Compare November 25, 2025 16:46
…the version and put it into the prepare script

On-behalf-of: Gergely Brautigam <gergely.brautigam@sap.com>

Signed-off-by: Gergely Brautigam <182850+Skarlso@users.noreply.github.com>
@Skarlso Skarlso force-pushed the create-root-plugin-registry-component branch from 7c57e17 to 249cd19 Compare November 25, 2025 16:47
Comment thread .github/scripts/compute-rc-version.js
Comment thread .github/workflows/update-plugin-registry.yaml Outdated
Comment thread .github/workflows/update-plugin-registry.yaml
Comment thread .github/workflows/update-plugin-registry.yaml Outdated
Comment thread .github/workflows/update-plugin-registry.yaml Outdated
Comment thread .github/scripts/prepare-registry-constructor.js Outdated
Comment thread .github/workflows/publish-helminput-plugin-component.yaml
Comment thread .github/workflows/publish-helminput-plugin-component.yaml
On-behalf-of: Gergely Brautigam <gergely.brautigam@sap.com>

Signed-off-by: Gergely Brautigam <182850+Skarlso@users.noreply.github.com>
@Skarlso

Skarlso commented Nov 26, 2025

Copy link
Copy Markdown
Contributor Author
Run actions/github-script@ed597411d8f924073f98dfc5c65a23a2325f34cd
Preparing registry constructor for helminput v3.1.6
Error: Plugin with name helminput and version 3.1.6 already exists in reference list

plugin exists check is working 🎉

@jakobmoellerdev

Copy link
Copy Markdown
Member

Right now, there is no hashing of the component reference or the plugin registry. This will be added in a follow-up.

Do we have a follow up for this already?

@jakobmoellerdev

Copy link
Copy Markdown
Member

Right now, the plugin checking itself and the updates and such are agnostic of the plugin's name and such that is being pushed. However, this presents and interesting problem. Do we just blindly add plugins all the time and the template is literally just the main part with an empty componentReference? Or do we restrict what plugins we know about by templating that part like it is right now, we DON'T allow pushing a plugin that does NOT exist like the above flow thus currently

this is very hard to understand for me, could you try to rephrase the problem in different wording so I understand what you are looking to solve?

Comment thread .github/workflows/update-plugin-registry.yaml Outdated
Comment thread .github/workflows/update-plugin-registry.yaml Outdated
@Skarlso

Skarlso commented Nov 26, 2025

Copy link
Copy Markdown
Contributor Author

Do we have a follow up for this already?

Yep. I have an issue for that #1227.

It's in needs refinement for sprint 52 and doesn't belong to an EPIC yet.

this is very hard to understand for me, could you try to rephrase the problem in different wording so I understand what you are looking to solve?

Nevermind, I already decided on what to do. :) You can ignore it.

@Skarlso

Skarlso commented Nov 26, 2025

Copy link
Copy Markdown
Contributor Author

Yay, successful run after fixes ( I'll push them in a sec ):
https://github.com/Skarlso/open-component-model/actions/runs/19696661625

Screenshot 2025-11-26 at 9 05 49

Whoops, previous is empty. :D

On-behalf-of: Gergely Brautigam <gergely.brautigam@sap.com>

Signed-off-by: Gergely Brautigam <182850+Skarlso@users.noreply.github.com>
On-behalf-of: Gergely Brautigam <gergely.brautigam@sap.com>

Signed-off-by: Gergely Brautigam <182850+Skarlso@users.noreply.github.com>

@jakobmoellerdev jakobmoellerdev left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

huge PR so no guarantee that I didnt catch some stuff but generally ok for me (I think) :D

On-behalf-of: Gergely Brautigam <gergely.brautigam@sap.com>

Signed-off-by: Gergely Brautigam <182850+Skarlso@users.noreply.github.com>
On-behalf-of: Gergely Brautigam <gergely.brautigam@sap.com>

Signed-off-by: Gergely Brautigam <182850+Skarlso@users.noreply.github.com>
@Skarlso

Skarlso commented Nov 26, 2025

Copy link
Copy Markdown
Contributor Author

Finally! Behold:

Run actions/github-script@ed597411d8f924073f98dfc5c65a23a2325f34cd
Adding helminput v3.1.13 to plugin registry
Fetching registry descriptor: ghcr.io/skarlso/plugins//ocm.software/plugin-registry
Found registry v0.4.4 with 8 plugin(s)
Publishing component to ghcr.io/skarlso/plugins
Verifying component: ocm.software/plugin-registry:0.4.5
Verification successful
Successfully published registry v0.4.5

Error scenario:

Adding helminput v3.1.13 to plugin registry
Fetching registry descriptor: ghcr.io/skarlso/plugins//ocm.software/plugin-registry
Found registry v0.4.5 with 9 plugin(s)
Error: Plugin helminput v3.1.13 already exists in registry

It's working <3.

I hand wrote the prepare-registry-constructor.js then run it through an llm to make it prettier and more robust because I suck at JS. But this had the added benefit that I understand how it works. :D It's pretty basic honestly. Nothing too fancy. I shifted most of the YAML logic into the Javascript so it's testable, and more importantly, maintainable instead of everything being in github action steps.

@Skarlso

Skarlso commented Nov 27, 2025

Copy link
Copy Markdown
Contributor Author

I removed a bunch of permissions and set some others to read. I'm running a test now.

Edit: Crap. It's required for the trigger and the attestation.

@matthiasbruns matthiasbruns left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

too much js for one day

but lgtm - a lot of testing and verification <3

@jakobmoellerdev jakobmoellerdev enabled auto-merge (squash) November 27, 2025 11:41
@jakobmoellerdev jakobmoellerdev merged commit de7753c into open-component-model:main Nov 27, 2025
19 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

component/github-actions Changes on GitHub Actions or within `.github/` directory kind/feature new feature, enhancement, improvement, extension size/l Large

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Create a root component that can act as a component registry

5 participants