WIP: add keda autoscaler section on runner installation #1073

Closed
cobak78 wants to merge 2 commits from cobak78/docs:keda-autoscaler into next
Member
No description provided.
Contributor

Before diving into details, there is one important aspect to address. Keda being Open Core and Forgejo being committed to providing software and documentation with no strings attached, it is necessary to clarify where to look for 100% Free Software Keda and if it is not crippled (because some companies do that, so it needs to be stated clearly).

To clarify my meaning, one of the worst example of software crippling is when security updates are provided with a delay to the general public, only after paying customers get the security fix.

Does that sound reasonable?

Before diving into details, there is one important aspect to address. Keda being Open Core and Forgejo being committed to providing software and documentation with no strings attached, it is necessary to clarify where to look for 100% Free Software Keda and if it is not crippled (because some companies do that, so it needs to be stated clearly). To clarify my meaning, one of the worst example of software crippling is when security updates are provided with a delay to the general public, only after paying customers get the security fix. Does that sound reasonable?
cobak78 force-pushed keda-autoscaler from 575fece9cb
Some checks failed
pr / preview (pull_request_target) Failing after 19s
to 7226f1edc7
Some checks failed
pr / preview (pull_request_target) Failing after 23s
2025-02-10 21:12:14 +01:00
Compare
First-time contributor

Hello
I'm Jorge, one of KEDA's maintainers. KEDA belongs to CNCF with the graduated status and it's mainly maintained by serval companies, including Microsoft, RedHat, SCRM Lidl International Hub (Schwarz Grouppe) or Kedify.. You can get more info about CNCF's Intellectual Property Rights Policy here, but basically all the code contributions are done under "Apache 2.0" license and docs contributions are licensed with "Creative Commons Attribution 4.0 International"

About our governance, you can see all the related things in our governance repo

About the question of offering security fixes only to paid users, we ship all the releases according our roadmap to all the users, usually we ship a release every 3 months, but we are currently discussing to change the release process because of the maturity of the project.

Keeping these points aside, there are enterprise providers which could offer (or not) better release cadence or improved security under a paywall. KEDA is open source community driven and it means that not all the vulnerabilities can be fixed and the hotfix shipped the day after the disclosure, but definitively we don't block any fix or security improvement that contributors do (and all the merged commits are built and published as main docker tag, so users can early adopt the fixes although a proper hotfix release hasn't been cut yet)

In any case, let me know if you have any doubt or concern and I'll try to solve it

Hello I'm Jorge, [one of KEDA's maintainers](https://github.com/kedacore/governance/blob/main/MAINTAINERS.md). KEDA belongs to [CNCF](https://www.cncf.io/) with the [graduated status](https://www.cncf.io/projects/keda/) and it's [mainly maintained by serval companies, including Microsoft, RedHat, SCRM Lidl International Hub (Schwarz Grouppe) or Kedify.](https://keda.devstats.cncf.io/d/5/companies-table?orgId=1&var-period_name=Last%20year&var-metric=contributions). You can get more info about CNCF's Intellectual Property Rights Policy [here](https://github.com/cncf/foundation/blob/main/charter.md#11-ip-policy), but basically all the code contributions are done under "Apache 2.0" license and docs contributions are licensed with "Creative Commons Attribution 4.0 International" About our governance, you can see all the related things in our [governance repo](https://github.com/kedacore/governance) About the question of offering security fixes only to paid users, we ship all the releases according [our roadmap](https://github.com/kedacore/keda/blob/main/ROADMAP.md) to all the users, usually we ship a release every 3 months, but we are currently [discussing to change the release process](https://github.com/kedacore/governance/issues/122) because of the maturity of the project. Keeping these points aside, [there are enterprise providers](https://keda.sh/enterprise/) which could offer (or not) better release cadence or improved security under a paywall. KEDA is open source community driven and it means that not all the vulnerabilities can be fixed and the hotfix shipped the day after the disclosure, but definitively we don't block any fix or security improvement that contributors do (and all the merged commits are built and published as `main` docker tag, so users can early adopt the fixes although a proper hotfix release hasn't been cut yet) In any case, let me know if you have any doubt or concern and I'll try to solve it
Contributor

My concern was specifically that there is a conflict of interest when Kedify who employs 50% of the maintainers sells a proprietary product based on Keda which is advertised as follows:

Enterprise-grade support, ensuring secure KEDA builds with timely updates, patches, and critical fixes.

In other words, the maintainers and the company are in a position where they need to choose between Keda and Kedify. If Keda gets updates, patches and critical fixes as fast as Kedify, the Enterprise-grade support is no longer a value proposition of the proprietary product over the Free Software product.

The Kedify maintainers may do their best so security patches are always simulatenously released on Keda and Kedify, so the Free Software community is not left behind with vulnerabilities, because it matters to them, personally. Or maybe they see such a difference as justified for some reason, I can't tell. And I can't say which way it will go because, and this is the problem, they have a conflict of interest.

My concern was specifically that there is a conflict of interest when Kedify who employs 50% of the maintainers sells a proprietary product based on Keda which is [advertised as](https://keda.sh/enterprise/) follows: > Enterprise-grade support, ensuring secure KEDA builds with timely updates, patches, and critical fixes. In other words, the maintainers and the company are in a position where they need to choose between Keda and Kedify. If Keda gets updates, patches and critical fixes as fast as Kedify, the **Enterprise-grade support** is no longer a value proposition of the proprietary product over the Free Software product. The Kedify maintainers may do their best so security patches are always simulatenously released on Keda and Kedify, so the Free Software community is not left behind with vulnerabilities, because it matters to them, personally. Or maybe they see such a difference as justified for some reason, I can't tell. And I can't say which way it will go because, and this is the problem, they have a conflict of interest.
First-time contributor

My concern was specifically that there is a conflict of interest when Kedify who employs 50% of the maintainers sells a proprietary product based on Keda which is advertised as follows:

Yeah, that's true and it's a legit concern, but it doesn't mean that there aren't other maintainers with different affiliations, and other contributors contributing (CNCF Metrics)

In other words, the maintainers and the company are in a position where they need to choose between Keda and Kedify. If Keda gets updates, patches and critical fixes as fast as Kedify, the Enterprise-grade support is no longer a value proposition of the proprietary product over the Free Software product.

That's true, but you're placing at the same label a startup with RedHat, Snowflake, Microsoft or Schwarz Group, any company/person that want to contribute, can contribute with security fixes or features. Keeping aside the point that Kedify doesn't own the project, the project belong to CNCF, which is backed by Linux Fundation, AWS, Google, Microsoft, Apple, etc... We as maintainers can't take those kind of decisions as the CNCF can takedown maintainers.

The open source commitment has been clear for more than 5 years, but it's true, something can change in the future because of any reason. It happens also with all the deps of a project, any dependency can change the license from one day to the next one.

Answering to the question about delaying the security fixes, the answer is: a user can expect a any merged change ready to use on their sides just in a couple of minutes via main (docker)tag and a "feature" release every 3 months with a hotfix release in the middle, like we have been doing with the possibility of changing a bit based on this discussion.In that discussion, we are aiming to cut a feature release probably every 4-6 months and hotfixes every month but it's not official yet and it's under discussion.

We built all the commits for AMD and ARM (the current supported architectures), the only difference is that you won't see it as v2.10.x tag as it requires a release process, updating helm charts and docs, but the fix is automatically published as soon as it merged.
There are automated tools opening PRs to bump deps when there is a CVE detected as well as a detection tools to monitor new CVEs introduced with PRs.

> My concern was specifically that there is a conflict of interest when Kedify who employs 50% of the maintainers sells a proprietary product based on Keda which is advertised as follows: Yeah, that's true and it's a legit concern, but it doesn't mean that there aren't other maintainers with different affiliations, and other contributors contributing ([CNCF Metrics](https://keda.devstats.cncf.io/d/5/companies-table?orgId=1&var-period_name=Last%20year&var-metric=contributions)) > In other words, the maintainers and the company are in a position where they need to choose between Keda and Kedify. If Keda gets updates, patches and critical fixes as fast as Kedify, the Enterprise-grade support is no longer a value proposition of the proprietary product over the Free Software product. That's true, but you're placing at the same label a startup with RedHat, Snowflake, Microsoft or [Schwarz Group](https://en.wikipedia.org/wiki/Schwarz_Group), any company/person that want to contribute, can contribute with security fixes or features. Keeping aside the point that Kedify doesn't own the project, the project belong to CNCF, which is backed by [Linux Fundation, AWS, Google, Microsoft, Apple](https://www.cncf.io/about/members/), etc... We as maintainers can't take those kind of decisions as the CNCF can takedown maintainers. The open source commitment has been clear for more than 5 years, but it's true, something can change in the future because of any reason. It happens also with all the deps of a project, any dependency can change the license from one day to the next one. Answering to the question about delaying the security fixes, the answer is: a user can expect a any merged change ready to use on their sides just in a couple of minutes via `main` (docker)tag and a "feature" release every 3 months with a hotfix release in the middle, like we have been doing with the possibility of changing a bit based on [this discussion](https://github.com/kedacore/governance/issues/122).In that discussion, we are aiming to cut a feature release probably every 4-6 months and hotfixes every month but it's not official yet and it's under discussion. We built all the commits for AMD and ARM (the current supported architectures), the only difference is that you won't see it as v2.10.x tag as it requires a release process, updating helm charts and docs, but the fix is automatically published as soon as it merged. There are automated tools opening PRs to bump deps when there is a CVE detected as well as a detection tools to monitor new CVEs introduced with PRs.
Contributor

I very much appreciate the time you took to explain the situation. And also for being understanding of my concerns 🙏 I consider them resolved as far as Forgejo is concerned. 👍

One last question out of curiosity: do you know what Kedify Enterprise-grade support, ensuring secure KEDA builds with timely updates, patches, and critical fixes. entails exactly? I know you're not an employee of Kedify and I'm interested in your outsider perspective.

I very much appreciate the time you took to explain the situation. And also for being understanding of my concerns 🙏 I consider them resolved as far as Forgejo is concerned. 👍 One last question out of curiosity: do you know what Kedify `Enterprise-grade support, ensuring secure KEDA builds with timely updates, patches, and critical fixes.` entails exactly? I know you're not an employee of Kedify and I'm interested in your outsider perspective.
@ -280,1 +280,4 @@
## Autoescaling runners with KEDA
Another type of configuration for the runners is to create a ScaledJob on KEDA that polls the job endpoints on your Forgejo instance. When it detects a job to be executed, it spawns a runner with the --one-job flag.
Contributor

A link to keda.sh would be useful for people who were never exposed to KEDA.

A link to keda.sh would be useful for people who were never exposed to KEDA.
cobak78 marked this conversation as resolved
Contributor

@cobak78 the examples / snippets look good but it always is challenging to go from there to something that one can actually play with and experiment from. I'm not sure how to do that in a semi-simple way though. If you have ideas I'll take them. The CI can run k3s clusters started from scratch so the context for checking an example actually works is there but... designing one is another matter. The forgejo-helm end to end tests are a source of inspiration but they lack the simplicity that is expected from an example.

@cobak78 the examples / snippets look good but it always is challenging to go from there to something that one can actually play with and experiment from. I'm not sure how to do that in a semi-simple way though. If you have ideas I'll take them. The CI can run k3s clusters started from scratch so the context for checking an example actually works is there but... designing one is another matter. The [forgejo-helm end to end tests](https://code.forgejo.org/forgejo-helm/forgejo-helm/src/commit/d6b799cd55fac8f61298c7380982d29a64433a2c/.forgejo/workflows/build.yml#L89-L101) are a source of inspiration but they lack the simplicity that is expected from an example.
@ -281,0 +284,4 @@
### Registering runners and binding config with autoscalers
To match job endpoints with job execution, we need to pre-register a runner, save the generated JSON file as a ConfigMap, and share it with the autoscaler. On register select the desired scope (user, repository or global) and create the corresponding JSON file.
Contributor

A link to the section about registering a runner would be good here.

A link to the section about registering a runner would be good here.
cobak78 marked this conversation as resolved
@ -281,0 +324,4 @@
### Create a token on forgejo
The autoscaler will continuously fetch new waiting jobs from Forgejo, so it requires a properly configured token with the appropriate permissions. Navigate to **Forgejo > User settings > Applications > Access Tokens** and generate a new token. This token will be used by the autoscaler
Contributor

Please add which permissions are required.

Please add which permissions are required.
cobak78 marked this conversation as resolved
@ -281,0 +328,4 @@
### Configuring KEDA Autoscaler
Create a new custom resource with the required parameters and custom fields based on the scope chosen during runner registration. For example, set `global="true"` for global runners or specify `repo=xxx` and `owner=xxx` for repository level runners.
Contributor

custom fields based on the scope chosen during runner registration

The word scope is usually associated with the token scope. Maybe the following clarifies that it is about something different:

custom fields based on where the runner is registered:

  • global="true"` for global runners or,
  • repo=xxx and owner=xxx for repository level runners.

Is it not possible to specify owner=xxx for user or organization runners?

> custom fields based on the scope chosen during runner registration The word `scope` is usually associated with the token scope. Maybe the following clarifies that it is about something different: > custom fields based on where the runner is registered: > > - global="true"` for global runners or, >- `repo=xxx` and `owner=xxx` for repository level runners. Is it not possible to specify `owner=xxx` for user or organization runners?
cobak78 marked this conversation as resolved
@ -281,0 +359,4 @@
containers:
- name: runner
image: forgejo:5001/runner:latest
command: ["sh", "-c", "stackit-git-runner job"]
Contributor

Is this intended?

Is this intended?
cobak78 marked this conversation as resolved
@ -281,0 +352,4 @@
volumes:
- name: runner-data
persistentVolumeClaim:
claimName: runner-pvc-claim
Contributor

Is this necessary in the context of this example?

Is this necessary in the context of this example?
cobak78 marked this conversation as resolved
First-time contributor

One last question out of curiosity: do you know what Kedify Enterprise-grade support, ensuring secure KEDA builds with timely updates, patches, and critical fixes. entails exactly? I know you're not an employee of Kedify and I'm interested in your outsider perspective.

The message displayed there is chosen by the provider, but I guess that they mean that they will update anything that is required out-of-the-box without 0 effort from users side (AFAIK, they use an agent to manage KEDA inside customers clusters).
Using OSS project, users need to update their version by themselves, checking if they work for them or not. Mostly this is just a change in the chart version used but we can fail (we try to improve our test suite but shit happens xD) and it can impact.

About critical fixes, I guess that it means that they solve any bug that happens instantly and ship a patch version and KEDA follows its release process with a feature release + 1 hotfix release between feature releases 🤷
I know them but I haven't asked about their business model around KEDA tbh, I know that they are building a business model around HTTP autoscaling, which is an add-on on top of KEDA to extend KEDA with HTTP Scaling capabilities but that's a split project from the core and it's something in beta atm

> One last question out of curiosity: do you know what Kedify Enterprise-grade support, ensuring secure KEDA builds with timely updates, patches, and critical fixes. entails exactly? I know you're not an employee of Kedify and I'm interested in your outsider perspective. The message displayed there is chosen by the provider, but I guess that they mean that they will update anything that is required out-of-the-box without 0 effort from users side (AFAIK, they use an agent to manage KEDA inside customers clusters). Using OSS project, users need to update their version by themselves, checking if they work for them or not. Mostly this is just a change in the chart version used but we can fail (we try to improve our test suite but shit happens xD) and it can impact. About critical fixes, I guess that it means that they solve any bug that happens instantly and ship a patch version and KEDA follows its release process with a feature release + 1 hotfix release between feature releases 🤷 I know them but I haven't asked about their business model around KEDA tbh, I know that they are building a business model around HTTP autoscaling, which is an [add-on](https://github.com/kedacore/http-add-on) on top of KEDA to extend KEDA with HTTP Scaling capabilities but that's a split project from the core and it's something in beta atm
Author
Member

@earl-warren wrote in #1073 (comment):

@cobak78 the examples / snippets look good but it always is challenging to go from there to something that one can actually play with and experiment from. I'm not sure how to do that in a semi-simple way though. If you have ideas I'll take them. The CI can run k3s clusters started from scratch so the context for checking an example actually works is there but... designing one is another matter. The forgejo-helm end to end tests are a source of inspiration but they lack the simplicity that is expected from an example.

I'm working on an end-to-end (e2e) test for the KEDA pull request, where a Forgejo instance with a populated database is set up. This setup will trigger the KEDA autoscaler to detect a job to run and spawn a ScaledJob. I will try to replicate this on Forgejo CI and document it here.

@earl-warren wrote in https://codeberg.org/forgejo/docs/pulls/1073#issuecomment-2811807: > @cobak78 the examples / snippets look good but it always is challenging to go from there to something that one can actually play with and experiment from. I'm not sure how to do that in a semi-simple way though. If you have ideas I'll take them. The CI can run k3s clusters started from scratch so the context for checking an example actually works is there but... designing one is another matter. The [forgejo-helm end to end tests](https://code.forgejo.org/forgejo-helm/forgejo-helm/src/commit/d6b799cd55fac8f61298c7380982d29a64433a2c/.forgejo/workflows/build.yml#L89-L101) are a source of inspiration but they lack the simplicity that is expected from an example. I'm working on an end-to-end (e2e) test for the KEDA pull request, where a Forgejo instance with a populated database is set up. This setup will trigger the KEDA autoscaler to detect a job to run and spawn a ScaledJob. I will try to replicate this on Forgejo CI and document it here.
Contributor

Great news 🎉

Note that the CI at https://codeberg.org/forgejo/forgejo/ or https://code.forgejo.org/forgejo/end-to-end/ do not have the required capabilities to run a k8s cluster. A dedicated repository can be created (e.g. https://code.forgejo.org/keda/end-to-end/?) and a runner with the required capabilities assigned to it.

Would that help?

Great news 🎉 Note that the CI at https://codeberg.org/forgejo/forgejo/ or https://code.forgejo.org/forgejo/end-to-end/ do not have the required capabilities to run a k8s cluster. A dedicated repository can be created (e.g. https://code.forgejo.org/keda/end-to-end/?) and a runner with the required capabilities assigned to it. Would that help?
cobak78 force-pushed keda-autoscaler from 7226f1edc7
Some checks failed
pr / preview (pull_request_target) Failing after 23s
to 816696c8fa
Some checks failed
pr / preview (pull_request_target) Failing after 21s
2025-02-16 22:36:42 +01:00
Compare
add links and fix comments
Some checks failed
pr / preview (pull_request_target) Failing after 20s
backport / backporting (pull_request_target) Has been skipped
b27f8144a7
Contributor

@cobak78 the v11 branch will be cut in a few days (26 March), do you have time to complete this in time?

@cobak78 the v11 branch will be cut in a few days (26 March), do you have time to complete this in time?
Contributor

@cobak78 ping?

@cobak78 ping?
Contributor

@cobak78 could you please rebase and fix this PR so it can be merged before v11 is published?

@cobak78 could you please rebase and fix this PR so it can be merged before v11 is published?
Contributor

@cobak78 ping?

@cobak78 ping?
earl-warren changed title from add keda autoscaler section on runner installation to WIP: add keda autoscaler section on runner installation 2025-06-21 10:08:40 +02:00
Contributor

Please re-open when you have time to complete this 🙏

Please re-open when you have time to complete this 🙏
earl-warren closed this pull request 2025-08-03 09:16:00 +02:00
Some checks failed
pr / preview (pull_request_target) Failing after 20s
Required
Details
backport / backporting (pull_request_target) Has been skipped

Pull request closed

Sign in to join this conversation.
No reviewers
No milestone
No project
No assignees
3 participants
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference
forgejo/docs!1073
No description provided.