design: Propose a new approach to controlled rollout of composition functions#6935
design: Propose a new approach to controlled rollout of composition functions#6935adamwg wants to merge 4 commits intocrossplane:mainfrom
Conversation
…unctions Previously, we accepted a design that allowed for controlled rollout of composition functions by enabling multiple function revisions to be active at once and adding revision selectors to composition pipeline steps. Further thinking and discussion has led us to a new, simpler design. Move the old design to defunct and introduce the new one. Signed-off-by: Adam Wolfe Gordon <awg@upbound.io>
📝 WalkthroughWalkthroughMarks the original design doc as superseded and adds a new v2 design that enables OCI package references for composition functions, introduces FunctionRuntime/ProviderRuntime resources managed by runtime controllers, and describes a staged rollout controlled by a feature flag. (47 words) Changes
Sequence Diagram(s)sequenceDiagram
participant CR as CompositionRevision Controller
participant RT as Package Runtime Controller
participant FR as FunctionRuntime (Resource)
participant Runner as FunctionRunner
Note over CR,RT: OCI-based Function Reference Flow
CR->>FR: Create/Update FunctionRuntime (functionRef.package)
RT->>FR: Reconcile FunctionRuntime -> deploy runtime from OCI
FR-->>CR: Status (endpoint, ready)
Runner->>FR: Lookup by source label when package ref set
FR-->>Runner: Return endpoint
Runner->>Runner: Invoke function endpoint
Note over CR,Runner: Name-based Reference Flow (fallback)
CR->>Runner: Reference via functionRef.name
Runner->>FR: Lookup FunctionRuntime by active revision
FR-->>Runner: Return endpoint
Runner->>Runner: Invoke function endpoint
Estimated code review effort🎯 3 (Moderate) | ⏱️ ~20 minutes
Would you like me to call out specific schema fields or draft a short checklist for validating the runtime-controller reconciliation behavior? Pre-merge checks❌ Failed checks (1 warning)
✅ Passed checks (3 passed)
Tip 📝 Customizable high-level summaries are now available in beta!You can now customize how CodeRabbit generates the high-level summary in your pull requests — including its content, structure, tone, and formatting.
Example instruction:
Note: This feature is currently in beta for Pro-tier users, and pricing will be announced later. Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
There was a problem hiding this comment.
Actionable comments posted: 1
🧹 Nitpick comments (1)
design/one-pager-controlled-rollout-of-composition-functions-v2.md (1)
134-134: Consider more concise phrasing."In order to" can be simplified to "To" for consistency with technical writing conventions.
In order to allow runtime-only packages, we will introduce new `FunctionRuntime` +To allow runtime-only packages, we will introduce new `FunctionRuntime`
📜 Review details
Configuration used: Path: .coderabbit.yaml
Review profile: CHILL
Plan: Pro
📒 Files selected for processing (2)
design/defunct/one-pager-controlled-rollout-of-composition-functions.md(2 hunks)design/one-pager-controlled-rollout-of-composition-functions-v2.md(1 hunks)
🧰 Additional context used
📓 Path-based instructions (2)
**/*.md
⚙️ CodeRabbit configuration file
**/*.md: Ensure Markdown files are wrapped at 100 columns for consistency and
readability. Lines can be longer if it makes links more readable, but
otherwise should wrap at 100 characters. Check for proper heading
structure, clear language, and that documentation is helpful for users.
Files:
design/defunct/one-pager-controlled-rollout-of-composition-functions.mddesign/one-pager-controlled-rollout-of-composition-functions-v2.md
**/design/**
⚙️ CodeRabbit configuration file
**/design/**: Focus on architectural decisions, user experience, and long-term
maintainability. Ask clarifying questions about design choices and
consider alternative approaches. Ensure the design aligns with
Crossplane's principles and provides good user experience.
Files:
design/defunct/one-pager-controlled-rollout-of-composition-functions.mddesign/one-pager-controlled-rollout-of-composition-functions-v2.md
🪛 LanguageTool
design/one-pager-controlled-rollout-of-composition-functions-v2.md
[grammar] ~123-~123: Ensure spelling is correct
Context: ...at other changes happen in the cluster; pacakges can be specified by digest for further ...
(QB_NEW_EN_ORTHOGRAPHY_ERROR_IDS_1)
[style] ~134-~134: Consider a more concise word here.
Context: ...ackages that have a runtime component. In order to allow runtime-only packages, we will in...
(IN_ORDER_TO_PREMIUM)
🔍 Remote MCP
Summary of Additional Context Found
Related Issue Context
Issue #6139 addresses the core problem this PR aims to solve: users with large custom composition functions cannot do staggered rollouts because once a new function is deployed, all composites immediately use it, bypassing composition revisions. The issue proposes tracking the version or revision of compositions and matching them to function revisions in the pipeline.
Previous Design Reference
The previous design document is superseded by this new design and was motivated by the desire to identify a single kind of Function artifact suitable for most people—similar to Providers.
Composition Functions Context
Composition Functions were introduced as an Alpha feature in Crossplane 1.11 and were promoted to Beta in version 1.14. Composition functions run as standard OCI containers following specific interface requirements. Crossplane sends a function the desired state accumulated by all previous functions in the pipeline, and the function adds to or updates the desired state before passing it on.
New Design Approach
The v2 design document proposes three key architectural changes:
- OCI-based function references in composition steps using
functionRef.packageinstead of just resource names - A new runtime resource model where functions share accumulated desired state across the pipeline, with each function building upon previous functions' output
- Feature flag-based gradual rollout of these changes
Key Design Motivations
The controlled rollout approach addresses the production constraint identified in issue #6139, enabling:
- Staggered function deployments tied to specific composition revisions
- Independent function version tracking alongside composition versions
- Backward compatibility during the transition
🔇 Additional comments (7)
design/defunct/one-pager-controlled-rollout-of-composition-functions.md (1)
5-5: Proper superseding of v1 design with clear cross-reference to v2.The status update and reference link are correctly formatted and will help readers understand the design evolution. The link to the v2 design is properly referenced in markdown link style.
Also applies to: 224-224
design/one-pager-controlled-rollout-of-composition-functions-v2.md (6)
89-91: Clarify validation approach for mutual exclusivity ofnameandpackagefields.The design requires that providing both
nameandpackageis invalid, but the implementation details of this validation aren't specified. Should this be:
- Schema-level validation (e.g., oneOf in OpenAPI schema)?
- Webhook validation?
- Controller-level validation with error conditions?
This would be helpful to document for implementers.
144-147: Clarify FunctionRuntime lifecycle and garbage collection behavior.The design mentions that a FunctionRuntime may be shared between a FunctionRevision and one or more CompositionRevisions (via owner references). Please clarify:
- How does garbage collection work when multiple owners are present? Should the FunctionRuntime persist if at least one owner exists?
- What is the naming scheme for FunctionRuntime resources created by the composition revision controller? How does it avoid naming conflicts?
- The design mentions using
pkg.crossplane.io/sourcelabel for lookup—can you specify the expected label value format?These details will be important for implementers and for ensuring predictable behavior in production.
151-155: Clarify the design philosophy around dependencies for OCI-referenced functions.The design notes that OCI-referenced functions won't have their dependencies installed (lines 151-153), which represents a significant behavioral difference from pre-installed functions. A few clarifying questions:
- Intentional limitation or future enhancement? Is this a deliberate design choice or an interim limitation you expect to address later?
- Implications for function authors? Should the documentation guide function authors away from taking dependencies, or is there a recommended pattern for bundling dependencies within the OCI image?
- Error handling? If a function does have dependencies that aren't satisfied, how should this be communicated to users?
This will help guide the documentation and implementation strategy.
98-115: Well-designed progressive rollout mechanism.The approach of leveraging existing
compositionUpdatePolicyandcompositionRevisionSelectormechanisms is elegant and provides a natural UX for users familiar with composition versioning. This cleanly solves the controlled rollout problem identified in #6139 without requiring new user-facing APIs.
168-170: Thoughtful backwards compatibility design.Maintaining existing lookup logic for
functionRef.nameand using feature flags for new functionality preserves the ability for existing compositions to continue working unchanged. This gradual adoption strategy is well-considered.
177-195: Clear alternatives analysis.The alternatives section appropriately contextualizes v2 relative to the previous design, explaining why v1 was superseded. The rationale for rejecting other approaches is sound.
| them. They also don't need to know how the package manager names dependency | ||
| packages to construct the right `functionRef.name` in their composition. They | ||
| are also guaranteed to get exactly the version they specify regardless of what | ||
| other changes happen in the cluster; pacakges can be specified by digest for |
There was a problem hiding this comment.
Fix spelling error: "pacakges" → "packages"
Typo detected in the sentence about specifying packages by digest.
-This also makes the package manager conceptually simpler, since each
-resource type will be reconciled by only one controller; pacakges can be specified by digest for
+This also makes the package manager conceptually simpler, since each
+resource type will be reconciled by only one controller; packages can be specified by digest forCommittable suggestion skipped: line range outside the PR's diff.
🧰 Tools
🪛 LanguageTool
[grammar] ~123-~123: Ensure spelling is correct
Context: ...at other changes happen in the cluster; pacakges can be specified by digest for further ...
(QB_NEW_EN_ORTHOGRAPHY_ERROR_IDS_1)
🤖 Prompt for AI Agents
In design/one-pager-controlled-rollout-of-composition-functions-v2.md around
line 123, there's a typo: "pacakges" should be corrected to "packages"; update
the word to the correct spelling in that sentence so it reads "...packages can
be specified by digest for".
| 1. Unlike other package types, functions do not package resources that should be | ||
| installed in the cluster (their input types do get installed today, but this | ||
| is not a desired behavior). |
There was a problem hiding this comment.
Some functions do deploy resources they care about, e.g. function-claude-status-transformer. See #6841.
There was a problem hiding this comment.
I wonder if there are other examples of this in the wild. We might be caught by Hyrum's Law.
There was a problem hiding this comment.
I couldn't recall, but the xpkg spec does indeed say about function packages:
The package manager will not actually create the supplied CRDs in the API server.
Of course, that doesn't mean people don't rely on it :-).
| functionRef: | ||
| package: xpkg.crossplane.io/crossplane-contrib/function-patch-and-transform:v0.8.2 |
There was a problem hiding this comment.
I'd prefer to make package a top-level field of the pipeline step. Per API conventions fooRef style fields typically represent a reference to a object in the Kubernetes API.
I could be convinced to name the top-level field function instead of package. function feels more intuitive to me, and makes the oneOf relationship with functionRef more obvious. OTOH I could see why we might want to go with package - communicates that it's the same thing as you write in a Function or Provider.
There was a problem hiding this comment.
👍 I've updated the doc using function:. I don't have a strong feeling on function vs. package, but agree that function makes the relationship with functionRef more obvious.
| possible for multiple revisions of a function to be active at once. The | ||
| background section of the original design describes the problem we wish to | ||
| solve. |
There was a problem hiding this comment.
Would you say the loose coupling CompositionRevisions have with FunctionRevisions today is part of the problem the original design describes? If not, it's worth pointing out here that this proposal improves on that as well. With this design you can write Compositions such that a particular CompositionRevision is always pinned to a particular snapshot of the function code.
There was a problem hiding this comment.
I didn't explicitly call out this loose coupling in the original design, but it is the underlying issue. I think both designs address the coupling problem, in different ways: the original design by tightening the coupling (making it explicit), and this one by removing it. Added a note to that effect in the doc.
| have revisions, so controlled rollout of them is not relevant, but the other | ||
| simplifications in this design are applicable. | ||
|
|
||
| ### Composition Changes |
There was a problem hiding this comment.
We might also want to think through crossplane render UX. It'll mostly work, but today we use annotations on the Function package to configure how to run functions. That won't work in this world where there isn't a Function package.
There was a problem hiding this comment.
Added a note about operations. They're a little different in that they don't have revisions, but the design still applies.
There was a problem hiding this comment.
Re render, I agree it's worth figuring out here, I'm not sure what the right solution is. Three options I can think of, none of which I love:
- Introduce a new configuration file just for
renderthat tells it how to run functions, maybe with selectors similar toImageConfig. - Use the existing functions file and annotations, matching the pipeline's
functionto a provided Function'spackageto determine how it should be run. - Provide configuration via flags instead of a file.
I can bring this up with #sig-cli as well, since it will affect tools like crossplane-diff too.
| In order to allow runtime-only packages, we will introduce new `FunctionRuntime` | ||
| and `ProviderRuntime` resources, which will be reconciled by the package runtime | ||
| controller. The revision controller will be changed to create a runtime resource | ||
| corresponding to the active revision; the runtime controller will no longer | ||
| operate on revisions directly. |
| This allows the composition revision controller to create its own | ||
| `FunctionRuntime` resources to run functions without creating `Function`s or | ||
| `FunctionRevision`s. The package revision and composition revision controllers | ||
| will use the same scheme to name their `FunctionRevision`s, ensuring that a | ||
| particular version of a function runs only once even if it's installed multiple | ||
| ways. I.e., a `FunctionRuntime` may be shared between a `FunctionRevision` and | ||
| one or more `CompositionRevision`s. |
There was a problem hiding this comment.
What'll the owner/controller refs of a FunctionRuntime with multiple owners look like? I could imagine e.g. one with an active FunctionRevision and several extant CompositionRevision owners.
There was a problem hiding this comment.
My thinking was:
- If a function is installed by the package manager, the active
FunctionRevisionsets a controller reference on theFunctionRuntime. - All
CompositionRevisions that use a particularFunctionRuntimeset a non-controlling but deletion-blocking owner reference.
This means there's no controller reference if a function is installed only by pipeline reference. I believe that's OK for deletion prevention and GC purposes, but not 100% confident in my memory here.
| 2. They will not have corresponding package or package revision resources, | ||
| meaning they will not show up in `kubectl get pkg` or similar commands. |
There was a problem hiding this comment.
I have mixed feelings about them not showing up in kubectl get pkg.
I like this design a lot overall, but I worry about the asymmetry we're introducing. Functions are still a package in the artifact sense, but no longer a package in the API object sense. If you run kubectl get pkg you're not really seeing all the packages that're "installed" on your control plane anymore.
| 1. If `functionRef.package` is specified, the relevant `FunctionRuntime` | ||
| resource will be looked up directly using the `pkg.crossplane.io/source` | ||
| label. |
There was a problem hiding this comment.
I'm fine using this label, but iirc it's now possible/cheaper to list by field path directly. I assume the FunctionRuntime will have a spec.package field we could use.
…ition functions Add `function` at the same level as `functionRef` instead of adding `package` to `functionRef`. Signed-off-by: Adam Wolfe Gordon <awg@upbound.io>
…ition functions Add a note about coupling. Signed-off-by: Adam Wolfe Gordon <awg@upbound.io>
…ition functions Add a note about operations. Signed-off-by: Adam Wolfe Gordon <awg@upbound.io>
There was a problem hiding this comment.
Actionable comments posted: 0
♻️ Duplicate comments (1)
design/one-pager-controlled-rollout-of-composition-functions-v2.md (1)
134-134: Fix spelling error: "pacakges" → "packages"-are also guaranteed to get exactly the version they specify regardless of what -other changes happen in the cluster; pacakges can be specified by digest for +are also guaranteed to get exactly the version they specify regardless of what +other changes happen in the cluster; packages can be specified by digest for
🧹 Nitpick comments (1)
design/one-pager-controlled-rollout-of-composition-functions-v2.md (1)
145-145: Simplify phrasing for clarity."In order to" can be shortened to "To". This makes the sentence more concise while retaining meaning.
-In order to allow runtime-only packages, we will introduce new `FunctionRuntime` +To allow runtime-only packages, we will introduce new `FunctionRuntime`
📜 Review details
Configuration used: Path: .coderabbit.yaml
Review profile: CHILL
Plan: Pro
📒 Files selected for processing (1)
design/one-pager-controlled-rollout-of-composition-functions-v2.md(1 hunks)
🧰 Additional context used
📓 Path-based instructions (2)
**/*.md
⚙️ CodeRabbit configuration file
**/*.md: Ensure Markdown files are wrapped at 100 columns for consistency and
readability. Lines can be longer if it makes links more readable, but
otherwise should wrap at 100 characters. Check for proper heading
structure, clear language, and that documentation is helpful for users.
Files:
design/one-pager-controlled-rollout-of-composition-functions-v2.md
**/design/**
⚙️ CodeRabbit configuration file
**/design/**: Focus on architectural decisions, user experience, and long-term
maintainability. Ask clarifying questions about design choices and
consider alternative approaches. Ensure the design aligns with
Crossplane's principles and provides good user experience.
Files:
design/one-pager-controlled-rollout-of-composition-functions-v2.md
🪛 LanguageTool
design/one-pager-controlled-rollout-of-composition-functions-v2.md
[style] ~145-~145: Consider a more concise word here.
Context: ...ackages that have a runtime component. In order to allow runtime-only packages, we will in...
(IN_ORDER_TO_PREMIUM)
🔇 Additional comments (3)
design/one-pager-controlled-rollout-of-composition-functions-v2.md (3)
75-135: Design choice: Top-level field name —functionvs.packageThe proposal introduces a new top-level
functionfield for OCI references. In past review, @negz raised a preference for eitherfunctionorpackagehere, noting thatfunctionRefuses theRefsuffix to indicate a resource reference per Kubernetes API conventions, and thatfunctionmakes the mutual exclusivity withfunctionRefclearer.Can you clarify the final design decision on the field name? The current doc uses
functionin examples, but it's worth confirming this is the chosen approach, especially given the semantic difference between:
function: emphasizes what it does (feels more intuitive for users)package: emphasizes the artifact model (aligns with Function/Provider naming)This affects downstream understanding and adoption.
146-166: Design trade-off: FunctionRuntime visibility vs. package management consistencyThe proposal acknowledges that functions managed by the composition revision controller will not appear in
kubectl get pkgorkubectl get function. This is a notable departure from the current package model where all installed packages are discoverable.From a user experience perspective:
- Pro: Functions can be managed inline without pre-installation overhead
- Con: Reduced observability into all runtime functions across the control plane
Have you considered alternative approaches, such as:
- Creating stub
Function/FunctionRevisionresources for visibility without requiring dependency installation?- Adding a separate management command/API that shows all active FunctionRuntimes across both installation modes?
- Documenting this asymmetry clearly in the user guide to set expectations?
This feels important to address before implementation to avoid user confusion.
152-158: Clarification needed: FunctionRuntime ownership model with multiple ownersThe design proposes that FunctionRuntime resources can be shared across a
FunctionRevisionand multipleCompositionRevisionowners. This is mentioned but not detailed:a
FunctionRuntimemay be shared between aFunctionRevisionand one or moreCompositionRevisions.How will lifecycle be managed here? For example:
- When a
FunctionRevisionis garbage collected, should the runtime persist ifCompositionRevisionowners remain?- What happens if a runtime is created by composition revision first, then later a
FunctionRevisionis installed for the same image?- Should there be precedence rules or warnings if the same function is managed through multiple paths?
Clarifying the ownership and garbage collection semantics will help during implementation.
|
Crossplane does not currently have enough maintainers to address every issue and pull request. This pull request has been automatically marked as |
|
/fresh |
Description of your changes
In #6398, we accepted a design that allowed for controlled rollout of composition functions by enabling multiple function revisions to be active at once and adding revision selectors to composition pipeline steps. Further thinking and discussion has led us to a new, simpler design. Move the old design to defunct and introduce the new one.
Toward #6139
I have:
earthly +reviewableto ensure this PR is ready for review.- [ ] Added or updated unit tests.- [ ] Added or updated e2e tests.- [ ] Linked a PR or a docs tracking issue to document this change.- [ ] Addedbackport release-x.ylabels to auto-backport this PR.- [ ] Followed the API promotion workflow if this PR introduces, removes, or promotes an API.