Skip to content

docs(ADR): propose support for flag set selection#1634

Merged
toddbaert merged 6 commits into
mainfrom
docs/adr-multiple-flagset-support
Jul 9, 2025
Merged

docs(ADR): propose support for flag set selection#1634
toddbaert merged 6 commits into
mainfrom
docs/adr-multiple-flagset-support

Conversation

@toddbaert

@toddbaert toddbaert commented May 28, 2025

Copy link
Copy Markdown
Member

The goal of this decision document is to establish flag sets as a first class concept in flagd, and support the dynamic addition/update/removal of flag sets at runtime.

See document for all justifications and background.

@dominikhaska @tangenti @cupofcat

Alternative proposal here: #1644 (comment)

@toddbaert toddbaert requested a review from a team May 28, 2025 20:10
@toddbaert toddbaert requested a review from a team as a code owner May 28, 2025 20:10
@dosubot dosubot Bot added the size:L This PR changes 100-499 lines, ignoring generated files. label May 28, 2025
@netlify

netlify Bot commented May 28, 2025

Copy link
Copy Markdown

Deploy Preview for polite-licorice-3db33c ready!

Name Link
🔨 Latest commit 6c0ea6c
🔍 Latest deploy log https://app.netlify.com/projects/polite-licorice-3db33c/deploys/686ec6ccac7c8b0008f4b07d
😎 Deploy Preview https://deploy-preview-1634--polite-licorice-3db33c.netlify.app
📱 Preview on mobile
Toggle QR Code...

QR Code

Use your smartphone camera to open QR code link.

To edit notification comments on pull requests, go to your Netlify project configuration.

Signed-off-by: Alexandra Oberaigner <alexandra.oberaigner@dynatrace.com>
Comment thread docs/architecture-decisions/multiple-flag-set-support.md Outdated

@aepfli aepfli left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you! This looks good to me. I do think that this is the next logical step for evolving flagd. Currently the retro fitted approach is errornous, and I am still wondering that a few bugs have not been reported by now.

Comment thread docs/architecture-decisions/multiple-flag-set-support.md Outdated

We don't want to support merging of flag sets, due to implementation efforts & potential confusing behaviour of the
merge strategy.
Therefore, we propose for the initial implementation, `flagSetId`s must be unique across different sources or the configuration is considered invalid.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Which source will be considered as invalid? The same order as merging sources today?

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

My naive thought was just to act as if an invalid configuration was loaded in this case - so if we're starting up initially, we'd exit with an error code, and if we got into this state with a change after already being in a running state, the change would have no effect and an error would be logged with every update until the error is corrected.

@guidobrei guidobrei Jun 3, 2025

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We could prepare for different configurable merge strategies, with initially implementing a fail-fast strategy, or use the current merge strategy (last one wins).

.build());
````

* With the proposed solution the `flagSetId` should be passed to the builder as selector argument instead of the source.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same comments as in #1610 (comment)

If we really want to stay with simple syntax, we should consider using something new, like flagSetSelector, instead of selector.

@toddbaert toddbaert May 30, 2025

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

flagd is only one implementation of the proto/API - the selector selects flag sets with this proposal, but another implementation could use the selector to select something else (maybe it could even be an expression), so I'm hesitant to explicitly bind it to the flag-set concept.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We could also think about not introducing flagSets, but instead make selectors explicit.
E.g., something like this:

{
  "$schema": "https://flagd.dev/schema/v1/selectors.json",
  "selectors": {
    "my-project-1": {
      "metadata": {
        ...
      },
      "flags": {
        "my-flag-1": {
          "metadata": {
            ...
          },
          ...
        },
        ...
      },
      "$evaluators": {
        ...
      }
    },
    "my-project-2": {
      ...
    }
  }
}

With this, we'd keep our flagd vocabulary.

### Consequences

* Good, because it decouples flag sets from the sources
* Good, because we will refactor the flagd storage layer (which is currently storing duplicate data & difficult to

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this is another problem. IMO the storage needs to be refactored because it mixes the interface and the in-memory impl, which makes the in-memory storage the only possible storage to use at the point.

understand)
* Good, because we can support backwards compatibility with the v0 schema
* Good, because the "null" flag set is logically treated as any other flag set, reducing overall implementation complexity.
* Bad, because there's additional complexity to support this new config schema as well as the current.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The semantics of selector will also change, and the change happens mostly on the server (source) side, so the clients can have unexpected results with the same config.

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is true, I will add that.

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

But it could also be done in a non-breaking way.
Flagd could still treat the source value as flagSetId/selector if the file content adheres to the v0/flags schema

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

True, so this becomes a question of tech debt and code complexity.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reading through this ADR and thinking about it it seems to me that this is quite specific to the implementation of the flagd Sync server but the decisions here will impact the interface that the provider needs to use. The challenge here is that there will be other Sync server implementations.

Thus, I would strongly suggest we start from the provider side. What is the intended developer and product experience for consumption of flags using flagd providers? If we believe that some sort of "flags namespace / flags set" concept is needed, then we should design the provider interface that makes it a first class citizen. Then, Sync API implementations will have a good spec and each can figure out how to support that.

We could still keep the selector() field, but we should probably define some rules for the semantics of it. Or perhaps, we should create a schema for it that has some fields as required and then some mechanism for custom extensions? Or, we could create additional methods in the provider, like flagSetSelector().

WDYT?

Co-authored-by: Simon Schrottner <simon.schrottner@dynatrace.com>
Signed-off-by: Todd Baert <todd.baert@dynatrace.com>
Comment thread docs/architecture-decisions/multiple-flag-set-support.md
Signed-off-by: Todd Baert <todd.baert@dynatrace.com>
understand)
* Good, because we can support backwards compatibility with the v0 schema
* Good, because the "null" flag set is logically treated as any other flag set, reducing overall implementation complexity.
* Bad, because there's additional complexity to support this new config schema as well as the current.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

But it could also be done in a non-breaking way.
Flagd could still treat the source value as flagSetId/selector if the file content adheres to the v0/flags schema

.build());
````

* With the proposed solution the `flagSetId` should be passed to the builder as selector argument instead of the source.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We could also think about not introducing flagSets, but instead make selectors explicit.
E.g., something like this:

{
  "$schema": "https://flagd.dev/schema/v1/selectors.json",
  "selectors": {
    "my-project-1": {
      "metadata": {
        ...
      },
      "flags": {
        "my-flag-1": {
          "metadata": {
            ...
          },
          ...
        },
        ...
      },
      "$evaluators": {
        ...
      }
    },
    "my-project-2": {
      ...
    }
  }
}

With this, we'd keep our flagd vocabulary.

Comment thread docs/architecture-decisions/multiple-flag-set-support.md Outdated
Comment thread docs/architecture-decisions/multiple-flag-set-support.md Outdated
Comment thread docs/architecture-decisions/multiple-flag-set-support.md Outdated
Signed-off-by: Todd Baert <todd.baert@dynatrace.com>
Signed-off-by: Todd Baert <todd.baert@dynatrace.com>
@toddbaert toddbaert requested review from guidobrei and tangenti June 9, 2025 15:44
Comment on lines +147 to +148
* make sure to still support static syncs

@toddbaert toddbaert Jun 9, 2025

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@tangenti could you outline some of your other ideas? I'm interested in your proposals here (specifically 1. and 2.).

For our internal teams' specific purposes, we need some way of logically restricting/scoping the flags a particular client can evaluate, assuming all flags are coming from a single source. I'm open to other suggestions.

@toddbaert toddbaert changed the title docs(ADR): propose support for dynamic flag set usage docs(ADR): propose support for flag set selection Jun 9, 2025
.resolverType(Config.Evaluator.IN_PROCESS)
.host("localhost")
.port(8015)
.selector("myFlags.json")

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Question: Would this selector be set to my-project-1 if someone were to only want to evaluate the flags under the flag set of my-project-1 in the example above which calls out the new schema structure?

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Exactly.

@justinabrahms

Copy link
Copy Markdown
Member

Flag sets are just namespaces for flags, right? Not understanding why a user wants them.

@chrfwow

chrfwow commented Jun 11, 2025

Copy link
Copy Markdown
Contributor

Flag sets are just namespaces for flags, right? Not understanding why a user wants them.

@justinabrahms Flagd might provide flags for completely different applications, with each flag having its own set flags, that should be isolated from those of other applications. even if they share the same flag keys.

toddbaert pushed a commit that referenced this pull request Jul 8, 2025
This is proposal for support flags with duplicate keys, as a follow up
of the discussion on #1634 and #1644.

The selector semantics change proposed in #1644 will be addressed in a
separate ADR.

---------

Signed-off-by: Hugo Huang <lorqor@gmail.com>
Signed-off-by: Todd Baert <todd.baert@dynatrace.com>
@toddbaert

Copy link
Copy Markdown
Member Author

@toddbaert toddbaert merged commit b638441 into main Jul 9, 2025
6 of 11 checks passed
toddbaert pushed a commit that referenced this pull request Jul 29, 2025
The ADR proposes a different approach to solve the problems stated in
#1634.

It's quite concise and simple at the point, assuming the reviewers
already have the context from the original discussions.

Will add more details after the initial feedback and inputs.

---------

Signed-off-by: Hugo Huang <lorqor@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

size:L This PR changes 100-499 lines, ignoring generated files.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

10 participants