Skip to content

structured config #1: introduce mapstructure decoder for yaml parsing#2628

Merged
tuunit merged 20 commits intooauth2-proxy:masterfrom
tuunit:use-mapstructures-for-parsing-and-merging
Nov 28, 2025
Merged

structured config #1: introduce mapstructure decoder for yaml parsing#2628
tuunit merged 20 commits intooauth2-proxy:masterfrom
tuunit:use-mapstructures-for-parsing-and-merging

Conversation

@tuunit
Copy link
Copy Markdown
Member

@tuunit tuunit commented May 4, 2024

Description

This PR introduces mapstructure for decoding and encoding the yaml files. Mapstructure is a defacto standard library used by many libraries like spf13/viper for more dynamic data loading between different interfaces.

Just using the golang json / yaml encoding / decoding would lead to overwriting the default set before loading the config file. Trying to load the default configs afterwards would be rather hard or impossible.

Motivation and Context

Cases to consider:

  1. When directly loading into a struct you cannot identify if the user explicitly set a boolean to false in the config file or if it was set to the boolean default of false. This is problematic as we have boolean like Cookie.Secure that are supposed to be set to true by default.
  2. For better readability we might want to squash elements in the yaml like HeaderValues either being a SecretSource or ClaimSource. Which isn't supported by the default json / yaml parser
# non-squashed
headers:
- name: X-Forwarded-User
  values:
  - claimSource:
      claim: user

# squashed
headers:
- name: X-Forwarded-User
  values:
  - claim: user
  1. Parsing of time primitives like Duration or Time through strings. If you want to be able to configure Duration types with values like 2h. You need to implement a custom wrapper type and Marshal and Unmarshal method. Mapstructure allows for decoding hooks through the usage of relection.

How Has This Been Tested?

  1. Test cases have been extend / adapted.
  2. Starting the application with toml config file
  3. Using the conversion function from toml to yaml
  4. Starting the application with yaml config file

Checklist:

  • My change requires a change to the documentation or CHANGELOG.
  • I have updated the documentation/CHANGELOG accordingly.
  • I have created a feature (non-master) branch for my PR.
  • I have written tests for my code changes.

@tuunit tuunit force-pushed the use-mapstructures-for-parsing-and-merging branch from 710f3a7 to 72dff61 Compare May 4, 2024 14:44
@github-actions github-actions Bot added the docs label May 4, 2024
@tuunit tuunit changed the title introduce mapstructure decoder for yaml parsing structured logging #1: introduce mapstructure decoder for yaml parsing May 4, 2024
@tuunit tuunit changed the title structured logging #1: introduce mapstructure decoder for yaml parsing structured config #1: introduce mapstructure decoder for yaml parsing May 4, 2024
@tuunit tuunit force-pushed the use-mapstructures-for-parsing-and-merging branch from 39b625e to 3fb3c1f Compare May 9, 2024 22:00
@tuunit tuunit marked this pull request as ready for review May 10, 2024 07:34
@tuunit tuunit requested a review from a team as a code owner May 10, 2024 07:34
Comment thread pkg/apis/options/header.go Outdated
@tuunit tuunit force-pushed the use-mapstructures-for-parsing-and-merging branch from 7f8def6 to cd566ee Compare May 12, 2024 17:40
@github-actions
Copy link
Copy Markdown
Contributor

This pull request has been inactive for 60 days. If the pull request is still relevant please comment to re-activate the pull request. If no action is taken within 7 days, the pull request will be marked closed.

@github-actions github-actions Bot added the Stale label Jul 12, 2024
Comment thread main.go Outdated
Comment thread pkg/apis/options/duration.go
Comment thread pkg/apis/options/duration.go
Comment thread pkg/apis/options/header.go Outdated
Comment thread pkg/apis/options/load.go Outdated
Comment thread pkg/apis/options/load.go
Comment thread pkg/apis/options/load.go Outdated
Comment thread pkg/apis/options/providers.go
Comment thread pkg/apis/options/secret_source.go Outdated
Comment thread pkg/apis/options/secret_source.go Outdated
@github-actions github-actions Bot removed the Stale label Jul 15, 2024
@tuunit tuunit changed the base branch from master to release/v8.0.0 August 20, 2024 08:44
@github-actions
Copy link
Copy Markdown
Contributor

This pull request has been inactive for 60 days. If the pull request is still relevant please comment to re-activate the pull request. If no action is taken within 7 days, the pull request will be marked closed.

@github-actions github-actions Bot added the Stale label Oct 20, 2024
@github-actions github-actions Bot closed this Nov 4, 2024
@tuunit tuunit reopened this Jan 4, 2025
@tuunit tuunit added this to the v8.0.0 milestone Jan 4, 2025
@github-actions github-actions Bot removed the Stale label Jan 5, 2025
@tuunit tuunit force-pushed the use-mapstructures-for-parsing-and-merging branch from 4941bf1 to 629ac24 Compare February 1, 2025 09:20
@github-actions github-actions Bot added the dependencies Pull requests that update a dependency file label Feb 1, 2025
@tuunit tuunit force-pushed the use-mapstructures-for-parsing-and-merging branch 2 times, most recently from a0420ca to f93271a Compare November 16, 2025 21:36
remove color output in tests for better readability in github actions

bugfix: remove google as default provider for alpha options

fix conversion flow for toml to yaml

revert ginkgo color deactivation

revert claim- and secret source back to pointers

regenerate alpha config

Signed-off-by: Jan Larwig <jan@larwig.com>
Signed-off-by: Jan Larwig <jan@larwig.com>
Signed-off-by: Jan Larwig <jan@larwig.com>
Signed-off-by: Jan Larwig <jan@larwig.com>
Signed-off-by: Jan Larwig <jan@larwig.com>
Signed-off-by: Jan Larwig <jan@larwig.com>
Signed-off-by: Jan Larwig <jan@larwig.com>
Signed-off-by: Jan Larwig <jan@larwig.com>
Signed-off-by: Jan Larwig <jan@larwig.com>
Signed-off-by: Jan Larwig <jan@larwig.com>
Signed-off-by: Jan Larwig <jan@larwig.com>
Signed-off-by: Jan Larwig <jan@larwig.com>
Signed-off-by: Jan Larwig <jan@larwig.com>
Signed-off-by: Jan Larwig <jan@larwig.com>
Signed-off-by: Jan Larwig <jan@larwig.com>
Signed-off-by: Jan Larwig <jan@larwig.com>
Signed-off-by: Jan Larwig <jan@larwig.com>
…oleans to pointers

Signed-off-by: Jan Larwig <jan@larwig.com>
@tuunit tuunit force-pushed the use-mapstructures-for-parsing-and-merging branch from f93271a to 15041dd Compare November 16, 2025 21:39
Comment thread pkg/apis/options/load.go Outdated
),
Metadata: nil, // Don't track any metadata
Result: result, // Decode the result into the prefilled options
TagName: "yaml", // Parse all fields that use the json tag
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Comment isn't updated based on this thread, also says json tag which is wrong

Comment thread pkg/apis/options/load.go Outdated
Comment on lines +185 to +189
// TBD: Uncomment as we add EnsureDefaults methods
// o.Cookie.EnsureDefaults()
// o.Session.EnsureDefaults()
// o.Templates.EnsureDefaults()
// o.Logging.EnsureDefaults()
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

When there is child defaulting to do, how do we ensure as we add new fields that we remember to call the default?

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I was already thinking about the issue on how to enforce that the EnsureDefaults() is called but didn't have a satisfactory idea yet

Signed-off-by: Jan Larwig <jan@larwig.com>
@tuunit tuunit merged commit e27921e into oauth2-proxy:master Nov 28, 2025
7 checks passed
Comment on lines 530 to +535
injectResponseHeaders:
- name: X-Secret
values:
- value: c2VjcmV0
- secretSource:
value: secret
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is now at odds with the documentation, which still says (in many places) that a secretSource value needs to be base64 encoded.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

dependencies Pull requests that update a dependency file docs e2e/failed go provider tests

Projects

Development

Successfully merging this pull request may close these issues.

6 participants