Skip to content

root: Improve Redis config and support#7118

Closed
PKizzle wants to merge 380 commits intogoauthentik:mainfrom
PKizzle:feature/improve-redis-config
Closed

root: Improve Redis config and support#7118
PKizzle wants to merge 380 commits intogoauthentik:mainfrom
PKizzle:feature/improve-redis-config

Conversation

@PKizzle
Copy link
Contributor

@PKizzle PKizzle commented Oct 9, 2023

Details

Resolves #5077, resolves #1285, resolves #3979, resolves #5531

External source code

Go

Python

Changes

New Features

  • Uses one URL for configuration of Redis (shared between Go and Python)
  • The Python URL parser has been created in a way to resemble the Go version
  • By default uses Redis for broker, channel, backend (current state), but also allows for custom options
  • Moves part of the configuration currently under redis category into new dedicated sections -> split into root: Restructure broker / cache / channel / result configuration #7097
  • Adds deprecation check to config parser to allow for easy transition between old and new configuration scheme
  • Old Redis configuration env vars are automatically converted to a Redis config URL if no Redis config URL has been configured otherwise they are ignored
  • The new Redis URL allows environment variables to be included in bash style ${VAR}

Breaking Changes

  • None -> Old configurations are still supported however users are advised to switch to the new URL-based configuration

Additional information

  • Newly supported Redis configurations are only available using the new Redis config URL environment variable
  • The Redis config URL requires special characters in parameters to be URL encoded (e.g. username, password)
  • Has been successfully tested using Redis cluster, sentinel and Unix socket connections

Todo

Sentinel

  • Handle ReadOnly error when master is demoted to slave
  • Fix unittests

Cluster

  • Remove all usage of KEYS, MGET and MSET in Django and Celery
  • Fix unittests
  • Check all connections are correctly closed e.g. in redis_middleware_channels.py

PKizzle added 30 commits May 10, 2023 16:12
…feature/add-redis-sentinel-support

# Conflicts:
#	go.mod
#	go.sum
# Conflicts:
#	Dockerfile
#	authentik/lib/default.yml
#	authentik/root/settings.py
#	go.mod
#	go.sum
#	poetry.lock
# Conflicts:
#	authentik/api/v3/config.py
#	authentik/flows/planner.py
#	authentik/lib/config.py
#	authentik/lib/tests/test_config.py
#	authentik/policies/process.py
#	authentik/policies/reputation/signals.py
#	authentik/root/settings.py
#	lifecycle/system_migrations/to_0_13_authentik.py
#	lifecycle/wait_for_db.py
#	poetry.lock
# Conflicts:
#	.gitignore
#	authentik/api/v3/config.py
#	authentik/flows/planner.py
#	authentik/lib/tests/test_config.py
#	authentik/policies/process.py
#	authentik/policies/reputation/signals.py
#	authentik/root/settings.py
#	lifecycle/wait_for_db.py
#	poetry.lock
# Conflicts:
#	go.mod
#	go.sum
#	poetry.lock
# Conflicts:
#	internal/outpost/proxyv2/application/session.go
#	poetry.lock
Last commit that prefers old Redis env vars over Redis config URL env var. Will change logic to always prefer a configured Redis config URL and only generate one using the old env vars if necessary.
Contrary to the initial implementation the Redis config URL will not always be updated with Redis environment variables. Instead the Redis config URL can include bash style placeholders ${VAR} that reference those environment variables. If `AUTHENTIK_REDIS__URL` is not set i.e. in an old Authentik setup the Redis config URL is automatically generated and a warning message is issued to make the user aware of the necessary future configuration changes.
# Conflicts:
#	internal/config/config.go
#	internal/outpost/proxyv2/application/session.go
Redis socket connections do not have socket keepalive options. Hence they shall not be present even if configured by the user.
@fullykubed
Copy link

I believe that this would resolve #9121 as well

PKizzle added 5 commits April 7, 2024 04:28
# Conflicts:
#	.github/actions/setup/action.yml
#	Dockerfile
#	lifecycle/wait_for_db.py
#	poetry.lock
#	proxy.Dockerfile
# Conflicts:
#	Dockerfile
#	authentik/lib/config.py
#	authentik/lib/default.yml
#	authentik/root/settings.py
#	go.mod
#	go.sum
#	internal/config/struct.go
#	internal/outpost/proxyv2/application/session.go
#	lifecycle/wait_for_db.py
#	poetry.lock
#	proxy.Dockerfile
#	website/docs/installation/configuration.mdx
@benedikt-bartscher
Copy link

Hi @PKizzle do you plan to continue this PR? Can we merge main to test it against latest authentik?
Thanks 🙏

@PKizzle
Copy link
Contributor Author

PKizzle commented Oct 17, 2024

@benedikt-bartscher I have not yet found the time to continue working on the PR and it will require quite some work to merge main again. The biggest issue however is that the code coverage is quite low as a lot of additional code had to be added in order to make redis-py behave correctly. To write these tests requires some deep knowledge of Redis that I do not have. (And since most of the code is deeply connected with redis-py any update on their side can easily break the current implementation). I have not fully given up on this PR but had hoped to see some improvements over at redis-py that would make maintaining it long-term much easier.

# Conflicts:
#	.github/actions/setup/action.yml
#	.github/actions/setup/docker-compose.yml
#	.github/workflows/ci-main.yml
#	Dockerfile
#	authentik/lib/tests/test_config.py
#	authentik/root/settings.py
#	authentik/root/tests.py
#	go.mod
#	go.sum
#	internal/config/struct.go
#	internal/outpost/proxyv2/application/session.go
#	lifecycle/wait_for_db.py
#	poetry.lock
#	proxy.Dockerfile
#	website/developer-docs/setup/full-dev-environment.md
#	website/docs/installation/configuration.mdx
@BeryJu
Copy link
Member

BeryJu commented Oct 1, 2025

Gonna close this since with 2025.10 we're fully removing redis as a dependency so this won't be needed anymore

@BeryJu BeryJu closed this Oct 1, 2025
@ojsef39
Copy link

ojsef39 commented Oct 5, 2025

Gonna close this since with 2025.10 we're fully removing redis as a dependency so this won't be needed anymore

sorry if i ask here and that information is stated obvious somewhere, but is there a release timeline for 2025.10? i have pretty dumb problems with redis or better said the subchart in my cluster and im wondering if its really worth it to still fix it or just wait it out :)

Thanks :)

@BeryJu
Copy link
Member

BeryJu commented Oct 5, 2025

we're targetting mid-late october, there should be a release candidate in a week or so

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Redis Cluster or other mechanisms available as ref: #3979 Redis (Unix) Socket? Add redis cluster support Support redis sentinel

7 participants