Skip to content

2025.8.0: Tight loop enqueuing SCIM/Google Workspace/Microsoft Entra sync tasks (not configured) when an LDAP source with many entries is enabled → Postgres “out of shared memory” #16314

@speedfreakw

Description

@speedfreakw

Describe the bug
After upgrading to authentik 2025.8.0, with an enabled LDAP source with many entries causes the worker to continuously enqueue and execute the following tasks in a tight loop, even though none of these providers (scim, google workspace, microsoft entra) are configured:

  • authentik.providers.scim.tasks.scim_sync_direct_dispatch
  • authentik.enterprise.providers.google_workspace.tasks.google_workspace_sync_direct_dispatch
  • authentik.enterprise.providers.microsoft_entra.tasks.microsoft_entra_sync_direct_dispatch

This leads to rapid lock growth in Postgres and eventually crashes with “FATAL: out of shared memory”. An instance without any LDAP source enabled does not exhibit the issue.

I belive it might be related to using an LDAP source, because a second instance of mine without an ldap source doesn't have the same issue.

steps to reproduce (not tested)

  1. Ensure SCIM, Google Workspace, and Microsoft Entra providers are not configured.
  2. Configure and enable an LDAP source with a large dataset (many entries).
  3. Upgrade to authentik 2025.8.0.
  4. let the LDAP sync run.
  5. Observe:
    • Worker logs spam “Task started/finished” for the three tasks above, multiple times per second. Switches to task enqued after a while if ldap sync is started because worker can't keep up.
    • pg_locks shows rapidly increasing ExclusiveLock counts.
    • Postgres log shows “FATAL: out of shared memory”.

Expected behavior

  • If SCIM/Google Workspace/Microsoft Entra providers are not configured, their sync tasks should not be enqueued.
  • LDAP sync should not indirectly trigger unrelated provider sync tasks.
  • Task scheduling should be bounded/rate-limited to avoid exhausting database locks/memory.

Logs

Postgres crash (excerpt):

authentik-postgresql  | 2025-08-21 17:18:14.249 CEST [59483] FATAL:  out of shared memory
authentik-postgresql  | 2025-08-21 17:18:14.249 CEST [59483] HINT:  You might need to increase max_locks_per_transaction.
authentik-worker      | ----------------------------------------
authentik-worker      | Exception occurred during processing of request from ('::ffff:127.0.0.1', 32774, 0, 0)
authentik-worker      | Traceback (most recent call last):
authentik-worker      |   File "/usr/local/lib/python3.13/socketserver.py", line 318, in _handle_request_noblock
authentik-worker      |     self.process_request(request, client_address)
authentik-worker      |     ~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^
authentik-worker      |   File "/usr/local/lib/python3.13/socketserver.py", line 349, in process_request
authentik-worker      |     self.finish_request(request, client_address)
authentik-worker      |     ~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^
authentik-worker      |   File "/usr/local/lib/python3.13/socketserver.py", line 362, in finish_request
authentik-worker      |     self.RequestHandlerClass(request, client_address, self)
authentik-worker      |     ~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
authentik-worker      |   File "/usr/local/lib/python3.13/socketserver.py", line 766, in __init__
authentik-worker      |     self.handle()
authentik-worker      |     ~~~~~~~~~~~^^
authentik-worker      |   File "/usr/local/lib/python3.13/http/server.py", line 436, in handle
authentik-worker      |     self.handle_one_request()
authentik-worker      |     ~~~~~~~~~~~~~~~~~~~~~~~^^
authentik-worker      |   File "/usr/local/lib/python3.13/http/server.py", line 424, in handle_one_request
authentik-worker      |     method()
authentik-worker      |     ~~~~~~^^
authentik-worker      |   File "/authentik/tasks/middleware.py", line 173, in do_HEAD
authentik-worker      |     db_conn.connect()
authentik-worker      |     ~~~~~~~~~~~~~~~^^
authentik-worker      |   File "/ak-root/.venv/lib/python3.13/site-packages/django/utils/asyncio.py", line 26, in inner
authentik-worker      |     return func(*args, **kwargs)
authentik-worker      |   File "/ak-root/.venv/lib/python3.13/site-packages/django/db/backends/base/base.py", line 255, in connect
authentik-worker      |     self.connection = self.get_new_connection(conn_params)
authentik-worker      |                       ~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^
authentik-worker      |   File "/ak-root/.venv/lib/python3.13/site-packages/django_prometheus/db/backends/postgresql/base.py", line 9, in get_new_connection
authentik-worker      |     conn = super().get_new_connection(*args, **kwargs)
authentik-worker      |   File "/ak-root/.venv/lib/python3.13/site-packages/django_prometheus/db/common.py", line 45, in get_new_connection
authentik-worker      |     return super().get_new_connection(*args, **kwargs)
authentik-worker      |            ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^
authentik-worker      |   File "/ak-root/.venv/lib/python3.13/site-packages/django/utils/asyncio.py", line 26, in inner
authentik-worker      |     return func(*args, **kwargs)
authentik-worker      |   File "/ak-root/.venv/lib/python3.13/site-packages/django/db/backends/postgresql/base.py", line 332, in get_new_connection
authentik-worker      |     connection = self.Database.connect(**conn_params)
authentik-worker      |   File "/ak-root/.venv/lib/python3.13/site-packages/psycopg/connection.py", line 118, in connect
authentik-worker      |     raise last_ex.with_traceback(None)
authentik-worker      | psycopg.OperationalError: connection failed: connection to server at "172.22.0.3", port 5432 failed: FATAL:  out of shared memory
authentik-worker      | HINT:  You might need to increase max_locks_per_transaction.
authentik-worker      | ----------------------------------------
authentik-worker      | {"event": "Consumer encountered a connection error: connection failed: connection to server at \"172.22.0.3\", port 5432 failed: FATAL:  out of shared memory\nHINT:  You might need to increase max_locks_per_transaction.", "level": "critical", "logger": "dramatiq.worker.ConsumerThread(default.DQ)", "timestamp": "2025-08-21T15:18:15.113311"}
authentik-worker      | {"event": "Restarting consumer in 3.00 seconds.", "level": "info", "logger": "dramatiq.worker.ConsumerThread(default.DQ)", "timestamp": "2025-08-21T15:18:15.113418"}
authentik-postgresql  | 2025-08-21 17:18:15.112 CEST [59484] FATAL:  out of shared memory
authentik-postgresql  | 2025-08-21 17:18:15.112 CEST [59484] HINT:  You might need to increase max_locks_per_transaction.

Task activity spam (excerpt):

authentik-worker      | {"domain_url": null, "event": "Task started", "level": "info", "logger": "authentik.tasks.middleware", "pid": 47, "schema_name": "public", "task_id": "57043120-0eb7-4634-9fc4-2b5c14d6e003", "task_name": "authentik.providers.scim.tasks.scim_sync_direct_dispatch", "timestamp": "2025-08-21T18:03:34.097409"}
authentik-worker      | {"domain_url": null, "event": "Task started", "level": "info", "logger": "authentik.tasks.middleware", "pid": 47, "schema_name": "public", "task_id": "2cf9f052-595c-489e-8c18-dd1e9d6fb2e6", "task_name": "authentik.enterprise.providers.google_workspace.tasks.google_workspace_sync_direct_dispatch", "timestamp": "2025-08-21T18:03:34.103704"}
authentik-worker      | {"domain_url": null, "event": "Task finished", "exc": null, "level": "info", "logger": "authentik.tasks.middleware", "pid": 47, "schema_name": "public", "task_id": "57043120-0eb7-4634-9fc4-2b5c14d6e003", "task_name": "authentik.providers.scim.tasks.scim_sync_direct_dispatch", "timestamp": "2025-08-21T18:03:34.115746"}
authentik-worker      | {"domain_url": null, "event": "Task finished", "exc": null, "level": "info", "logger": "authentik.tasks.middleware", "pid": 47, "schema_name": "public", "task_id": "2cf9f052-595c-489e-8c18-dd1e9d6fb2e6", "task_name": "authentik.enterprise.providers.google_workspace.tasks.google_workspace_sync_direct_dispatch", "timestamp": "2025-08-21T18:03:34.121907"}
authentik-worker      | {"domain_url": null, "event": "Task started", "level": "info", "logger": "authentik.tasks.middleware", "pid": 47, "schema_name": "public", "task_id": "a684e913-fc20-4fb9-82d4-1f11edec14f2", "task_name": "authentik.providers.scim.tasks.scim_sync_direct_dispatch", "timestamp": "2025-08-21T18:03:34.140334"}
authentik-worker      | {"domain_url": null, "event": "Task started", "level": "info", "logger": "authentik.tasks.middleware", "pid": 47, "schema_name": "public", "task_id": "8fa84d21-c4fb-44e9-a1cb-8c3ea1a70189", "task_name": "authentik.enterprise.providers.microsoft_entra.tasks.microsoft_entra_sync_direct_dispatch", "timestamp": "2025-08-21T18:03:34.143802"}
authentik-worker      | {"domain_url": null, "event": "Task finished", "exc": null, "level": "info", "logger": "authentik.tasks.middleware", "pid": 47, "schema_name": "public", "task_id": "a684e913-fc20-4fb9-82d4-1f11edec14f2", "task_name": "authentik.providers.scim.tasks.scim_sync_direct_dispatch", "timestamp": "2025-08-21T18:03:34.153991"}
... repeats multiple times per second ...

authentik-worker      | {"domain_url": null, "event": "Task enqueued", "level": "info", "logger": "authentik.tasks.middleware", "pid": 49, "schema_name": "public", "task_id": "60fca088-95e3-48c3-ab5a-916f56740dae", "task_name": "authentik.enterprise.providers.google_workspace.tasks.google_workspace_sync_direct_dispatch", "timestamp": "2025-08-21T18:01:32.809038"}
authentik-worker      | {"domain_url": null, "event": "Task enqueued", "level": "info", "logger": "authentik.tasks.middleware", "pid": 49, "schema_name": "public", "task_id": "cf0d9f14-5c2d-4b95-b3ba-5f096cd69ef0", "task_name": "authentik.enterprise.providers.microsoft_entra.tasks.microsoft_entra_sync_direct_dispatch", "timestamp": "2025-08-21T18:01:32.815200"}
authentik-worker      | {"domain_url": null, "event": "Task finished", "exc": null, "level": "info", "logger": "authentik.tasks.middleware", "pid": 49, "schema_name": "public", "task_id": "00d17987-8f91-473d-af0c-a9ac9ce484e6", "task_name": "authentik.sources.ldap.tasks.ldap_sync_page", "timestamp": "2025-08-21T18:01:33.010876"}
authentik-worker      | {"domain_url": null, "event": "Task started", "level": "info", "logger": "authentik.tasks.middleware", "pid": 49, "schema_name": "public", "task_id": "9d2c8ec4-23df-4999-82d5-32624634f645", "task_name": "authentik.sources.ldap.tasks.ldap_sync_page", "timestamp": "2025-08-21T18:01:33.030772"}
authentik-worker      | {"domain_url": null, "event": "Task enqueued", "level": "info", "logger": "authentik.tasks.middleware", "pid": 49, "schema_name": "public", "task_id": "dce1b762-f7ea-4e6d-ad09-080b8ad0d50e", "task_name": "authentik.providers.scim.tasks.scim_sync_direct_dispatch", "timestamp": "2025-08-21T18:01:33.678772"}
authentik-worker      | {"domain_url": null, "event": "Task enqueued", "level": "info", "logger": "authentik.tasks.middleware", "pid": 49, "schema_name": "public", "task_id": "184ce4de-8584-4c94-8028-409bbd1c61a8", "task_name": "authentik.enterprise.providers.google_workspace.tasks.google_workspace_sync_direct_dispatch", "timestamp": "2025-08-21T18:01:33.684895"}
authentik-worker      | {"domain_url": null, "event": "Task enqueued", "level": "info", "logger": "authentik.tasks.middleware", "pid": 49, "schema_name": "public", "task_id": "fc8e46b7-e362-4435-aeb7-58f21126b620", "task_name": "authentik.enterprise.providers.microsoft_entra.tasks.microsoft_entra_sync_direct_dispatch", "timestamp": "2025-08-21T18:01:33.689974"}
authentik-worker      | {"domain_url": null, "event": "Task enqueued", "level": "info", "logger": "authentik.tasks.middleware", "pid": 49, "schema_name": "public", "task_id": "6e4f77c1-40ea-466c-b62e-3b01e7d1a202", "task_name": "authentik.providers.scim.tasks.scim_sync_direct_dispatch", "timestamp": "2025-08-21T18:01:33.888572"}
authentik-worker      | {"domain_url": null, "event": "Task enqueued", "level": "info", "logger": "authentik.tasks.middleware", "pid": 49, "schema_name": "public", "task_id": "f5a3da8e-419b-49bd-b991-90dc51229044", "task_name": "authentik.enterprise.providers.google_workspace.tasks.google_workspace_sync_direct_dispatch", "timestamp": "2025-08-21T18:01:33.893827"}
authentik-worker      | {"domain_url": null, "event": "Task enqueued", "level": "info", "logger": "authentik.tasks.middleware", "pid": 49, "schema_name": "public", "task_id": "f80fa638-14aa-4034-8688-1a6a524c79d5", "task_name": "authentik.enterprise.providers.microsoft_entra.tasks.microsoft_entra_sync_direct_dispatch", "timestamp": "2025-08-21T18:01:33.898800"}
authentik-worker      | {"domain_url": null, "event": "Task enqueued", "level": "info", "logger": "authentik.tasks.middleware", "pid": 49, "schema_name": "public", "task_id": "c2c8f97b-e3eb-4e23-8bbe-83e862026f64", "task_name": "authentik.providers.scim.tasks.scim_sync_direct_dispatch", "timestamp": "2025-08-21T18:01:34.099370"}
authentik-worker      | {"domain_url": null, "event": "Task enqueued", "level": "info", "logger": "authentik.tasks.middleware", "pid": 49, "schema_name": "public", "task_id": "b684c015-ae7c-4764-bf96-92b5bb07aaa4", "task_name": "authentik.enterprise.providers.google_workspace.tasks.google_workspace_sync_direct_dispatch", "timestamp": "2025-08-21T18:01:34.104540"}
... repeats multiple times per second ...

Postgres lock growth while issue is active (samples):

docker exec -it authentik-postgresql psql -U authentik -d authentik -c "SELECT mode, count(*) FROM pg_locks GROUP BY mode ORDER BY count(*) DESC;"
      mode       | count 
-----------------+-------
 ExclusiveLock   |  14402
 AccessShareLock |     1

Version and Deployment

  • authentik version: 2025.8.0
  • Deployment: docker-compose
  • Postgres 16-alpine: standard container from authentik’s stack

Additional context

  • The problem only occurs on the instance where an LDAP source (with many entries) is enabled. A second instance without LDAP behaves normally.
  • The three provider “direct_dispatch” tasks appear to get re-enqueued immediately upon finishing, suggesting a feedback loop possibly triggered by LDAP sync events or signals.
  • Request: prevent auto-enqueue of these provider tasks when providers are not configured, and/or add safeguards (rate limits/locks) to avoid tight-loop re-enqueue driven by LDAP sync activity.

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't workingbug/confirmedConfirmed bugs

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions