Skip to content

SQLA: Replace the deprecated lazy="noload" with lazy="raise"#64262

Merged
jscheffl merged 1 commit into
apache:mainfrom
Dev-iL:2603/avoid_noload
Mar 28, 2026
Merged

SQLA: Replace the deprecated lazy="noload" with lazy="raise"#64262
jscheffl merged 1 commit into
apache:mainfrom
Dev-iL:2603/avoid_noload

Conversation

@Dev-iL

@Dev-iL Dev-iL commented Mar 26, 2026

Copy link
Copy Markdown
Collaborator

SQLAlchemy 2.1 deprecated the noload lazy loading strategy (sqlalchemy/sqlalchemy#11045). noload silently returns None/empty collections — essentially incorrect results — and will be removed in a future release.

This PR replaces all 5 occurrences of lazy="noload" with lazy="raise", which raises InvalidRequestError if the relationship is accessed without an explicit eager load (e.g. joinedload). This is a safe drop-in that also catches missing eager loads at development time instead of silently returning None.

Changed models:

  • Log.task_instance
  • TaskInstance.rendered_task_instance_fields
  • TaskInstance.hitl_detail
  • TaskInstanceHistory.hitl_detail
  • XComModel.task

Additional fixes required by switching from noload to raise:

  1. passive_deletes=True on TaskInstance.rendered_task_instance_fields and TaskInstance.hitl_detail: With noload, SQLAlchemy silently skipped ORM-level cascade processing. With raise, it attempts ORM cascade which fails because RenderedTaskInstanceFields uses a composite PK (also the FK), so SQLAlchemy cannot null out FK columns during delete. Adding passive_deletes=True tells SQLAlchemy to defer to the DB-level ON DELETE CASCADE instead.

  2. Eager-load rendered_task_instance_fields in HITL endpoints: The get_hitl_detail and get_hitl_details endpoints serialize a TaskInstanceResponse which accesses rendered_task_instance_fields via a Pydantic AliasPath. With lazy="raise" this raised during serialization. Fixed by adding joinedload(TI.rendered_task_instance_fields) to the query options, and returning an explicitly serialized Pydantic model from get_hitl_detail (same pattern as update_hitl_detail).

related: #61229


Was generative AI tooling used to co-author this PR?
  • Yes (please specify the tool below)

Generated-by: Claude Sonnet 4.6 following the guidelines

@Dev-iL Dev-iL requested review from XD-DENG and ashb as code owners March 26, 2026 15:10
@Dev-iL Dev-iL requested review from XD-DENG, ashb, jscheffl, potiuk and vincbeck and removed request for XD-DENG and ashb March 26, 2026 15:10
SQLAlchemy 2.1 deprecated the `noload` lazy loading strategy
(sqlalchemy/sqlalchemy#11045). `noload` silently returns `None`/empty
collections — essentially incorrect results — and will be removed in a
future release.

This PR replaces all 5 occurrences of `lazy="noload"` with `lazy="raise"`,
which raises `InvalidRequestError` if the relationship is accessed without
an explicit eager load (e.g. `joinedload`). All affected relationships are
already properly loaded via `joinedload()` wherever they're accessed, so
this is a safe drop-in that also catches missing eager loads at development
time instead of silently returning `None`.

Two callers needed fixes to work correctly with `lazy="raise"`:
- `TaskInstance.rendered_task_instance_fields` and `TaskInstance.hitl_detail`
  needed `passive_deletes=True` to tell SQLAlchemy to rely on the DB-level
  `ON DELETE CASCADE` rather than attempting ORM-level cascade processing
  (which would fail since FK columns are also PK columns on RTIF, and
  `lazy="raise"` prevents the ORM from loading the collection to clear them).
- The HITL API endpoints needed `joinedload(TI.rendered_task_instance_fields)`
  added to their queries, since `TaskInstanceResponse` accesses
  `rendered_task_instance_fields` during Pydantic serialization. The
  `get_hitl_detail` endpoint also needed an explicit `model_validate()` call
  so serialization happens while the session is still active.

**Changed models:**
- `Log.task_instance`
- `TaskInstance.rendered_task_instance_fields`
- `TaskInstance.hitl_detail`
- `TaskInstanceHistory.hitl_detail`
- `XComModel.task`

related: apache#61229
@Dev-iL Dev-iL force-pushed the 2603/avoid_noload branch from 1c184d3 to 7920321 Compare March 27, 2026 20:08
@jscheffl jscheffl merged commit 06930d1 into apache:main Mar 28, 2026
282 of 283 checks passed
@Dev-iL Dev-iL deleted the 2603/avoid_noload branch March 28, 2026 11:00
nailo2c pushed a commit to nailo2c/airflow that referenced this pull request Mar 30, 2026
…che#64262)

SQLAlchemy 2.1 deprecated the `noload` lazy loading strategy
(sqlalchemy/sqlalchemy#11045). `noload` silently returns `None`/empty
collections — essentially incorrect results — and will be removed in a
future release.

This PR replaces all 5 occurrences of `lazy="noload"` with `lazy="raise"`,
which raises `InvalidRequestError` if the relationship is accessed without
an explicit eager load (e.g. `joinedload`). All affected relationships are
already properly loaded via `joinedload()` wherever they're accessed, so
this is a safe drop-in that also catches missing eager loads at development
time instead of silently returning `None`.

Two callers needed fixes to work correctly with `lazy="raise"`:
- `TaskInstance.rendered_task_instance_fields` and `TaskInstance.hitl_detail`
  needed `passive_deletes=True` to tell SQLAlchemy to rely on the DB-level
  `ON DELETE CASCADE` rather than attempting ORM-level cascade processing
  (which would fail since FK columns are also PK columns on RTIF, and
  `lazy="raise"` prevents the ORM from loading the collection to clear them).
- The HITL API endpoints needed `joinedload(TI.rendered_task_instance_fields)`
  added to their queries, since `TaskInstanceResponse` accesses
  `rendered_task_instance_fields` during Pydantic serialization. The
  `get_hitl_detail` endpoint also needed an explicit `model_validate()` call
  so serialization happens while the session is still active.

**Changed models:**
- `Log.task_instance`
- `TaskInstance.rendered_task_instance_fields`
- `TaskInstance.hitl_detail`
- `TaskInstanceHistory.hitl_detail`
- `XComModel.task`

related: apache#61229
Suraj-kumar00 pushed a commit to Suraj-kumar00/airflow that referenced this pull request Apr 7, 2026
…che#64262)

SQLAlchemy 2.1 deprecated the `noload` lazy loading strategy
(sqlalchemy/sqlalchemy#11045). `noload` silently returns `None`/empty
collections — essentially incorrect results — and will be removed in a
future release.

This PR replaces all 5 occurrences of `lazy="noload"` with `lazy="raise"`,
which raises `InvalidRequestError` if the relationship is accessed without
an explicit eager load (e.g. `joinedload`). All affected relationships are
already properly loaded via `joinedload()` wherever they're accessed, so
this is a safe drop-in that also catches missing eager loads at development
time instead of silently returning `None`.

Two callers needed fixes to work correctly with `lazy="raise"`:
- `TaskInstance.rendered_task_instance_fields` and `TaskInstance.hitl_detail`
  needed `passive_deletes=True` to tell SQLAlchemy to rely on the DB-level
  `ON DELETE CASCADE` rather than attempting ORM-level cascade processing
  (which would fail since FK columns are also PK columns on RTIF, and
  `lazy="raise"` prevents the ORM from loading the collection to clear them).
- The HITL API endpoints needed `joinedload(TI.rendered_task_instance_fields)`
  added to their queries, since `TaskInstanceResponse` accesses
  `rendered_task_instance_fields` during Pydantic serialization. The
  `get_hitl_detail` endpoint also needed an explicit `model_validate()` call
  so serialization happens while the session is still active.

**Changed models:**
- `Log.task_instance`
- `TaskInstance.rendered_task_instance_fields`
- `TaskInstance.hitl_detail`
- `TaskInstanceHistory.hitl_detail`
- `XComModel.task`

related: apache#61229
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants