Snow 893080 feat bulk save base model#673
Merged
Merged
Conversation
…atching Introduce src/snowflake/sqlalchemy/orm.py with three public utilities: - _snowflake_constructor: custom ORM __init__ that pre-populates every plain-nullable mapped column with None (or its scalar default) at construction time, so every instance always has the same state_dict key set regardless of which kwargs the caller supplied. Mirrors SA's mapper._insert_cols_as_none exclusion logic: primary keys, server_default columns, callable defaults, SQL-expression defaults, and should_evaluate_none columns are intentionally left absent. - snowflake_declarative_base(): function-based factory (works in both SA 1.4 and SA 2.x) that installs _snowflake_constructor via declarative_base(constructor=...). - SnowflakeBase: SA 2.x DeclarativeBase subclass whose __init__ delegates to _snowflake_constructor. Guarded behind IS_VERSION_20 so the module loads correctly under SA 1.4. - SnowflakeSession: Session subclass that overrides bulk_save_objects to call _bulk_save_mappings with render_nulls=True. Without this flag, SA strips None values from the parameter dict before grouping rows, so constructor-pre-populated Nones would still be stripped and objects with col=None vs col='value' would land in different INSERT batches. Both parts are required: the base class normalises the param-key set; the session override prevents those Nones from being stripped. All three names are exported from snowflake.sqlalchemy.__init__.py. Unit tests in tests/test_unit_orm.py verify (no DB required): - plain nullable column pre-populated with None - primary key NOT pre-populated - server_default column NOT pre-populated - scalar-default column pre-populated with the scalar value - callable-default column NOT pre-populated - SQL-expression-default column NOT pre-populated - user-supplied values preserved and override defaults - invalid kwargs raise TypeError - all objects produce identical state_dict key sets (core invariant) - SnowflakeSession calls _bulk_save_mappings with render_nulls=True - public exports available from snowflake.sqlalchemy Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> SNOW-893080: [N] refine: fix __all__ export, add missing tests, apply lint fixes - Add SnowflakeBase to __all__ (conditionally on SA 2.x) so it is exposed via wildcard imports - Add should_evaluate_none (JSON) column to test model and verify it is NOT pre-populated by _snowflake_constructor - Add test for user-supplied value on server_default column - Add end-to-end test combining SnowflakeBase + SnowflakeSession to verify uniform parameter-key sets and single _bulk_save_mappings call - Add empty-list edge case test for bulk_save_objects - Add __all__ membership tests for all three public exports - Apply pyupgrade/black/isort formatting fixes from pre-commit hooks Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> [D] SNOW-893080: add README docs for SnowflakeBase and SnowflakeSession Add a new "Bulk Insert Optimization for ORM Models" section to README.md documenting SnowflakeBase, snowflake_declarative_base(), and SnowflakeSession. Covers when/why to use these utilities, SA 1.4 vs 2.x differences, the pairing requirement between the base class and SnowflakeSession, and full code examples for both SA versions. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
a4f0afe to
a7741e5
Compare
sfc-gh-jcieslak
approved these changes
Apr 17, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to subscribe to this conversation on GitHub.
Already have an account?
Sign in.
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Please answer these questions before submitting your pull requests. Thanks!
What GitHub issue is this PR addressing? Make sure that there is an accompanying issue to your PR.
Fixes SNOW-893080:
session.bulk_save_objectsdoes not put all objects in one INSERT #441Fill out the following pre-review checklist:
Please describe how your code solves the related issue.
Fixes #441.
session.bulk_save_objects()emits O(N)INSERTstatements when ORM models have randomly-populated nullable columns, because SQLAlchemy groups rows by their set ofnon-None parameter keys.
How
Two new components in
src/snowflake/sqlalchemy/orm.py:SnowflakeBase/snowflake_declarative_base()— declarative base whose__init__pre-populates every plain-nullable column withNone(or its scalar default) so all instances share the same key set. Primary keys, server_default, callable, and SQL-expression defaults are excluded, mirroring SQLAlchemy's own_insert_cols_as_nonelogic.SnowflakeSession— Session subclass that passesrender_nulls=Trueto the internal bulk-save call, preventing pre-populated Nones from being stripped before grouping.Both parts are required.
SnowflakeBaseis SA 2.x only;snowflake_declarative_base()works on SA 1.4 and 2.x. All three names exported from snowflake.sqlalchemy.Tests
Unit tests in
tests/test_unit_orm.py(no database required) cover column pre-population rules, key-set uniformity,render_nulls=Truecall, and public exports.