Skip to content

fix strict mode validation for nested filters#7264

Merged
generall merged 2 commits intodevfrom
fix-strict-mode-for-nested
Sep 16, 2025
Merged

fix strict mode validation for nested filters#7264
generall merged 2 commits intodevfrom
fix-strict-mode-for-nested

Conversation

@generall
Copy link
Member

Related to #7250

Strict mode check was missing wildcard

@generall generall requested a review from agourlay September 16, 2025 08:30
coderabbitai[bot]

This comment was marked as resolved.

@qdrant qdrant deleted a comment from coderabbitai bot Sep 16, 2025
Copy link
Member

@timvisee timvisee left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the test 👍

@coderabbitai
Copy link
Contributor

coderabbitai bot commented Sep 16, 2025

📝 Walkthrough

Walkthrough

  • In lib/collection/src/problems/unindexed_field.rs, Extractor::update_from_condition now extends JSON paths for Condition::Nested using nested.raw_key().array_key() instead of nested.raw_key().
  • In lib/collection/src/tests/payload.rs, test expectations and index creation paths were updated from "location.lat" to "location[].lat" to align with array-aware nested paths.
  • Added tests/openapi/test_strict_mode_nested.py to validate strict mode behavior with nested filters: creates indices at country.cities[].population and country.cities[].sightseeing, asserts failure before both indices exist, and success after.

Estimated code review effort

🎯 3 (Moderate) | ⏱️ ~20 minutes

Possibly related PRs

Suggested reviewers

  • coszio

Pre-merge checks and finishing touches

❌ Failed checks (1 warning)
Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 25.00% which is insufficient. The required threshold is 80.00%. You can run @coderabbitai generate docstrings to improve docstring coverage.
✅ Passed checks (2 passed)
Check name Status Explanation
Title Check ✅ Passed The PR title "fix strict mode validation for nested filters" is concise and directly describes the primary change—addressing strict-mode validation for nested filters by adding the missing wildcard—so it accurately reflects the code and test changes in this changeset.
Description Check ✅ Passed The PR description references the related issue and succinctly states the root cause ("Strict mode check was missing wildcard"), which is directly related to the code and test changes that adjust nested/array-aware path handling, so it is on-topic and satisfies this lenient check.
✨ Finishing touches
  • 📝 Generate Docstrings
🧪 Generate unit tests
  • Create PR with unit tests
  • Post copyable unit tests in a comment
  • Commit unit tests in branch fix-strict-mode-for-nested

Tip

👮 Agentic pre-merge checks are now available in preview!

Pro plan users can now enable pre-merge checks in their settings to enforce checklists before merging PRs.

  • Built-in checks – Quickly apply ready-made checks to enforce title conventions, require pull request descriptions that follow templates, validate linked issues for compliance, and more.
  • Custom agentic checks – Define your own rules using CodeRabbit’s advanced agentic capabilities to enforce organization-specific policies and workflows. For example, you can instruct CodeRabbit’s agent to verify that API documentation is updated whenever API schema files are modified in a PR. Note: Upto 5 custom checks are currently allowed during the preview period. Pricing for this feature will be announced in a few weeks.

Please see the documentation for more information.

Example:

reviews:
  pre_merge_checks:
    custom_checks:
      - name: "Undocumented Breaking Changes"
        mode: "warning"
        instructions: |
          Pass/fail criteria: All breaking changes to public APIs, CLI flags, environment variables, configuration keys, database schemas, or HTTP/GraphQL endpoints must be documented in the "Breaking Change" section of the PR description and in CHANGELOG.md. Exclude purely internal or private changes (e.g., code not exported from package entry points or explicitly marked as internal).

Please share your feedback with us on this Discord post.


Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

Copy link
Contributor

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 0

🧹 Nitpick comments (3)
tests/openapi/test_strict_mode_nested.py (3)

95-97: Index type for sightseeing should match payload (strings)
sightseeing is an array of strings; using "integer" may confuse future readers and tools. "keyword" is the appropriate type; it still satisfies values_count in strict mode.

Apply:

-            "field_name": "country.cities[].sightseeing",
-            "field_schema": "integer"
+            "field_name": "country.cities[].sightseeing",
+            "field_schema": "keyword"

84-86: Assert the expected error code, not just “not ok”
Make the failure explicit to guard against false positives.

Apply:

-    assert not response.ok
+    assert response.status_code in (400, 422)

140-140: Typo in comment: “Tokio” → “Tokyo”

Apply:

-    # Only Tokio has more than 8M inhabitants and less than 3 sightseeing
+    # Only Tokyo has more than 8M inhabitants and less than 3 sightseeing
📜 Review details

Configuration used: CodeRabbit UI

Review profile: CHILL

Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between 634efc5 and 27b70ea.

📒 Files selected for processing (3)
  • lib/collection/src/problems/unindexed_field.rs (1 hunks)
  • lib/collection/src/tests/payload.rs (1 hunks)
  • tests/openapi/test_strict_mode_nested.py (1 hunks)
🧰 Additional context used
📓 Path-based instructions (3)
**/*.rs

📄 CodeRabbit inference engine (.github/review-rules.md)

**/*.rs: Prefer explicit SomeType::from(x) over implicit x.into() in Rust code
Do not use transmute_from_u8, transmute_to_u8, transmute_from_u8_to_slice, transmute_from_u8_to_mut_slice, transmute_to_u8_slice in new code; use bytemuck or zerocopy instead

Files:

  • lib/collection/src/problems/unindexed_field.rs
  • lib/collection/src/tests/payload.rs
**/src/**/*.rs

📄 CodeRabbit inference engine (.github/review-rules.md)

**/src/**/*.rs: Prefer exhaustive match arms over a catch-all _ arm to avoid missing new enum variants (except in tests/benchmarks or when provably safe)
Prefer explicit field ignoring with : _ over .. in struct patterns (except in tests/benchmarks or when provably safe)

Files:

  • lib/collection/src/problems/unindexed_field.rs
  • lib/collection/src/tests/payload.rs
**/{tests,benches}/**/*.rs

📄 CodeRabbit inference engine (.github/review-rules.md)

Using .unwrap() and panic!() in tests and benchmarks is acceptable and should not be flagged

Files:

  • lib/collection/src/tests/payload.rs
🧠 Learnings (1)
📚 Learning: 2025-08-10T18:30:02.986Z
Learnt from: generall
PR: qdrant/qdrant#7006
File: lib/collection/src/operations/verification/update.rs:158-174
Timestamp: 2025-08-10T18:30:02.986Z
Learning: In Qdrant's strict mode verification code (lib/collection/src/operations/verification/update.rs), exhaustive pattern matching without `..` is intentionally used for structs like PointsBatch and PointsList. This design pattern ensures compilation fails when new fields are added, forcing developers to explicitly consider how new fields should be handled in the indexed_filter_write method. This provides visibility and compile-time safety for struct evolution.

Applied to files:

  • lib/collection/src/tests/payload.rs
🧬 Code graph analysis (3)
tests/openapi/test_strict_mode_nested.py (3)
tests/openapi/helpers/helpers.py (1)
  • request_with_validation (41-116)
tests/openapi/test_nested_payload_indexing.py (1)
  • nested_payload_collection_setup (7-221)
tests/openapi/conftest.py (2)
  • on_disk_vectors (11-12)
  • on_disk_payload (16-17)
lib/collection/src/problems/unindexed_field.rs (1)
lib/segment/src/json_path/mod.rs (1)
  • extend_or_new (195-197)
lib/collection/src/tests/payload.rs (1)
lib/collection/src/shards/replica_set/mod.rs (1)
  • payload_index_schema (1183-1185)
⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (12)
  • GitHub Check: storage-compat-test
  • GitHub Check: rust-tests (windows-latest)
  • GitHub Check: rust-tests (macos-latest)
  • GitHub Check: rust-tests (ubuntu-latest)
  • GitHub Check: e2e-tests
  • GitHub Check: rust-tests-no-rocksdb (ubuntu-latest)
  • GitHub Check: test-shard-snapshot-api-s3-minio
  • GitHub Check: lint
  • GitHub Check: test-consensus-compose
  • GitHub Check: test-consistency
  • GitHub Check: integration-tests
  • GitHub Check: integration-tests-consensus
🔇 Additional comments (2)
lib/collection/src/tests/payload.rs (1)

116-117: Tests updated to array-aware paths look good

Expectations and index creation now use location[].lat; repo search found no remaining code uses of "location.lat" — only a comment at lib/collection/src/tests/payload.rs:108. Updated uses are at lib/collection/src/tests/payload.rs:116 and 123.

lib/collection/src/problems/unindexed_field.rs (1)

366-369: Correct: use array-aware key when descending into nested filters

Switching to nested.raw_key().array_key() ensures paths like foo[].bar are inferred for indexing under strict mode and aligns unindexed detection with API/index semantics.

Location: lib/collection/src/problems/unindexed_field.rs (lines 366–369)

Run this locally to list any remaining .raw_key() uses not followed by .array_key():

#!/usr/bin/env bash
set -euo pipefail
rg --type=rust -n '\.raw_key\(' || exit 0
rg --type=rust -n '\.raw_key\(' | while IFS=: read -r file line _; do
  ctx=$(sed -n "${line},$((line+8))p" "$file" | tr '\n' ' ')
  if ! printf '%s' "$ctx" | grep -q '\.array_key\s*('; then
    printf '%s:%s\n' "$file" "$line"
    sed -n "${line},$((line+8))p" "$file"
    echo "----"
  fi
done

@generall generall merged commit bff9cb1 into dev Sep 16, 2025
16 checks passed
@generall generall deleted the fix-strict-mode-for-nested branch September 16, 2025 10:03
timvisee pushed a commit that referenced this pull request Sep 29, 2025
* fix strict mode validation for nested filters

* fix test
@timvisee timvisee mentioned this pull request Sep 29, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants