Reviewing files that changed from the base of the PR and between 31384f1 and 1ecfcf9.

• lib/segment/src/index/field_index/full_text_index/immutable_inverted_index.rs (4 hunks)
• lib/segment/src/index/field_index/full_text_index/inverted_index.rs (4 hunks)
• lib/segment/src/index/field_index/full_text_index/mmap_inverted_index/mod.rs (4 hunks)
• lib/segment/src/index/field_index/full_text_index/mutable_inverted_index.rs (2 hunks)
• lib/segment/src/index/field_index/full_text_index/postings_iterator.rs (2 hunks)
• lib/segment/src/index/field_index/full_text_index/text_index.rs (2 hunks)

• GitHub Check: lint
• GitHub Check: test-snapshot-operations-s3-minio
• GitHub Check: test-low-resources
• GitHub Check: test-shard-snapshot-api-s3-minio
• GitHub Check: storage-compat-test
• GitHub Check: integration-tests-consensus
• GitHub Check: test-consistency
• GitHub Check: integration-tests
• GitHub Check: test-consensus-compose
• GitHub Check: Basic TLS/HTTPS tests
• GitHub Check: rust-tests (windows-latest)
• GitHub Check: rust-tests (ubuntu-latest)
• GitHub Check: rust-tests (macos-latest)

lib/segment/src/index/field_index/full_text_index/text_index.rs (2)

259-261: LGTM! Correctly implements the new ParsedQuery enum structure.

The conversion from collecting tokens into a vector to using TokenSet and wrapping in ParsedQuery::Tokens aligns perfectly with the refactoring objectives. The optional collection pattern handles missing tokens appropriately.

---

620-622: LGTM! Test helper updated consistently.

The test helper function is updated to match the same pattern as the main implementation, maintaining consistency across the codebase.

lib/segment/src/index/field_index/full_text_index/mutable_inverted_index.rs (2)

74-97: LGTM! Well-structured subset filtering logic.

The new filter_has_subset method correctly:

• Handles unseen tokens by returning empty iterator
• Handles empty token sets by returning empty iterator
• Uses intersect_postings_iterator for efficient intersection
• Includes helpful comments explaining the logic

This centralizes the filtering logic and improves code organization.

---

207-209: LGTM! Clean delegation to the new subset filtering method.

The filter method correctly matches on the ParsedQuery::Tokens variant and delegates to the new filter_has_subset method, maintaining the same behavior while improving code structure.

lib/segment/src/index/field_index/full_text_index/postings_iterator.rs (2)

2-2: LGTM! Updated imports for the generic refactoring.

The import updates correctly include the types needed for the generic intersect_compressed_postings_iterator function.

---

24-25: LGTM! Good generalization of the intersection function.

Making intersect_compressed_postings_iterator generic over PostingValue improves reusability across different posting list types while maintaining the same intersection logic. The generic constraint V: PostingValue + 'a is appropriate.

lib/segment/src/index/field_index/full_text_index/immutable_inverted_index.rs (4)

23-57: LGTM! Comprehensive subset filtering implementation.

The filter_has_subset method correctly:

• Filters out deleted documents using point_to_tokens_count
• Handles unseen tokens by returning empty iterator
• Handles empty token sets by returning empty iterator
• Uses intersect_compressed_postings_iterator for efficient intersection
• Maintains consistency with the mutable implementation pattern

The nested intersection function provides good separation of concerns.

---

59-83: LGTM! Efficient subset checking logic.

The check_has_subset method efficiently checks if a specific point contains all tokens by:

• Handling empty token sets appropriately
• Checking document presence first
• Using posting_list.visitor().contains() for efficient lookup
• Leveraging the nested check_intersection function for clarity

This provides a more efficient alternative to filtering when checking a single point.

---

126-128: LGTM! Clean delegation pattern.

The filter method correctly delegates to filter_has_subset for the ParsedQuery::Tokens variant, maintaining consistency with other inverted index implementations.

---

149-151: LGTM! Consistent delegation for match checking.

The check_match method correctly delegates to check_has_subset for the ParsedQuery::Tokens variant, providing efficient single-point subset checking.

lib/segment/src/index/field_index/full_text_index/mmap_inverted_index/mod.rs (5)

20-20: LGTM: Import addition is correct.

The TokenSet import is properly added to support the new abstraction.

---

148-182: LGTM: Well-implemented token subset filtering.

The filter_has_subset method correctly:

• Filters out deleted points using the is_active check
• Handles edge cases (empty queries, missing tokens) by returning empty iterators
• Uses the established intersect_compressed_postings_iterator for efficient intersection
• Properly threads the hardware counter through to posting list access

The implementation is consistent with similar methods in other index implementations.

---

184-218: LGTM: Correct implementation of subset checking.

The check_has_subset method properly:

• Validates non-empty queries and point existence
• Uses efficient contains checks on posting list visitors
• Handles the hardware counter correctly for posting list access
• Follows the same pattern as implementations in other index types

The use of &TokenSet (by reference) is appropriate for this read-only check operation.

---

306-308: LGTM: Correct delegation to the new helper method.

The filter method properly matches on the ParsedQuery enum and delegates to filter_has_subset for the Tokens variant, maintaining the existing interface while using the new abstraction.

---

335-337: LGTM: Consistent enum handling and delegation.

The check_match method correctly matches on the ParsedQuery enum and delegates to check_has_subset, maintaining consistency with the filter method's approach.

lib/segment/src/index/field_index/full_text_index/inverted_index.rs (6)

37-39: LGTM: Efficient and correct subset checking.

The has_subset method efficiently checks subset containment using all() and the existing contains() method (which uses binary search). The implementation is clear and performant.

---

76-82: LGTM: Well-designed enum preparing for future functionality.

The refactoring of ParsedQuery to an enum with the Tokens(TokenSet) variant is well-structured and clearly prepares for future phrase matching capabilities. The documentation clearly explains the current variant's behavior.

---

85-96: LGTM: Clean implementation using the new abstraction.

The updated check_match method correctly:

• Matches on the ParsedQuery enum
• Handles empty query edge cases appropriately
• Leverages the new has_subset method for clean, readable logic

---

158-164: LGTM: Proper delegation pattern.

The estimate_cardinality method correctly matches on the ParsedQuery enum and delegates to the appropriate estimation method, maintaining clean separation of concerns.

---

166-171: LGTM: Consistent API usage and clear naming.

The method is appropriately renamed to estimate_has_subset_cardinality for clarity, and correctly uses tokens.tokens() to access the underlying token vector through the TokenSet API.

Also applies to: 174-176

---

302-303: LGTM: Test helper correctly updated for new enum.

The test helper function is properly updated to construct ParsedQuery::Tokens(tokens) instead of the previous struct format, maintaining test compatibility with the new enum structure.