-
Notifications
You must be signed in to change notification settings - Fork 190
[RFC] Fix Type Conflict Schema Merging for Cross-Index Fields in Append and Multisearch Commands #4383
Description
Problem Statement
Both append and multisearch commands currently handle schema merging with type conflicts using a field renaming workaround instead of implementing intelligent type resolution. While both commands now work functionally, they create confusing user experiences with duplicated semantic fields.
Current Behavior Analysis
Both append and multisearch commands now work around type conflicts by renaming conflicting fields:
Example with Type Conflict:
// Append syntax:
source=account | fields firstname, age, balance | head 2 |
append [ source=locations | fields description, age, place_id | head 2 ]
// Multisearch syntax:
| multisearch
[search source=account | fields firstname, age, balance | head 2]
[search source=locations | fields description, age, place_id | head 2]
Current Result Schema (Both Commands):
firstname(string) - from accountage(bigint) - from account (original)balance(bigint) - from accountdescription(string) - from locationsage0(string) - from locations (renamed to avoid conflict)place_id(int) - from locations
Problem: This creates confusing column names and duplicated semantic fields (age and age0 both represent the same logical concept).
Test Index Schemas Used
ACCOUNT Index:
firstname(string),age(bigint),balance(bigint),account_number(bigint)
LOCATIONS_TYPE_CONFLICT Index:
description(string),age(string),place_id(integer)
Long-Term Goals
Expected Behavior: Both commands should intelligently merge schemas with type conflicts by:
-
No Type Conflict: Automatically resolve type conflicts
age(bigint) +age(string) →agebalance(int) +balance(double) →balance
-
Consistent Behavior: Both append and multisearch should use identical schema merging logic
Potential solution
Metadata
Metadata
Assignees
Labels
Type
Projects
Status