-
Notifications
You must be signed in to change notification settings - Fork 4.5k
Closed
Labels
BuildFeatures planned for next Build conferenceFeatures planned for next Build conferencemsft.ext.vectordataRelated to Microsoft.Extensions.VectorDataRelated to Microsoft.Extensions.VectorData
Description
@eavanvalkenburg internally raised the good question of our SQL Server text mapping (nvarchar(255) vs. nvarchar(max)).
This is indeed a tricky aspect of SQL Server (not of SQL databases in general, SQLite/PG are fine). nvarchar(max) cannot be indexed (and therefore also can't be a key). We could be clever, using nvarchar(255) (or nvarchar(4000) only if IsFilterable=false, and nvarchar(max) for string keys and properties with IsFilterable=true; FWIW this is what EF does, but that means that if the user sets/unsets IsFilterable, their database table will be out of sync etc. (EF actually generates a migration to change the type for this case, but of course it's out of scope for us here).
Options:
- First, we can increase 255 to 4000 (the maximum that isn't max). One side of me wants to do things and just keep things super simple, and document the limitation for now. We do plan to allow a database-specific way to let users configure the type anyway, so at that point they'd be able to configure nvarchar(max) for a specific property.
- But if we think users would want to store large texts (>4000), this would be a problematic limitation. So we could vary the type based on IsFilterable as above (the EF way). If users change filterability, it's up to them to make the change in the type database themselves (like they already do when they change other aspects in the schema).
- Otherwise, we could keep nvarchar(4000) by default, but introduce a store-wide config option to default to nvarchar(max) instead.
Thoughts?
Reactions are currently unavailable
Metadata
Metadata
Assignees
Labels
BuildFeatures planned for next Build conferenceFeatures planned for next Build conferencemsft.ext.vectordataRelated to Microsoft.Extensions.VectorDataRelated to Microsoft.Extensions.VectorData
Type
Projects
Status
Sprint: Planned