Skip to content

[MEVD] [SQL Server] String mapping to nvarchar(255) vs. nvarchar(max) #10975

@roji

Description

@roji

@eavanvalkenburg internally raised the good question of our SQL Server text mapping (nvarchar(255) vs. nvarchar(max)).

This is indeed a tricky aspect of SQL Server (not of SQL databases in general, SQLite/PG are fine). nvarchar(max) cannot be indexed (and therefore also can't be a key). We could be clever, using nvarchar(255) (or nvarchar(4000) only if IsFilterable=false, and nvarchar(max) for string keys and properties with IsFilterable=true; FWIW this is what EF does, but that means that if the user sets/unsets IsFilterable, their database table will be out of sync etc. (EF actually generates a migration to change the type for this case, but of course it's out of scope for us here).

Options:

  • First, we can increase 255 to 4000 (the maximum that isn't max). One side of me wants to do things and just keep things super simple, and document the limitation for now. We do plan to allow a database-specific way to let users configure the type anyway, so at that point they'd be able to configure nvarchar(max) for a specific property.
  • But if we think users would want to store large texts (>4000), this would be a problematic limitation. So we could vary the type based on IsFilterable as above (the EF way). If users change filterability, it's up to them to make the change in the type database themselves (like they already do when they change other aspects in the schema).
  • Otherwise, we could keep nvarchar(4000) by default, but introduce a store-wide config option to default to nvarchar(max) instead.

Thoughts?

Metadata

Metadata

Assignees

Labels

BuildFeatures planned for next Build conferencemsft.ext.vectordataRelated to Microsoft.Extensions.VectorData

Type

No type

Projects

Status

Sprint: Planned

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions