Refactoring of the types for collection configurations #530

kevin-dp · 2025-09-11T10:54:13Z

This PR addresses #396 and supersedes #401. It refactors the types for collection configs.

Problem: `ResolveType` helper doesn't work with inferred types

The main problem was that we had different type parameters TExplicit, TSchema, and TFallback and we would resolve the type from them based on the ResolveType helper. While ResolveType works correctly when you provide it the types, it doesn't play nicely when it comes to type inference. Essentially, the compiler sometimes had to infer the types for TExplicit, TSchema, and TFallback based on ResolveType<TExplicit, TSchema, TFallback> and that doesn't work:

type ResolveType<
  TExplicit,
  TSchema extends StandardSchemaV1 = never,
  TFallback extends object = Record<string, unknown>,
> = unknown extends TExplicit
  ? [TSchema] extends [never]
    ? TFallback
    : InferSchemaOutput<TSchema>
  : TExplicit extends object
    ? TExplicit
    : Record<string, unknown>

function foo<TExplicit, TSchema extends StandardSchemaV1, TFallback extends object>(
  a: ResolveType<TExplicit, TSchema, TFallback>
): TExplicit {
  return null as unknown as TExplicit // don't care about runtime here
}

const schema = z.object({
  id: z.string(),
  schema: z.boolean(),
})

// `res` is typed as typeof schema
// which means that TExplicit got inferred as typeof schema
// which messes up ResolveType logic because it means that ResolveType will return typeof schema
// instead of InferSchemaOutput<TSchema>
const res = foo(schema)

This is one of the problems, there were many other type problems related to this logic. For instance, a user could provide an explicit type to a collection config and then pass in a getKey function whose type doesn't correspond to the explicitly passed type, and it would type check without errors while you would expect it to complain because the argument of the getKey function has a different type than TExplicit.

Solution

Remove `ResolveType` and infer the types from the schema

When creating a collection configuration the user can either pass in a schema or not pass in a schema. If a schema is provided the types need to be inferred from the schema. If no schema is provided, they need to be inferred from getKey. Note: if a schema is provided then the schema types and the getKey types must match.

To track whether or not a schema is provided we need to overload the factories that create collection options. This is the new implementation of createCollection:

// Overload for when schema is provided
export function createCollection<
  T extends StandardSchemaV1,
  TKey extends string | number = string | number,
  TUtils extends UtilsRecord = {},
>(
  options: CollectionConfig<InferSchemaOutput<T>, TKey, T> & {
    schema: T
    utils?: TUtils
  }
): Collection<InferSchemaOutput<T>, TKey, TUtils, T, InferSchemaInput<T>>

// Overload for when no schema is provided
// the type T needs to be passed explicitly unless it can be inferred from the getKey function in the config
export function createCollection<
  T extends object,
  TKey extends string | number = string | number,
  TUtils extends UtilsRecord = {},
>(
  options: CollectionConfig<T, TKey, never> & {
    schema?: never // prohibit schema if an explicit type is provided
    utils?: TUtils
  }
): Collection<T, TKey, TUtils, never, T>

// Implementation
export function createCollection(
  options: CollectionConfig<any, string | number, any> & {
    schema?: StandardSchemaV1
    utils?: UtilsRecord
  }
): Collection<any, string | number, UtilsRecord, any, any> {
  // actual implementation
}

Notice that in the first overload, Typescript can infer T from the schema that the user provides. We then derive the input and output types from that schema, i.e. from T. In the 2nd overload, we explicitly require the schema to be absent (i.e. schema?: never) and thus the input and output types are the same as they are just T. Note that T corresponds to the type of values that are handled by the getKey function from the config. So Typescript can infer it from that function, or the user can pass it explicitly (in which case it must typecheck with the actual types of the getKey function).

So far so good, we managed to remove the ResolveType logic.

Track schemas in the output type of the create collection config factories

We managed to remove the ResolveType logic but the createCollection function now needs to know whether a schema was provided or not. However, our existing collection config factories don't provide that information. Consider for instance the localOnlyCollectionOptions factory implementation:

export function localOnlyCollectionOptions<
  TExplicit = unknown,
  TSchema extends StandardSchemaV1 = never,
  TFallback extends Record<string, unknown> = Record<string, unknown>,
  TKey extends string | number = string | number,
>(
  config: LocalOnlyCollectionConfig<TExplicit, TSchema, TFallback, TKey>
): CollectionConfig<ResolveType<TExplicit, TSchema, TFallback>, TKey> & {
  utils: LocalOnlyCollectionUtils
} {
  // actual implementation
}

The return type of this factory function is:

CollectionConfig<ResolveType<TExplicit, TSchema, TFallback>, TKey> & {
  utils: LocalOnlyCollectionUtils
}

Since CollectionConfig defined a property schema?: TSchema that means that we can't actually know whether or not the schema was provided (since it's always typed as an optional). But we need to know whether it is provided or not in order to pick the right overload.

Hence, we need to also overload the config factories such that their return type is explicit about whether or not the schema was provided:

// Overload for when schema is provided
export function localOnlyCollectionOptions<
  T extends StandardSchemaV1,
  TKey extends string | number = string | number,
>(
  config: LocalOnlyCollectionConfig<InferSchemaOutput<T>, T, TKey> & {
    schema: T
  }
): CollectionConfig<InferSchemaOutput<T>, TKey, T> & {
  utils: LocalOnlyCollectionUtils
  schema: T
}

// Overload for when no schema is provided
// the type T needs to be passed explicitly unless it can be inferred from the getKey function in the config
export function localOnlyCollectionOptions<
  T extends object,
  TKey extends string | number = string | number,
>(
  config: LocalOnlyCollectionConfig<T, never, TKey> & {
    schema?: never // prohibit schema
  }
): CollectionConfig<T, TKey, never> & {
  utils: LocalOnlyCollectionUtils
  schema?: never // no schema in the result
}

export function localOnlyCollectionOptions(
  config: LocalOnlyCollectionConfig<any, any, string | number>
): CollectionConfig<any, string | number, any> & {
  utils: LocalOnlyCollectionUtils
  schema?: StandardSchemaV1
} {
  // actual implementation
}

Finally, now we are tracking whether or not the schema was provided throughout the factories and all types can be inferred correctly.

changeset-bot · 2025-09-11T10:54:16Z

🦋 Changeset detected

Latest commit: 2ade434

The changes in this PR will be included in the next version bump.

This PR includes changesets to release 12 packages

Name	Type
@tanstack/electric-db-collection	Patch
@tanstack/query-db-collection	Patch
@tanstack/db	Patch
@tanstack/db-example-react-todo	Patch
@tanstack/angular-db	Patch
@tanstack/react-db	Patch
@tanstack/rxdb-db-collection	Patch
@tanstack/solid-db	Patch
@tanstack/svelte-db	Patch
@tanstack/trailbase-db-collection	Patch
@tanstack/vue-db	Patch
todos	Patch

Not sure what this means? Click here to learn what changesets are.

Click here if you're a maintainer who wants to add another changeset to this PR

pkg-pr-new · 2025-09-11T10:55:36Z

More templates

@tanstack/angular-db

npm i https://pkg.pr.new/@tanstack/angular-db@530

@tanstack/db

npm i https://pkg.pr.new/@tanstack/db@530

@tanstack/db-ivm

npm i https://pkg.pr.new/@tanstack/db-ivm@530

@tanstack/electric-db-collection

npm i https://pkg.pr.new/@tanstack/electric-db-collection@530

@tanstack/query-db-collection

npm i https://pkg.pr.new/@tanstack/query-db-collection@530

@tanstack/react-db

npm i https://pkg.pr.new/@tanstack/react-db@530

@tanstack/rxdb-db-collection

npm i https://pkg.pr.new/@tanstack/rxdb-db-collection@530

@tanstack/solid-db

npm i https://pkg.pr.new/@tanstack/solid-db@530

@tanstack/svelte-db

npm i https://pkg.pr.new/@tanstack/svelte-db@530

@tanstack/trailbase-db-collection

npm i https://pkg.pr.new/@tanstack/trailbase-db-collection@530

@tanstack/vue-db

npm i https://pkg.pr.new/@tanstack/vue-db@530

commit: 2ade434

github-actions · 2025-09-11T10:56:49Z

Size Change: +108 B (+0.16%)

Total Size: 66.7 kB

Filename	Size	Change
`./packages/db/dist/esm/collection.js`	10.6 kB	+114 B (+1.08%)
`./packages/db/dist/esm/local-storage.js`	2.02 kB	-6 B (-0.3%)

ℹ️ View Unchanged

Filename	Size
`./packages/db/dist/esm/change-events.js`	1.13 kB
`./packages/db/dist/esm/deferred.js`	230 B
`./packages/db/dist/esm/errors.js`	3.1 kB
`./packages/db/dist/esm/index.js`	1.55 kB
`./packages/db/dist/esm/indexes/auto-index.js`	745 B
`./packages/db/dist/esm/indexes/base-index.js`	605 B
`./packages/db/dist/esm/indexes/btree-index.js`	1.74 kB
`./packages/db/dist/esm/indexes/lazy-index.js`	1.25 kB
`./packages/db/dist/esm/local-only.js`	827 B
`./packages/db/dist/esm/optimistic-action.js`	294 B
`./packages/db/dist/esm/proxy.js`	3.87 kB
`./packages/db/dist/esm/query/builder/functions.js`	615 B
`./packages/db/dist/esm/query/builder/index.js`	3.93 kB
`./packages/db/dist/esm/query/builder/ref-proxy.js`	938 B
`./packages/db/dist/esm/query/compiler/evaluators.js`	1.52 kB
`./packages/db/dist/esm/query/compiler/expressions.js`	631 B
`./packages/db/dist/esm/query/compiler/group-by.js`	2.08 kB
`./packages/db/dist/esm/query/compiler/index.js`	2.27 kB
`./packages/db/dist/esm/query/compiler/joins.js`	2.52 kB
`./packages/db/dist/esm/query/compiler/order-by.js`	1.23 kB
`./packages/db/dist/esm/query/compiler/select.js`	1.28 kB
`./packages/db/dist/esm/query/ir.js`	508 B
`./packages/db/dist/esm/query/live-query-collection.js`	333 B
`./packages/db/dist/esm/query/live/collection-config-builder.js`	2.59 kB
`./packages/db/dist/esm/query/live/collection-subscriber.js`	2.4 kB
`./packages/db/dist/esm/query/optimizer.js`	3.05 kB
`./packages/db/dist/esm/SortedMap.js`	1.24 kB
`./packages/db/dist/esm/transactions.js`	2.29 kB
`./packages/db/dist/esm/utils.js`	943 B
`./packages/db/dist/esm/utils/btree.js`	6.02 kB
`./packages/db/dist/esm/utils/comparison.js`	718 B
`./packages/db/dist/esm/utils/index-optimization.js`	1.62 kB

_{compressed-size-action::db-package-size}

github-actions · 2025-09-11T10:58:02Z

Size Change: 0 B

Total Size: 1.18 kB

ℹ️ View Unchanged

Filename	Size
`./packages/react-db/dist/esm/index.js`	152 B
`./packages/react-db/dist/esm/useLiveQuery.js`	1.02 kB

_{compressed-size-action::react-db-package-size}

kevin-dp · 2025-09-11T11:04:47Z

@ignatz I had a look at the trailbase collection to see if it needs similar changes. But i wasn't sure. The changes in this PR are needed for collections that accept a user-provided schema. At the type level the trailbase collection does accept the schema option since the TrailBaseCollectionConfig interface extends Omit<CollectionConfig<TItem, TKey>, "sync" | "onInsert" | "onUpdate" | "onDelete">. So, since it's not omitting schema it will inherit it from CollectionConfig.

However, looking further at the TrailBaseCollectionConfig options i notice that it requires users to pass a parse function. That seems to be very similar to a user-provided schema. So i'd say that one of them is obsolete and we have 2 options here:

keep it as it is but modify the type to also omit the schema key
replace the parse option and use the schema option instead

ignatz · 2025-09-11T11:26:58Z

@kevin-dp , thanks for improving inference.

The TB collection was very closely modeled after the electric one. It has been a few days. Let me have a quick look, to make sure I'm up-to-date with the latest changes, and come back to you 🙏

ignatz · 2025-09-11T11:43:32Z

replace the parse option and use the schema option instead

IIUC, for this to work the trailbase client would need to emit a zod schema for tanstack/db to use, similar to how drizzle produces one that is used in the example. I think that's a fine approach but I wouldn't want to hold you up.

keep it as it is but modify the type to also omit the schema key

This seems like the straightforward approach. Removing the schema argument from the example seems to just work. Do you foresee any issues with this approach?

Naively, I'd be leaning (1). Thanks 🙏

kevin-dp · 2025-09-11T12:08:27Z

This seems like the straightforward approach. Removing the schema argument from the example seems to just work. Do you foresee any issues with this approach?

Yep that's what i was thinking. Since the user can provide a custom parser it seems that they can do in the parser what the schema would usually do. Although it may be nicer with a schema. Wondering now if from a user perspective it makes sense to provide a parser to parse TRecords into TItems and also provide a schema to further validate those TItems and possibly also transform them. What do you think? @ignatz

kevin-dp · 2025-09-11T12:11:55Z

IIUC, for this to work the trailbase client would need to emit a zod schema for tanstack/db to use, similar to how drizzle produces one that is used in the example. I think that's a fine approach but I wouldn't want to hold you up.

Not really, the schema is usually provided by the user. It's optional though. If the user provides a schema, then tanstack/db uses that schema to validate the input and possible transform it into the output type. Tanstack/db can also infer the type of the items in the collection from the schema. If no schema is provided, then ts/db doesn't do any validation and infers the type of the collection from the getKey method.

ignatz · 2025-09-11T12:40:21Z

IIUC, for this to work the trailbase client would need to emit a zod schema for tanstack/db to use, similar to how drizzle produces one that is used in the example. I think that's a fine approach but I wouldn't want to hold you up.

Not really, the schema is usually provided by the user. It's optional though. If the user provides a schema, then tanstack/db uses that schema to validate the input and possible transform it into the output type. Tanstack/db can also infer the type of the items in the collection from the schema. If no schema is provided, then ts/db doesn't do any validation and infers the type of the collection from the getKey method.

The TrailBase client is also typed. If schema, it would make sense to infer it. Otherwise, there's the risk of skew and parlor usability.

The parseapproach was very adhoc just to make the Todo example work with a shared schema. Based on your reply that may have been misguided. Happy to revisit

kevin-dp · 2025-09-11T13:51:10Z

@ignatz if it makes sense to replace parser by the schema option we already support that would be great!

…ions factories

…alOnlyCollectionOptions

…d of the input type

…ionConfig no longer has this type param

…efault

…ovided

samwillis

LGTM

kevin-dp requested a review from samwillis September 11, 2025 11:10

kevin-dp mentioned this pull request Sep 11, 2025

fix: explicitly type electricCollectionOptions return value #502

Closed

kevin-dp added 17 commits September 15, 2025 09:39

Change update to accept TInput instead of TItem.

cd4c4de

Pick unit tests for Sam's PR

328561c

Correctly handle TExplicit that is never in type resolvers

5302f42

Simplify type resolvers by introducing helper types

11fcc63

Rename ResolveInsertInput type to ResolveInput

6b9ac1c

Pass TKey explicitly in the local-only typing test

28c6917

Explicitly pass input type to CollectionConfig type in collection opt…

9e9bff8

…ions factories

Modify overloads for createCollection and introduce overloads for loc…

89c0a16

…alOnlyCollectionOptions

Fix type tests for local only collections based on new overloads

b356afc

Fix types of collection in mockSyncCollectionOptions

d0c0886

Fix type overloads in local storage collection

7c8dd30

Fix type of onInsert handler which should take the output type instea…

f4eb5b4

…d of the input type

Fix type errors in tests

0dec216

Remove the TInput type that was passed to CollectionConfig as Collect…

b2f206a

…ionConfig no longer has this type param

Remove the ResolveType and ResolveInput types.

bbff4dc

No longer passing never as type param since that corresponds to its d…

0196622

…efault

Add type overloads to electric-db-collection

dc3ab12

kevin-dp added 3 commits September 15, 2025 09:44

Overload query collection config for when a schema or no schema is pr…

909f564

…ovided

Changeset

a0c494e

Update docstrings

bdeaa91

kevin-dp force-pushed the kevin/update-input-schema-type branch from 8ccf7db to bdeaa91 Compare September 15, 2025 07:46

kevin-dp added 3 commits September 15, 2025 10:05

Add overloads to rxdb collection

effe1ce

Remove explicit type parameter since it has a default

5779705

Fix type of utils in query collection options function overloads

2ade434

samwillis approved these changes Sep 15, 2025

View reviewed changes

kevin-dp merged commit b03894d into main Sep 15, 2025
6 checks passed

kevin-dp deleted the kevin/update-input-schema-type branch September 15, 2025 11:04

github-actions bot mentioned this pull request Sep 15, 2025

ci: Version Packages #553

Merged

pawelblaszczyk5 mentioned this pull request Sep 15, 2025

Types mismatch between electricCollectionOptions and createCollection #392

Closed

0xlakshan mentioned this pull request Sep 17, 2025

feat: Add select option to extract items while preserving metadata #551

Merged

5 tasks

KyleAMathews mentioned this pull request Sep 17, 2025

Types are incorrect for schema, where input & output are different #396

Closed

Uziniii pushed a commit to Uziniii/db that referenced this pull request Sep 19, 2025

Refactoring of the types for collection configurations (TanStack#530)

27df19c

samwillis mentioned this pull request Sep 27, 2025

WIP improve schema support and refactor collection options factory function types #401

Closed

Refactoring of the types for collection configurations #530

Refactoring of the types for collection configurations #530

Uh oh!

Conversation

kevin-dp commented Sep 11, 2025

Problem: ResolveType helper doesn't work with inferred types

Solution

Remove ResolveType and infer the types from the schema

Track schemas in the output type of the create collection config factories

Uh oh!

changeset-bot bot commented Sep 11, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

🦋 Changeset detected

Uh oh!

pkg-pr-new bot commented Sep 11, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions bot commented Sep 11, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions bot commented Sep 11, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

kevin-dp commented Sep 11, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

ignatz commented Sep 11, 2025

Uh oh!

ignatz commented Sep 11, 2025

Uh oh!

kevin-dp commented Sep 11, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

kevin-dp commented Sep 11, 2025

Uh oh!

ignatz commented Sep 11, 2025

Uh oh!

kevin-dp commented Sep 11, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

samwillis left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

Problem: `ResolveType` helper doesn't work with inferred types

Remove `ResolveType` and infer the types from the schema

changeset-bot bot commented Sep 11, 2025 •

edited

Loading

pkg-pr-new bot commented Sep 11, 2025 •

edited

Loading

github-actions bot commented Sep 11, 2025 •

edited

Loading

github-actions bot commented Sep 11, 2025 •

edited

Loading

kevin-dp commented Sep 11, 2025 •

edited

Loading

kevin-dp commented Sep 11, 2025 •

edited

Loading

kevin-dp commented Sep 11, 2025 •

edited

Loading