Skip to content

Add select Option to query-db-collection #345

@KyleAMathews

Description

@KyleAMathews

Background

Many APIs return data wrapped in metadata (pagination info, timestamps, etc.). Currently, query-db-collection expects queryFn to return just the array of items. This forces developers to transform data in the queryFn, losing access to metadata.

Problem

// API returns: { data: Todo[], total: number, page: number }
// But collection needs: Todo[]

const todoCollection = createCollection(
  queryCollectionOptions({
    queryKey: ['todos'],
    queryFn: async () => {
      const response = await api.getTodos()
      // Metadata is lost here
      return response.data
    },
    getKey: (item) => item.id,
    queryClient,
  })
)

Proposed Solution

Add a select option that extracts the item array from the query response:

export interface QueryCollectionConfig<
  TItem extends object,
  TError = unknown,
  TQueryKey extends QueryKey = QueryKey,
  TQueryData = unknown, // New generic for raw query data
> {
  // ... existing options ...
  
  /**
   * Transform the query response to extract the items array
   * @param data - The raw response from queryFn
   * @returns The array of items for the collection
   */
  select?: (data: TQueryData) => Array<TItem>
}

Implementation Examples

Example 1: Paginated API Response

interface PaginatedResponse<T> {
  data: T[]
  total: number
  page: number
  pageSize: number
  hasMore: boolean
}

const todoCollection = createCollection(
  queryCollectionOptions({
    queryKey: ['todos', { page: 1 }],
    queryFn: async () => {
      // Returns PaginatedResponse<Todo>
      return api.getTodos({ page: 1, pageSize: 100 })
    },
    select: (response) => response.data, // Extract just the items
    getKey: (item) => item.id,
    queryClient,
  })
)

// The full response is still cached by TanStack Query
// Can access it via queryClient if needed:
const fullResponse = queryClient.getQueryData<PaginatedResponse<Todo>>(['todos', { page: 1 }])
console.log(`Total items: ${fullResponse?.total}`)

Example 2: GraphQL Response

interface GraphQLResponse<T> {
  data: T
  errors?: Array<{ message: string }>
}

interface TodosQuery {
  todos: {
    edges: Array<{ node: Todo }>
    pageInfo: { hasNextPage: boolean }
  }
}

const todoCollection = createCollection(
  queryCollectionOptions({
    queryKey: ['todos-graphql'],
    queryFn: async () => {
      return graphqlClient.request<GraphQLResponse<TodosQuery>>(TODOS_QUERY)
    },
    select: (response) => {
      // Extract and flatten the nodes
      return response.data.todos.edges.map(edge => edge.node)
    },
    getKey: (item) => item.id,
    queryClient,
  })
)

Example 3: API with Metadata

interface ApiResponse<T> {
  result: T
  metadata: {
    timestamp: string
    version: string
    requestId: string
  }
}

const configCollection = createCollection(
  queryCollectionOptions({
    queryKey: ['config'],
    queryFn: async (): Promise<ApiResponse<ConfigItem[]>> => {
      return api.getConfig()
    },
    select: (response) => response.result,
    getKey: (item) => item.key,
    queryClient,
    
    // Can still access metadata in onUpdate
    onUpdate: async ({ transaction }) => {
      const response = await api.updateConfig(changes)
      
      // Log metadata for debugging
      console.log('Update requestId:', response.metadata.requestId)
      
      return response
    }
  })
)

Technical Implementation

In the query observer callback:

const actualUnsubscribeFn = localObserver.subscribe((result) => {
  if (result.isSuccess) {
    // If select is provided, use it to extract items
    const rawData = result.data
    const newItemsArray = config.select 
      ? config.select(rawData)
      : rawData
    
    // Continue with existing sync logic...
    if (\!Array.isArray(newItemsArray)) {
      console.error('[QueryCollection] select function must return an array')
      return
    }
    
    // Rest of sync logic...
  }
})

Benefits

  1. Preserves metadata: Full response stays in TanStack Query cache
  2. Type safety: Can type both the full response and extracted items
  3. Flexibility: Works with any API response shape
  4. Familiar pattern: Matches TanStack Query's select option
  5. Debugging: Can still access full response via QueryClient

Testing Requirements

  1. Test that select properly extracts array from wrapped response
  2. Test error handling when select returns non-array
  3. Test that full response is cached in QueryClient
  4. Test type inference with select option
  5. Test that mutations still work correctly

Related Issue

This addresses #339

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions