Carretto is a modular utility suite designed to simplify and enhance working with dataloader in TypeScript/Node.js projects. It provides a robust foundation and powerful adapters for efficient batching and caching across REST APIs and MongoDB, allowing for seamless query aggregation and field projection with minimal setup.

You will need Node.js v20+ and pnpm 10+.

# Install the Carretto package you need (example with MongoDB adapter)
pnpm add @carretto/http

The following example demonstrates how to perform efficient batched queries using DataloaderHttp:

import { DataloaderHttp } from "@carretto/http";

const loader = new DataloaderHttp();

// Load a resource using a RESTful endpoint with field projections
const result = await loader.load({
  query: new URL("http://localhost:3000/users"),
  projection: { firstName: 1, lastName: 1 },
});

console.log(result);
// Output (example): { firstName: 'Mario', lastName: 'Rossi' }

This pattern gives you deduplicated, batched API calls with customizable field projections.

All loaders use a key object with the following fields:

• query (object or URL): Defines the resource or query for the backend (DB query, REST endpoint, etc).
• projection (object): Which resource fields to retrieve { fieldName: 1 }.
• skip (optional, number): Items to skip (pagination).
• limit (optional, number): Max number of items to return (pagination).
• sort (optional, object): Sort order (MongoDB only).

Query aggregation: All calls with equivalent query values are batched into a single backend request.

Fetches and batches resources from HTTP endpoints.

import { DataloaderHttp } from "@carretto/http";

const loader = new DataloaderHttp();
await loader.load({
  query: new URL("http://api.example.com/data"),
  projection: { fieldA: 1, fieldB: 1 },
  skip: 5,
  limit: 10,
});

Batches MongoDB queries for fine-grained control over expected fields, pagination, and sorting.

import { DataloaderMongoDB } from "@carretto/mongodb";

const loader = new DataloaderMongoDB(db.collection("Users"));
await loader.load({
  query: { age: { $gt: 30 } },
  projection: { firstName: 1, lastName: 1 },
  sort: { createdAt: -1 },
});

Build your own adapters for custom data sources.

Batch multiple requests with different projections—Carretto merges requested fields and sends a single HTTP call:

const loader = new DataloaderHttp();
const [a, b] = await Promise.all([
  loader.load({
    query: new URL("http://api/my"),
    projection: { firstName: 1 },
  }),
  loader.load({ query: new URL("http://api/my"), projection: { lastName: 1 } }),
]);
// Both requests hit backend only once with merged projection: { firstName: 1, lastName: 1 }

Combining requests for different slices of a dataset results in optimized backend queries:

const loader = new DataloaderMongoDB(collection);
const [usersA, usersB] = await Promise.all([
  loader.loadMany({
    query: { status: "active" },
    projection: { name: 1 },
    skip: 0,
    limit: 10,
  }),
  loader.loadMany({
    query: { status: "active" },
    projection: { email: 1 },
    skip: 5,
    limit: 15,
  }),
]);
// Only one .find() call is performed with skip: 0, limit: 15 and merged projection

Leverage MainLoader to introduce a new database or data source:

import { MainLoader } from "@carretto/main-loader";

class MyCustomLoader extends MainLoader<MyType, MyQueryType> {
  protected async execute(key) {
    // Integrate any backend with batching/projection logic
    return myBackend.query(key.query, key.projection, key.skip, key.limit);
  }
}

We welcome contributions of all kinds:

• Bug reports: Use the issue templates to submit bugs or regressions.
• Features & Improvements: Open a discussion or PR—see CONTRIBUTING.md for details.
• Code style: Use pnpm lint, pnpm format, and follow conventional commits.
• Testing: pnpm test executes the full suite, including coverage reporting.

To get started:

git clone https://github.com/ducktors/carretto.git
cd carretto
pnpm install
pnpm test

• License: MIT
• Author: Alessandro Magionami - @alemagio
• Inspired by: dataloader
• See adapters’ and dependencies' own LICENSE files for details.