Sometimes batching support is provided by services directly for example in on jsonplaceholder

GET One

GET https://jsonplaceholder.typicode.com/users/1

GET Many

GET https://jsonplaceholder.typicode.com/users?id=1&id=2&id=3

When enabled this can reduce the load upstream drastically. We could make One Http call instead of many to load data for multiple user-ids.

Approach

• Create a DL (Data Loader) corresponding to each endpoint. This needs to be created once at the time of server start.
• All batchable requests (GET, DELETE etc.) should go via their corresponding Data Loader.
• Endpoint will need to maintain batching information which is how is the URL going to be generated if it's a call for many ids.
• DL should use the information in the endpoint about batching and perform additional operations if the endpoint supports batching.

DSL Changes

type Query {
  posts: [Post] @http(path: "/posts")
  post(id: Int): Post @http(path: "/posts/:id")
}

type Post {
  title: String
  userId: User

  user: User 
    @http(path: "/users", query: [{key: "id", value: "{{value.userId}}"}]) 
    @batch(key: "userId", path: ["id"])
}
type User { ... }

Technical Requirements
Option 1

• We can generate endpoint_id: usize while creating the blueprint, and use endpoint id to perform a lookup in an Array instead of using a hashmap. This would save us the cost of creating a hash for each upstream call.
• Generated DLs should be saved in ServerContext.

Option 2

• We could also consider saving the DL in Expression::Endpoint with the RequestTemplate.
• The fact that we use the DL or not should remain with the Expression::eval for now.

Additional Info:

• Currently we have one data loader shared across all endpoint. This increases thread contention across HTTP requests. With one DL per endpoint, thread contention will also reduce 🤞 resulting in a little better performance when enabled.

Limitations:

• Batching is only supported for GET requests with query params only.