Skip to content

Conversation

@jeremyteyssedre
Copy link
Contributor

@jeremyteyssedre jeremyteyssedre commented Sep 2, 2025

Fixes #11640

Description of change

This PR fixes a bug in lazy count optimization of geManyAndCount. (Introduced in #11524)

Currently, when using limit/offset, the count query can be skipped if the total can be deduced from the number of returned rows. However, there was an incorrect behavior when the offset value was greater than the total number of entities.

Before: lazyCount returned an incorrect total (equal to offset + 0).
Now: lazyCount correctly returns undefined in this case, forcing a proper COUNT(*) query to be executed.

Pull-Request Checklist

  • Code is up-to-date with the master branch
  • This pull request links relevant issues as Fixes #00000
  • There are new or updated unit tests validating the change
  • Documentation has been updated to reflect this change

Summary by CodeRabbit

  • Bug Fixes
    • Ensures total counts are correctly reported for paginated queries when offset/skip exceeds available items, avoiding missing or undefined totals for empty result sets.
  • Tests
    • Added tests to confirm a separate count query runs in offset/skip scenarios, validating accurate totals even when no records are returned.

@coderabbitai
Copy link
Contributor

coderabbitai bot commented Sep 2, 2025

Walkthrough

Adds a guard in SelectQueryBuilder.lazyCount to return undefined when no entities are returned and skip/offset is active, forcing getManyAndCount to execute a separate COUNT query. Adds tests verifying count query is run and no entities are returned when offset or skip exceed total rows.

Changes

Cohort / File(s) Summary
Query builder logic
src/query-builder/SelectQueryBuilder.ts
Added a conditional in lazyCount that returns undefined when result set is empty and skip/offset are present, causing getManyAndCount to issue an explicit COUNT query. No public signatures changed.
Tests (lazy-count)
test/other-issues/lazy-count/lazy-count.ts
Added two tests: one for join+skip beyond total entities and one for offset beyond total entities. Each asserts count equals total rows, returned entities are empty, and a COUNT query was recorded via AfterQuerySubscriber.

Sequence Diagram(s)

sequenceDiagram
  autonumber
  participant C as Caller
  participant SQB as SelectQueryBuilder
  participant DB as Database

  C->>SQB: getManyAndCount()
  activate SQB
  SQB->>DB: Execute main SELECT (with LIMIT/OFFSET or SKIP)
  DB-->>SQB: rows (possibly empty)
  alt rows.length == 0 and (hasSkip or hasOffset)
    note right of SQB #DDFFDD: lazyCount returns undefined\nto force count query
    SQB->>DB: Execute COUNT(*) query
    DB-->>SQB: total count
    SQB-->>C: entities = [], count = total
  else
    SQB->>SQB: lazyCount follows existing logic (infer or reuse count)
    SQB-->>C: entities = rows, count = inferred/queried
  end
  deactivate SQB
Loading

Estimated code review effort

🎯 3 (Moderate) | ⏱️ ~20 minutes

Possibly related PRs

Suggested reviewers

  • sgarner
  • gioboa
  • alumni

Poem

A rabbit inspects an empty row,
Hops past offsets where the carrots go.
"No peas here — I'll count them true,"
Thump! A COUNT query hops into view.
Five carrots found, none in my basket—hooray! 🥕🐇


📜 Recent review details

Configuration used: CodeRabbit UI

Review profile: CHILL

Plan: Pro

💡 Knowledge Base configuration:

  • MCP integration is disabled by default for public repositories
  • Jira integration is disabled
  • Linear integration is disabled

You can enable these sources in your CodeRabbit configuration.

📥 Commits

Reviewing files that changed from the base of the PR and between 6b653c5 and f58e56f.

📒 Files selected for processing (2)
  • src/query-builder/SelectQueryBuilder.ts (1 hunks)
  • test/other-issues/lazy-count/lazy-count.ts (1 hunks)
🚧 Files skipped from review as they are similar to previous changes (2)
  • src/query-builder/SelectQueryBuilder.ts
  • test/other-issues/lazy-count/lazy-count.ts
✨ Finishing Touches
  • 📝 Generate Docstrings
🧪 Generate unit tests
  • Create PR with unit tests
  • Post copyable unit tests in a comment

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share
🪧 Tips

Chat

There are 3 ways to chat with CodeRabbit:

  • Review comments: Directly reply to a review comment made by CodeRabbit. Example:
    • I pushed a fix in commit <commit_id>, please review it.
    • Open a follow-up GitHub issue for this discussion.
  • Files and specific lines of code (under the "Files changed" tab): Tag @coderabbitai in a new review comment at the desired location with your query.
  • PR comments: Tag @coderabbitai in a new PR comment to ask questions about the PR branch. For the best results, please provide a very specific query, as very limited context is provided in this mode. Examples:
    • @coderabbitai gather interesting stats about this repository and render them as a table. Additionally, render a pie chart showing the language distribution in the codebase.
    • @coderabbitai read the files in the src/scheduler package and generate a class diagram using mermaid and a README in the markdown format.

Support

Need help? Create a ticket on our support page for assistance with any issues or questions.

CodeRabbit Commands (Invoked using PR/Issue comments)

Type @coderabbitai help to get the list of available commands.

Other keywords and placeholders

  • Add @coderabbitai ignore or @coderabbit ignore anywhere in the PR description to prevent this PR from being reviewed.
  • Add @coderabbitai summary to generate the high-level summary at a specific location in the PR description.
  • Add @coderabbitai anywhere in the PR title to generate the title automatically.

CodeRabbit Configuration File (.coderabbit.yaml)

  • You can programmatically configure CodeRabbit by adding a .coderabbit.yaml file to the root of your repository.
  • Please see the configuration documentation for more information.
  • If your editor has YAML language server enabled, you can add the path at the top of this file to enable auto-completion and validation: # yaml-language-server: $schema=https://coderabbit.ai/integrations/schema.v2.json

Status, Documentation and Community

  • Visit our Status Page to check the current availability of CodeRabbit.
  • Visit our Documentation for detailed information on how to use CodeRabbit.
  • Join our Discord Community to get help, request features, and share feedback.
  • Follow us on X/Twitter for updates and announcements.

Copy link
Contributor

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 0

🧹 Nitpick comments (2)
src/query-builder/SelectQueryBuilder.ts (1)

1942-1944: Only trigger COUNT when pagination actually shifts the window; return 0 otherwise (and drop the stray semicolon).

Unconditionally forcing a COUNT on zero rows regresses the lazy-count optimization when there’s no skip/offset. If there’s no pagination offset, zero entities implies total=0 and we can skip the extra COUNT. Also, the semicolon is inconsistent with surrounding style.

Apply:

-        if (entitiesAndRaw.entities.length === 0) {
-            return undefined;
-        }
+        if (entitiesAndRaw.entities.length === 0) {
+            const hasSkip =
+                this.expressionMap.skip !== undefined &&
+                this.expressionMap.skip !== null &&
+                this.expressionMap.skip > 0
+            const hasOffset =
+                this.expressionMap.offset !== undefined &&
+                this.expressionMap.offset !== null &&
+                this.expressionMap.offset > 0
+            // If pagination shifted the window, we can't deduce total => force COUNT.
+            // Otherwise, we know total is 0 without an extra query.
+            return hasSkip || hasOffset ? undefined : 0
+        }
test/other-issues/lazy-count/lazy-count.ts (1)

269-295: Good coverage for offset > total; tighten COUNT detection regex and consider a skip/take variant.

  • Minor: use word boundaries to avoid false positives in query text matching.
  • Optional: add a companion test for take/skip where skip > total to cover both pagination APIs.

Regex tweak in this block:

-                expect(
-                    afterQuery
-                        .getCalledQueries()
-                        .filter((query) => query.match(/(count|cnt)/i)),
-                ).not.to.be.empty
+                expect(
+                    afterQuery
+                        .getCalledQueries()
+                        .filter((query) => /\b(count|cnt)\b/i.test(query)),
+                ).not.to.be.empty

Optional additional test (outside this hunk) mirroring the scenario with take/skip:

it("run count query when skip is greater than total entities", () =>
  Promise.all(
    connections.map(async (connection) => {
      await savePostEntities(connection, 5)

      const afterQuery = connection.subscribers[0] as AfterQuerySubscriber
      afterQuery.clear()

      const [entities, count] = await connection.manager
        .createQueryBuilder(Post, "post")
        .take(10)
        .skip(20)
        .orderBy("post.id")
        .getManyAndCount()

      expect(count).to.be.equal(5)
      expect(entities.length).to.be.equal(0)
      expect(
        afterQuery.getCalledQueries().filter((q) => /\b(count|cnt)\b/i.test(q)),
      ).not.to.be.empty
    }),
  ))
📜 Review details

Configuration used: CodeRabbit UI

Review profile: CHILL

Plan: Pro

💡 Knowledge Base configuration:

  • MCP integration is disabled by default for public repositories
  • Jira integration is disabled
  • Linear integration is disabled

You can enable these sources in your CodeRabbit configuration.

📥 Commits

Reviewing files that changed from the base of the PR and between 6965fa2 and 6b653c5.

📒 Files selected for processing (2)
  • src/query-builder/SelectQueryBuilder.ts (1 hunks)
  • test/other-issues/lazy-count/lazy-count.ts (1 hunks)
🧰 Additional context used
🧬 Code graph analysis (1)
test/other-issues/lazy-count/lazy-count.ts (1)
test/other-issues/lazy-count/subscribers/AfterQuerySubscriber.ts (1)
  • afterQuery (11-13)

Copy link
Collaborator

@sgarner sgarner left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good catch @jeremyteyssedre and thanks for the PR 💜

@pkg-pr-new
Copy link

pkg-pr-new bot commented Sep 2, 2025

typeorm-sql-js-example

npm i https://pkg.pr.new/typeorm/typeorm@11634

commit: f58e56f

@coveralls
Copy link

coveralls commented Sep 2, 2025

Coverage Status

coverage: 76.352% (+0.004%) from 76.348%
when pulling f58e56f on jeremyteyssedre:fix-lazy-count-offset-over-limit
into 6965fa2 on typeorm:master.

Copy link
Collaborator

@sgarner sgarner left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM!

Thanks for your contribution @jeremyteyssedre 💜

@sgarner sgarner requested review from alumni and gioboa September 3, 2025 22:58
Copy link
Collaborator

@gioboa gioboa left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @jeremyteyssedre
Well done ✅

@sgarner sgarner merged commit 1f90467 into typeorm:master Sep 5, 2025
58 checks passed
@sgarner sgarner changed the title fix(query-builder): handle empty result set when offset exceeds total fix(query-builder): don't use lazy count when offset exceeds total in getManyAndCount Sep 11, 2025
ThbltLmr pushed a commit to ThbltLmr/typeorm that referenced this pull request Dec 2, 2025
…typeorm#11634)

* fix(query-builder): handle empty result set when offset exceeds total

* fix: lazy count can still be trusted when the result set is empty
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

getManyAndCount() returns incorrect count value for out-of-bounds skip

5 participants