-
-
Notifications
You must be signed in to change notification settings - Fork 6.5k
fix(query-builder): don't use lazy count when offset exceeds total in getManyAndCount #11634
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
fix(query-builder): don't use lazy count when offset exceeds total in getManyAndCount #11634
Conversation
WalkthroughAdds a guard in SelectQueryBuilder.lazyCount to return undefined when no entities are returned and skip/offset is active, forcing getManyAndCount to execute a separate COUNT query. Adds tests verifying count query is run and no entities are returned when offset or skip exceed total rows. Changes
Sequence Diagram(s)sequenceDiagram
autonumber
participant C as Caller
participant SQB as SelectQueryBuilder
participant DB as Database
C->>SQB: getManyAndCount()
activate SQB
SQB->>DB: Execute main SELECT (with LIMIT/OFFSET or SKIP)
DB-->>SQB: rows (possibly empty)
alt rows.length == 0 and (hasSkip or hasOffset)
note right of SQB #DDFFDD: lazyCount returns undefined\nto force count query
SQB->>DB: Execute COUNT(*) query
DB-->>SQB: total count
SQB-->>C: entities = [], count = total
else
SQB->>SQB: lazyCount follows existing logic (infer or reuse count)
SQB-->>C: entities = rows, count = inferred/queried
end
deactivate SQB
Estimated code review effort🎯 3 (Moderate) | ⏱️ ~20 minutes Possibly related PRs
Suggested reviewers
Poem
📜 Recent review detailsConfiguration used: CodeRabbit UI Review profile: CHILL Plan: Pro 💡 Knowledge Base configuration:
You can enable these sources in your CodeRabbit configuration. 📒 Files selected for processing (2)
🚧 Files skipped from review as they are similar to previous changes (2)
✨ Finishing Touches
🧪 Generate unit tests
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. 🪧 TipsChatThere are 3 ways to chat with CodeRabbit:
SupportNeed help? Create a ticket on our support page for assistance with any issues or questions. CodeRabbit Commands (Invoked using PR/Issue comments)Type Other keywords and placeholders
CodeRabbit Configuration File (
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Actionable comments posted: 0
🧹 Nitpick comments (2)
src/query-builder/SelectQueryBuilder.ts (1)
1942-1944: Only trigger COUNT when pagination actually shifts the window; return 0 otherwise (and drop the stray semicolon).Unconditionally forcing a COUNT on zero rows regresses the lazy-count optimization when there’s no skip/offset. If there’s no pagination offset, zero entities implies total=0 and we can skip the extra COUNT. Also, the semicolon is inconsistent with surrounding style.
Apply:
- if (entitiesAndRaw.entities.length === 0) { - return undefined; - } + if (entitiesAndRaw.entities.length === 0) { + const hasSkip = + this.expressionMap.skip !== undefined && + this.expressionMap.skip !== null && + this.expressionMap.skip > 0 + const hasOffset = + this.expressionMap.offset !== undefined && + this.expressionMap.offset !== null && + this.expressionMap.offset > 0 + // If pagination shifted the window, we can't deduce total => force COUNT. + // Otherwise, we know total is 0 without an extra query. + return hasSkip || hasOffset ? undefined : 0 + }test/other-issues/lazy-count/lazy-count.ts (1)
269-295: Good coverage for offset > total; tighten COUNT detection regex and consider a skip/take variant.
- Minor: use word boundaries to avoid false positives in query text matching.
- Optional: add a companion test for take/skip where skip > total to cover both pagination APIs.
Regex tweak in this block:
- expect( - afterQuery - .getCalledQueries() - .filter((query) => query.match(/(count|cnt)/i)), - ).not.to.be.empty + expect( + afterQuery + .getCalledQueries() + .filter((query) => /\b(count|cnt)\b/i.test(query)), + ).not.to.be.emptyOptional additional test (outside this hunk) mirroring the scenario with
take/skip:it("run count query when skip is greater than total entities", () => Promise.all( connections.map(async (connection) => { await savePostEntities(connection, 5) const afterQuery = connection.subscribers[0] as AfterQuerySubscriber afterQuery.clear() const [entities, count] = await connection.manager .createQueryBuilder(Post, "post") .take(10) .skip(20) .orderBy("post.id") .getManyAndCount() expect(count).to.be.equal(5) expect(entities.length).to.be.equal(0) expect( afterQuery.getCalledQueries().filter((q) => /\b(count|cnt)\b/i.test(q)), ).not.to.be.empty }), ))
📜 Review details
Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro
💡 Knowledge Base configuration:
- MCP integration is disabled by default for public repositories
- Jira integration is disabled
- Linear integration is disabled
You can enable these sources in your CodeRabbit configuration.
📒 Files selected for processing (2)
src/query-builder/SelectQueryBuilder.ts(1 hunks)test/other-issues/lazy-count/lazy-count.ts(1 hunks)
🧰 Additional context used
🧬 Code graph analysis (1)
test/other-issues/lazy-count/lazy-count.ts (1)
test/other-issues/lazy-count/subscribers/AfterQuerySubscriber.ts (1)
afterQuery(11-13)
sgarner
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Good catch @jeremyteyssedre and thanks for the PR 💜
commit: |
sgarner
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM!
Thanks for your contribution @jeremyteyssedre 💜
gioboa
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks @jeremyteyssedre
Well done ✅
…typeorm#11634) * fix(query-builder): handle empty result set when offset exceeds total * fix: lazy count can still be trusted when the result set is empty
Fixes #11640
Description of change
This PR fixes a bug in lazy count optimization of geManyAndCount. (Introduced in #11524)
Currently, when using limit/offset, the count query can be skipped if the total can be deduced from the number of returned rows. However, there was an incorrect behavior when the offset value was greater than the total number of entities.
Before: lazyCount returned an incorrect total (equal to offset + 0).
Now: lazyCount correctly returns undefined in this case, forcing a proper COUNT(*) query to be executed.
Pull-Request Checklist
masterbranchFixes #00000Summary by CodeRabbit