-
Notifications
You must be signed in to change notification settings - Fork 2.1k
Prisma with @prisma/adapter-mssql: Connection Pool Exhaustion and "EREQINPROG" Errors in Production #28812
Description
Bug description
After upgrading from Prisma v6 and switching to the "rust free" prisma and adapter-mssql driver adapter, our production Node.js server experiences periodic hangs where it stops responding to requests.
Root Cause
issue manifests as a connection pool exhaustion + transaction rollback race condition, when the server hang happens we can find stream of logs entries like this:
TransactionError: Can't rollback transaction. There is a request in progress.
at Transaction._rollback (...)
code: 'EREQINPROG'
TimeoutError: operation timed out for an unknown reason
at /usr/src/app/dist/node_modules/tarn/dist/PendingOperation.js:17:27 {
clientVersion: '6.19.0'
}
server does not crash but also does not recover from this without restarting. To us this looks like connection pooler is "stuck".
Severity
🚨 Critical: Data loss, app crash, security issue
Reproduction
We don't have exact steps as this happens random and we only observed it in production on real live traffic. looks like some specific combination of queries result in stuck state (possible multiple longer running queries)
This started happening after upgrade to 6.19 and "rust free configuration". With rust engine we never saw this, now it happens daily. For now we are reverting to rust engine.
Expected vs. Actual Behavior
Expected
- Hanging queries should be cancelled cleanly
- Connections should always be released back to the pool
- The pool should never enter a state where it cannot service new requests
Actual
- Connection pool becomes exhausted
- Server stops responding but doesn't crash
- Only recovery is container restart
Frequency
Intermittent / Random
Does this occur in development or production?
Only in production (e.g., query engine, generated client)
Is this a regression?
yes this problem was not present in all previous version with "rust engine".
Workaround
use prisma with rust engine
Prisma Schema & Queries
// Add relevant schema.prisma snippet// Add relevant Prisma Client queryPrisma Config
import { PrismaClient } from './prisma/generated/client'
import { PrismaMssql } from '@prisma/adapter-mssql'
const sqlConfig = {
user: process.env.SQL_USER!,
password: process.env.SQL_PASSWORD!,
database: process.env.SQL_DATABASE!,
server: process.env.SQL_SERVER!,
requestTimeout: Number(process.env.SQL_REQUEST_TIMEOUT_MS) || 10000,
pool: {
max: Number(process.env.SQL_SERVER_POOL_MAX) || 50,
min: Number(process.env.SQL_SERVER_POOL_MIN) || 5,
idleTimeoutMillis:
Number(process.env.SQL_SERVER_POOL_IDLE_TIMEOUT_MS) || 30000,
},
options: {
encrypt: true
},
}
const adapter = new PrismaMssql(sqlConfig)
export const prisma = new PrismaClient({ adapter })Logs & Debug Info
TransactionError: Can't rollback transaction. There is a request in progress.
at Transaction._rollback (...)
code: 'EREQINPROG'
TimeoutError: operation timed out for an unknown reason
at /usr/src/app/packages/partner-backend/dist/node_modules/tarn/dist/PendingOperation.js:17:27 {
clientVersion: '6.19.0'
Environment & Setup
- Prisma Version: 6.19.0
- Adapter: @prisma/adapter-mssql
- Database: Azure SQL Server
- Runtime: Node.js in Docker container on Azure App Service (Linux
Prisma Version
6.19.0
might be related: #28002