Skip to content

repeatable pgsql segfault in session reset timeout #5039

@ncleatonhrt

Description

@ncleatonhrt

When using a postgres backend, proxysql segfaults if there's a timeout running the DISCARD ALL command to make the backend connection ready for reuse.

This repo inserts a transparent tcp proxy between proxysql and the postgresql backend, to add a delay to the DISCARD ALL command and trigger the segfault.

  1. have a local postgres server with a user test and password testtest
  2. start the proxy (attached) in the background:
./psql-proxy-delay-discard.py &
  1. launch proxysql, with PGSSLMODE=disable in the environment to ensure no ssl to the backend, so that the proxy can see and react to the DISCARD ALL:
PGSSLMODE=disable /home/admin/proxysql -c /home/admin/proxysql.cnf -f --idle-threads -D /tmp/proxysql &
  1. Connect through proxysql, issue a SET command so that a reset will be needed, disconnect:
psql -h 127.0.0.1 -U test -p 6133 -d postgres -c 'set search_path to public'
Password for user test:
2025-07-10 10:01:36 PgSQL_Session.cpp:5774:unable_to_parse_set_statement(): [WARNING] Unable to parse unknown SET query from client 127.0.0.1:38080. Setting lock_hostgroup. Please report a bug for future enhancements:set search_path to public
> '\x00\x00\x00Å\x00\x03\x00\x00user\x00test\x00database\x00postgres\x00options\x00-c datestyle=ISO,\\ MDY -c intervalstyle=postgres -c standard_conforming_strings=on -c timezone=GMT \x00application_name\x00proxysql\x00client_encoding\x00SQL_ASCII\x00\x00'
< 'R\x00\x00\x00\x17\x00\x00\x00\nSCRAM-SHA-256\x00\x00'
> 'p\x00\x00\x006SCRAM-SHA-256\x00\x00\x00\x00 n,,n=,r=TwnJBOUKdlXFZGk/ZOAc7/R5'
< 'R\x00\x00\x00\\\x00\x00\x00\x0br=TwnJBOUKdlXFZGk/ZOAc7/R5BokVXq0xssJ/2ntpyWfFnCZu,s=lR+4HjuMXUe8rsCLeurCRQ==,i=4096'
> 'p\x00\x00\x00lc=biws,r=TwnJBOUKdlXFZGk/ZOAc7/R5BokVXq0xssJ/2ntpyWfFnCZu,p=ESBQijzVAC7XrNsM7ShghrOVzRmhkaXxmtc/YONCUp8='
< 'R\x00\x00\x006\x00\x00\x00\x0cv=hPBeEcHrACHvKUxFctYm1Zv5vJxNaJDpquXtG6fjKa0=R\x00\x00\x00\x08\x00\x00\x00\x00S\x00\x00\x00\x17in_hot_standby\x00off\x00S\x00\x00\x00\x19integer_datetimes\x00on\x00S\x00\x00\x00\x11TimeZone\x00GMT\x00S\x00\x00\x00\x1bIntervalStyle\x00postgres\x00S\x00\x00\x00\x15is_superuser\x00off\x00S\x00\x00\x00\x1eapplication_name\x00proxysql\x00S\x00\x00\x00&default_transaction_read_only\x00off\x00S\x00\x00\x00\x1ascram_iterations\x004096\x00S\x00\x00\x00\x17DateStyle\x00ISO, MDY\x00S\x00\x00\x00#standard_conforming_strings\x00on\x00S\x00\x00\x00\x1fsession_authorization\x00test\x00S\x00\x00\x00\x1eclient_encoding\x00SQL_ASCII\x00S\x00\x00\x002server_version\x0017.5 (Debian 17.5-1.pgdg120+1)\x00S\x00\x00\x00\x19server_encoding\x00UTF8\x00K\x00\x00\x00\x0c\x00\x00\x10 x\x14É\x13Z\x00\x00\x00\x05I'
> 'Q\x00\x00\x00\x1eset search_path to public\x00'
< 'C\x00\x00\x00\x08SET\x00Z\x00\x00\x00\x05I'
SET
> 'Q\x00\x00\x00\x10DISCARD ALL\x00'
delaying DISCARD command
admin@ip-172-31-35-238:~$ 2025-07-10 10:01:37 PgSQL_Session.cpp:1315:handler_again___status_RESETTING_CONNECTION(): [ERROR] Resetting Connection timeout during Reset Session on 127.0.0.1 , 7001
Error: signal 11:
/home/admin/proxysql(_Z13crash_handleri+0x2a)[0x55be45b9ef0a]
/lib/x86_64-linux-gnu/libc.so.6(+0x3c050)[0x7f14e645b050]
/lib/x86_64-linux-gnu/libc.so.6(+0x1679d8)[0x7f14e65869d8]
/home/admin/proxysql(_ZN13PgSQL_Session10RequestEndEP17PgSQL_Data_StreamjPKc+0x193)[0x55be45dcf2e3]
/home/admin/proxysql(_ZN13PgSQL_Session43handler_again___status_RESETTING_CONNECTIONEv+0x198)[0x55be45dcd688]
/home/admin/proxysql(_ZN13PgSQL_Session7handlerEv+0x268)[0x55be45dddc88]
/home/admin/proxysql(_ZN12PgSQL_Thread20process_all_sessionsEv+0x3b6)[0x55be45dc67c6]
/home/admin/proxysql(_ZN12PgSQL_Thread3runEv+0x54c)[0x55be45dc6fac]
/home/admin/proxysql(_Z24pgsql_worker_thread_funcPv+0xec)[0x55be45b6fc23]
/lib/x86_64-linux-gnu/libc.so.6(+0x891f5)[0x7f14e64a81f5]
/lib/x86_64-linux-gnu/libc.so.6(+0x10989c)[0x7f14e652889c]
 ---- /home/admin/proxysql(_Z13crash_handleri+0x2a) [0x55be45b9ef0a] : crash_handler(int)
 ---- /home/admin/proxysql(_ZN13PgSQL_Session10RequestEndEP17PgSQL_Data_StreamjPKc+0x193) [0x55be45dcf2e3] : PgSQL_Session::RequestEnd(PgSQL_Data_Stream*, unsigned int, char const*)
 ---- /home/admin/proxysql(_ZN13PgSQL_Session43handler_again___status_RESETTING_CONNECTIONEv+0x198) [0x55be45dcd688] : PgSQL_Session::handler_again___status_RESETTING_CONNECTION()
 ---- /home/admin/proxysql(_ZN13PgSQL_Session7handlerEv+0x268) [0x55be45dddc88] : PgSQL_Session::handler()
 ---- /home/admin/proxysql(_ZN12PgSQL_Thread20process_all_sessionsEv+0x3b6) [0x55be45dc67c6] : PgSQL_Thread::process_all_sessions()
 ---- /home/admin/proxysql(_ZN12PgSQL_Thread3runEv+0x54c) [0x55be45dc6fac] : PgSQL_Thread::run()
 ---- /home/admin/proxysql(_Z24pgsql_worker_thread_funcPv+0xec) [0x55be45b6fc23] : pgsql_worker_thread_func(void*)
To report a crashing bug visit: https://github.com/sysown/proxysql/issues
For support visit: https://proxysql.com/services/support/
> 'X\x00\x00\x00\x04'
< 'C\x00\x00\x00\x10DISCARD ALL\x00Z\x00\x00\x00\x05I'

[2]+  Segmentation fault      (core dumped) PGSSLMODE=disable /home/admin/proxysql -c /home/admin/proxysql.cnf -f --idle-threads -D /tmp/proxysql

The proxysql binary was copied from the proxysql/proxysql 3.0.1-debian image on docker.io.

/home/admin/proxysql --version
2025-07-10 10:38:24 [INFO] Using OpenSSL version: OpenSSL 3.0.15 3 Sep 2024
2025-07-10 10:38:24 main.cpp:348:check_openssl_version(): [WARNING] Detected OpenSSL version has known performance regressions, please upgrade to '3.2.0' or later. For further details, please refer to: https://github.com/openssl/openssl/issues/18814
ProxySQL version 3.0.1-420-g2c26a42, codename Truls

proxysql-segfault.log

proxysql.core.gz

proxysql.cnf.gz

psql-proxy-delay-discard.py.gz

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions