What happened?
ChromaDB 0.5.0 suddenly crashed one morning due to a query after running normally for two months.
The last few logs are (The last one is "Starting component PersistentLocalHnswSegment" everytime when it crashes):
[32mINFO[0m: [04-11-2024 12:19:31] myIP:61952 - "GET /api/v1/collections/redmine?tenant=default_tenant&database=default_database HTTP/1.1" 200
[32mINFO[0m: [04-11-2024 12:21:43] myIP:62055 - "GET /api/v1/pre-flight-checks HTTP/1.1" 200
[36mDEBUG[0m: [04-11-2024 12:21:43] Starting component PersistentLocalHnswSegment
I eventually discovered that the reason was that the collection I was querying had more than 99 records, while another collection with only 83 records didn't have this issue. However, this doesn't make sense because this collection had been used normally with over a thousand records for two whole months!
Yes, there are some similar problems reported (such as #2513).
However, this time the situation is somewhat special. Two months ago, I installed and used ChromaDB, and after using it for a week, I encountered a similar issue. At that time, the database I created using chromadb.PersistentClient() would silently crash without any warning when inserting more than 99 records.
So, I followed the solution from the aforementioned issue(#2513) and downgraded ChromaDB to version 0.5.0 and hnswlib to 0.7.3, but it was of no use. Later, I accidentally discovered that when I switched to using chromadb.HttpClient() to start the database, everything returned to normal. Although the issue wasn't completely resolved, I felt that as long as the program could run, it was fine.
Therefore, ChromaDB worked normally for two months, then suddenly crashed during a query last Friday. Moreover, I can no longer insert any data into the database if the target collection already has more than 99 records or if my insertion would exceed 99 records.
People often say that when God closes a door, He opens a window. Yes, I once had that window, which was chromadb.HttpClient(). Now, this last window has been mercilessly closed. It was still working normally on Thursday night—it ran smoothly for two months, and the next day when I returned, it crashed? What the hell is that?! XD
And you know what? The exact same code and the exact same ChromaDB now work perfectly in a Linux virtual machine on my Windows system. Although it can work in a Linux virtual machine's Docker, this still causes unnecessary trouble for me when testing the program on the Windows system. Anyway, I hope this issue gets resolved soon (maybe when I come back tomorrow morning, it will suddenly work again, since if it can suddenly stop working, it might just as suddenly start working again).
Versions
Chroma 0.5.0
hnswlib 0.7.3
python 3.12.6
Windows10 Corp 22H2
CPU: Intel(R) Core(TM) i7-9700F CPU @ 3.00GHz
RAM: 32.0 GB
Relevant log output
[32mINFO[0m: [04-11-2024 12:19:03] Uvicorn running on http://0.0.0.0:8000 (Press CTRL+C to quit)
[32mINFO[0m: [04-11-2024 12:19:29] myIP:61951 - "GET /api/v1/tenants/default_tenant HTTP/1.1" 200
[32mINFO[0m: [04-11-2024 12:19:29] myIP:61951 - "GET /api/v1/databases/default_database?tenant=default_tenant HTTP/1.1" 200
[32mINFO[0m: [04-11-2024 12:19:30] myIP:61952 - "POST /api/v1/collections?tenant=default_tenant&database=default_database HTTP/1.1" 200
[32mINFO[0m: [04-11-2024 12:19:30] myIP:61952 - "POST /api/v1/collections?tenant=default_tenant&database=default_database HTTP/1.1" 200
[32mINFO[0m: [04-11-2024 12:19:30] myIP:61952 - "POST /api/v1/collections?tenant=default_tenant&database=default_database HTTP/1.1" 200
[32mINFO[0m: [04-11-2024 12:19:31] myIP:61952 - "POST /api/v1/collections?tenant=default_tenant&database=default_database HTTP/1.1" 200
[32mINFO[0m: [04-11-2024 12:19:31] myIP:61952 - "GET /api/v1/collections/redmine?tenant=default_tenant&database=default_database HTTP/1.1" 200
[32mINFO[0m: [04-11-2024 12:21:43] myIP:62055 - "GET /api/v1/pre-flight-checks HTTP/1.1" 200
[36mDEBUG[0m: [04-11-2024 12:21:43] Starting component PersistentLocalHnswSegment
What happened?
ChromaDB 0.5.0 suddenly crashed one morning due to a query after running normally for two months.
The last few logs are (The last one is "Starting component PersistentLocalHnswSegment" everytime when it crashes):
I eventually discovered that the reason was that the collection I was querying had more than 99 records, while another collection with only 83 records didn't have this issue. However, this doesn't make sense because this collection had been used normally with over a thousand records for two whole months!
Yes, there are some similar problems reported (such as #2513).
However, this time the situation is somewhat special. Two months ago, I installed and used ChromaDB, and after using it for a week, I encountered a similar issue. At that time, the database I created using chromadb.PersistentClient() would silently crash without any warning when inserting more than 99 records.
So, I followed the solution from the aforementioned issue(#2513) and downgraded ChromaDB to version 0.5.0 and hnswlib to 0.7.3, but it was of no use. Later, I accidentally discovered that when I switched to using chromadb.HttpClient() to start the database, everything returned to normal. Although the issue wasn't completely resolved, I felt that as long as the program could run, it was fine.
Therefore, ChromaDB worked normally for two months, then suddenly crashed during a query last Friday. Moreover, I can no longer insert any data into the database if the target collection already has more than 99 records or if my insertion would exceed 99 records.
People often say that when God closes a door, He opens a window. Yes, I once had that window, which was chromadb.HttpClient(). Now, this last window has been mercilessly closed. It was still working normally on Thursday night—it ran smoothly for two months, and the next day when I returned, it crashed? What the hell is that?! XD
And you know what? The exact same code and the exact same ChromaDB now work perfectly in a Linux virtual machine on my Windows system. Although it can work in a Linux virtual machine's Docker, this still causes unnecessary trouble for me when testing the program on the Windows system. Anyway, I hope this issue gets resolved soon (maybe when I come back tomorrow morning, it will suddenly work again, since if it can suddenly stop working, it might just as suddenly start working again).
Versions
Chroma 0.5.0
hnswlib 0.7.3
python 3.12.6
Windows10 Corp 22H2
CPU: Intel(R) Core(TM) i7-9700F CPU @ 3.00GHz
RAM: 32.0 GB
Relevant log output