hey @gvidano,
sorry to hear you’re experiencing an issue. Can you give me a few more details?
what database type are you using? (WordPress, pinecone, or openai vector id)
how many entries are do you have in your database?
what chat model and embedding model?
thanks,
Maxwell
Hi Maxwell
Thanks for your quick response!
I am using wordpress database.
There are close to 6k entries on the knowledge base but planning on using up to 30k entries in the future as I add more content.
Chat model: GPT-4o
Embedding model: Ada2
Thanks so much for your help!
Hey @gvidano,
Thanks for the info. I’ve seen this error only a few times (one example from another user) and every single time it’s been related to the number of entries in the database when using WordPress. The WordPress database isn’t designed to handle that sort of scale and it’s often recommended if you’re going over 1k entries to use Pincone. I will add a warning message about this in the database section in the next update. It’s hard to give an exact number because it varies depending on your hosting provider and server environment.
Sometimes there are work arounds to make larger datasets work on WordPress (batching the requests), but this is extremely slow and you’re already pushing the limits on that with 6k and if you plan to go to 30k, certainly wouldn’t work.
I would for sure use Pinecone. It’s very easy to setup and has a very generous free tier. I can help you do it if needed, but you simply create an account, create your API key, click create database and simply match the dimensions of your embedding model (the number next to the embedding model name).
A simple way to test if this is for sure your issue (which im very confident it is given the number of entries + error) is to delete most of the information in your knowledge database and see if the chatbot works again.
Hey @gvidano,
I went back to your website and tested and its now working. I assume you either lowered your entries or switched to Pinecone. If you haven’t switched to Pinecone and are having trouble, please let me know and I can offer guidance.
*edit* Also, in the recent update just released today 3.0.5 it will now display a warning message to users who are using the WordPress database that if they plan for large database entries to use Pinecone to help guide users better in the future.
Thanks,
Maxwell
-
This reply was modified 1 month, 1 week ago by
m4xw3ll.
Hi
Thanks for your help!
I did erase all knowledge base and added just under 1k entries and the bot works good now.
I am trying to switch to pinecone but I am lost on how to add the knowledge base in pinecone from my sitemap URL’s as all the content is within the pages of my website. Could you guide me on how to do that please?
I was able to create an api key, a new index and all that but I am not sure how to add content to the database.
Thanks again for all your help!
disregard my last entry, I just realized you add all the content to the knowledge base to pinecone directly from the plugin.
Thanks so much for all your help, I am adding all content now! π
@gvidano yep you got it! I was just about to type up a response. You do it exactly how you did the WordPreess but it will show pinecone active in the sidebar.