Hi,
Unfortunately, it really depends on the model. For me, it’s working perfectly with Nous-Hermes-2-Mixtral-8x7B-DPO, but it doesn’t work for gemma-7b, for example. I don’t get the error messages you got, they are actually quite odd and don’t really make sense since Hugging Face is supposed to use the same API as OpenAI; I think everything they do on the API-side is unfortunately extremely beta still.
I suggest you try to get in touch with them to get more information about those errors. I would love to work with them better; I tried to contact them but they never replied.
Thread Starter
abulu
(@abulu)
Thank you for your reply, sir.
Hi, I am getting the error “AI Engine: No environment found for type (Array) when using Hugging Face model: Nous-Hermes-2-Mixtral-8x7B-DPO (or any models from Hugging Face for that matter).
May I know if I have missed out on some settings? I have followed the instruction to retrieve the API_URL, which in this case is “https://api-inference.huggingface.co/models/NousResearch/Nous-Hermes-2-Mixtral-8x7B-DPO”. Is this correct?
Thank you.