Hmm yeah I think I misunderstood --embedding. I thought that it means "enable embd && disable completion"

So just to confirm, the server does support having some slots running embd and some slots running completion at the same time, right?

I'm asking this because I can't find llama_set_causal_attn anywhere in the server.cpp code

So just to confirm, the server does support having some slots running embd and some slots running completion at the same time, right?

I'm asking this because I can't find llama_set_causal_attn anywhere in the server.cpp code

I'm not really sure what is the state of this functionality, and AFAIK most people use the "embedding + completion" mode just for testing purposes (i.e. avoid starting 2 separate instances of llama-server). Technically, for getting the embeddings from a LLaMA model for example, you don't need to call llama_set_causal_attn(false). Just llama_set_embeddings(true) which we already do. There are models like GritLM which would require correct calls to llama_set_causal_attn and this is not supported by llama-server atm.