Cake: a Rust framework for distributed inference of large models like LLama3
Distributed LLM inference for mobile, desktop and server. - evilsocket/cake... (more…)
Read more »
In my spare time I’m a volunteer with my state’s emergency services and we have our own web app for managing unit-specific things like attendance, vehicle checks, newsletters, on-call rosters, and so on.
It’s actually a really useful tool, but there is on… Read more