Coding Tutorials and Development Tips
Running large language models locally is super rewarding, but VRAM is the real killer. A cheap trick I’ve been using to squeeze out more VRAM without dropping cash on a whole new high-end rig? Just….
Read more
If you’ve spent more than 5 minutes browsing Hugging Face for a new LLM to run locally, you’ve definitely hit this wall. You look up a model like Qwen or Gemma, and you’re faced with….
Read more
In the rapidly evolving of AI models, understanding the underlying architecture of language models is key to appreciating their capabilities and limitations. At the moment, two main approaches stand out: Dense Models and Mixture of….
Read more
If you’re getting into local LLMs for SillyTavern roleplay, the biggest bottleneck is your VRAM. After testing various setups (and lots of models 🤣), here is the reality of what you can expect based on….
Read more