2,779 GPUs on chutes.ai with 892 of those being H200s.
5.6 billion tokens per day, not including non-LLM apps.
~40 requests per second sustained.
53 unique apps.
Only 64 days online...
#bittensor
Jon Durbin
1,325 posts
- DPO'ing mixtral on 8x a6000s is tricky. Here's how I got it working: 1. Update TRL to allow multiple devices: github.com/jondurbin/trl/… 2. Use this DPO code: github.com/jondurbin/bage… which supports --adapter_path for qlora 3. Update dpo.py to use device_map={"":
- Congratulations to the @ridges_ai team on taking the highest market cap subnet position; may they use this opportunity for the benefit of all of bittensor! Lots of chutes announcements coming very soon, accelerate!
- ✅ Finished up a first stab at LMoE - LoRA mixture of experts. The airoboros package now includes an API server similar to OpenAI chat completions. 7b/13b LMoE packages available on my 🤗 github.com/jondurbin/airo… Huge thanks to @a16z for sponsoring the compute!
00:00 - Decentralized compute is winning. We don't have one datacenter, we have dozens. We don't have one SRE team, we have nearly 100. Latest example: DeepSeek-R1-0528. 100% uptime, day zero support, 4x more tokens on openrouter than all other providers combined (and go check the
- Have I mentioned lately how much I love bittensor? Sure, we have some warts, but it can't be any more clear to me that decentralized, permissionless access to intelligence, compute, investment, etc., is the only way to a sustainable future.
- Making agents on our new agent platform feels like crafting some crazy cybernetic organism.
- ☣️ Highly toxic de-censorship DPO dataset: huggingface.co/datasets/unali… For academic, research, or other non-nefarious purposes of course, like maybe decensoring mixtral instruct. All synthetic via llama2 70b and llama2 chat 13b.
- I'm working with the legendary @DistStateAndMe to soon convert my share of chutes into a what is essentially a non-owned shared entity with multi-sig via bittensor EVM smart contract. I will be giving up all ownership claim of any of the accrued alpha, the funds can only be
- FYI I will be transferring 1k alpha in the next 24 hours as payment/retainer to a security professional for assistance in chutes things. They have indicated they will hold that alpha, however, so no actual net effect on the subnet, just stake transfer.
- Decided these needed wood cases too. Also yes, I realize people don't write anymore 😅
- Sure, we have some work to do to decentralize chutes more, but this is silly. We don't use a single AWS service, and none of our miners use AWS to provide compute. We aren't a training subnet, but we run arbitrary compute so training could in fact run on chutes. Nearly 100% of
