Dmytro Dzhulgakov (@dzhulgakov) / X

Dmytro Dzhulgakov

393 posts

Dmytro Dzhulgakov

@dzhulgakov

Co-founder and CTO @FireworksAI_HQ. PyTorch core maintainer. Previously FB Ads. Ex-Pro Competitive Programmer

San Francisco, CA

Joined April 2013

Dmytro Dzhulgakov
@dzhulgakov
Jul 30, 2023
GIL in Python will be no more. Huge win for AI ecosystem. Congrats to @colesbury - it took 4+ years of amazing engineering and advocacy. Many parts of @PyTorch could become simpler: DataLoader, Multi-gpu support (DDP), Python deployment (torch::deploy), … Here’s why. 🧵
Soumith Chintala
@soumithchintala
Jul 30, 2023
No More GIL! the Python team has officially accepted the proposal. Congrats @colesbury on his multi-year brilliant effort to remove the GIL, and a heartfelt thanks to the Python Steering Council and Core team for a thoughtful plan to make this a reality. discuss.python.org/t/a-steering-c…
543K
Dmytro Dzhulgakov
@dzhulgakov
Feb 28, 2025
Just had a crazy subtle bug in PyTorch/Triton code. Asked all reasoning LLMs: Claude 3.7 Thinking, Grok 3 Thinking, o1, o3-mini-high and DeepSeek R1 (on Fireworks of course) Only DeepSeek 🐳 R1 found the bug. Others hallucinated non-existent issues. Open source is the frontier!
86K
Dmytro Dzhulgakov
@dzhulgakov
Dec 11, 2023
@MistralAI model is hot: with mixture-of-experts, like GPT-4! It promises faster speed and lower cost than model of the same size and quality Fascinatingly, the speed-up is unevenly distributed: running on a laptop or the biggest GPU server benefit the most. Here’s why 🧵 1/7
128K
Dmytro Dzhulgakov
@dzhulgakov
Dec 8, 2023
Run @MistralAI MoE 8x7B locally with hacked-up official Llama repo! First cut, not optimized, requires 2 80GB cards. github.com/dzhulgakov/lla… The optimized version is coming soon to @thefireworksai . Stay tuned!
97K
Dmytro Dzhulgakov
@dzhulgakov
Oct 29, 2021
PyTorch is ready for the Metaverse. Move your models to virtual reality by passing device='meta' and save memory too! To align with this direction, starting next release Tensor․is_meta (pytorch.org/docs/stable/ge…) will always return True
Dmytro Dzhulgakov
@dzhulgakov
Sep 12, 2022
PyTorch has been truly open source since its inception 6 years ago. Today it gets truly open governance too: we join the Linux Foundation! As a maintainer, looking forward to continuing to build the best AI tools with the brilliant developer community!
ai.meta.com
Announcing the PyTorch Foundation: A new era for the cutting-edge AI framework
PyTorch is moving to a new, independent PyTorch Foundation. The project will join the Linux Foundation with a diverse governing board composed of representatives from AMD, Amazon Web Services, Google...
Dmytro Dzhulgakov
@dzhulgakov
Feb 27, 2025
Diffusion... for text, wow 🤯. Here's what it means: 1/ Super-speedy generation on GPUs. Groq/Cerebras are at a disadvantage here. Diffusion models (just like LLM training) are all about FLOPs, great for GPUs. Regular LLM inference is more about memory b/w and needs a lot of
Inception
@_inception_ai
Feb 26, 2025
We are excited to introduce Mercury, the first commercial-grade diffusion large language model (dLLM)! dLLMs push the frontier of intelligence and speed with parallel, coarse-to-fine text generation.
00:00
9.2K
Dmytro Dzhulgakov
@dzhulgakov
Aug 27, 2024
Congrats to Cerebras on the impressive results! How SRAM-only ASICs like it stack up against GPUs? Spoiler: GPUs still rock for throughput, custom models, large models and prompts (common "prod" things). SRAM ASICs shine for pure generation. Long 🧵
Cerebras
@cerebras
Aug 27, 2024
Introducing Cerebras Inference ‣ Llama3.1-70B at 450 tokens/s – 20x faster than GPUs ‣ 60c per M tokens – a fifth the price of hyperscalers ‣ Full 16-bit precision for full model accuracy ‣ Generous rate limits for devs Try now: inference.cerebras.ai
00:00
34K
Dmytro Dzhulgakov
@dzhulgakov
Jan 6, 2025
Replying to @isidentical
MoEs are interesting: they benefit most local bs=1 inference or huge scale deployments. DS has a really big deployment (300-400 gpus) based on their paper. We rushed to enable the model over holidays, more optimizations are on the way.
Dmytro Dzhulgakov
@dzhulgakov
Dec 11, 2023
@MistralAI model is hot: with mixture-of-experts, like GPT-4! It promises faster speed and lower cost than model of the same size and quality Fascinatingly, the speed-up is unevenly distributed: running on a laptop or the biggest GPU server benefit the most. Here’s why 🧵 1/7
16K
Dmytro Dzhulgakov
@dzhulgakov
Jul 31, 2024
This you? We ran your show-case example 3 times on Together playground, and it infinitely looped or answered incorrectly every time. Curious how that slipped through all 5 steps of your quality testing 🤔 Other examples from your blog fail to reproduce too. 🧵
00:00
53K
Dmytro Dzhulgakov
@dzhulgakov
Feb 15, 2024
Guessing Gemini Pro 1.5 uses regular self-attention based on speed. > Gemini 1.5 Pro successfully completed all the tasks asked of it, but not particularly quickly. Each took between ~20 seconds and a minute to process Assuming 1M tokens in 60 seconds on a 64-layer model with
Jeff Dean
@JeffDean
Feb 15, 2024
Gemini 1.5 Pro - A highly capable multimodal model with a 10M token context length Today we are releasing the first demonstrations of the capabilities of the Gemini 1.5 series, with the Gemini 1.5 Pro model. One of the key differentiators of this model is its incredibly long
41K
Dmytro Dzhulgakov
@dzhulgakov
Jan 31, 2025
Fastest DeepSeek 🐳 in the West from @FireworksAI_HQ (🇺🇸/🇪🇺 hosted) Note: serverless API speed varies as we deal with huge interest. More improvements to speed, throughput and cost are on the way. Talk to us about dedicated deployments for predictable speed.
01:02
Lin Qiao
@lqiao
Jan 31, 2025
🔥 @FireworksAI_HQ reaching 95 t/s for @deepseek_ai r1 671B 🔥 We know both latency and cost are important to creative application developers. As we continue to optimize our Deepseek serving stack, we are excited to see a wide range of applications onboarded from coding
49K
Dmytro Dzhulgakov
@dzhulgakov
Dec 19, 2021
Highlights of @PyTorch in 2021. Huge thanks to the community for making it awesome! Onwards to 2022!
Dmytro Dzhulgakov
@dzhulgakov
Mar 2, 2024
We’ve been cooking! Up to 300 tokens/s for Mixtral (and still fast with long prompts!) Reduced pricing, simplified billing, dedicated deployments and more. Enjoy!
Fireworks AI
@FireworksAI_HQ
Mar 1, 2024
Happy March! Fireworks is announcing our Spring 2024 platform updates - designed for improved production usage at scale. fireworks.ai/blog/spring-up… 🧵 💨Faster, more prod-ready serverless models - Mixtral Instruct and Llama 70B have become even faster, with speeds up to 300
26K