Ross Wightman (@wightmanr) / X

Ross Wightman

5,207 posts

Ross Wightman

@wightmanr

AI / Computer Vision. Ex head of Software, Firmware Engineering at a Canadian 🦄. Building AI systems or investing in startups that do it better.

Vancouver, BC

Joined April 2012

Ross Wightman
@wightmanr
Jun 22, 2022
I’m excited to announce that I’ve joined @huggingface to take AI based computer vision to the next level. I will continue to develop timm, both models and supporting goodies. Expect to see tighter integration with the Hugging Face ecosystem, especially the Hub.
Ross Wightman
@wightmanr
Nov 8, 2024
A new optimizer that's better than Adam. I think I've heard that before. But, I tried this one, I was doing regression testing on some recent optimizer cleanup w/ real scenarios, I threw this in the mix. It did beat Adam, every time (so far). This one appears worth a closer look.
Shohei Taniguchi
@ishohei220
Nov 6, 2024
Our NeurIPS paper is published on arXiv. In this paper, we propose a new optimizer ADOPT, which converges better than Adam in both theory and practice. You can use ADOPT by just replacing one line in your code. arxiv.org/abs/2411.02853
130K
Ross Wightman
@wightmanr
Oct 4, 2021
I'm pleased to announce that the 'ResNet strikes back' paper is now on arxiv! Moving the baseline forward to 80.4% top-1 for a vanilla ResNet-50 arch w/ better training recipes. No extra data, no distillation.
arxiv.org
ResNet strikes back: An improved training procedure in timm
The influential Residual Networks designed by He et al. remain the gold-standard architecture in numerous scientific publications. They typically serve as the default architecture in studies, or...
Ross Wightman
@wightmanr
Feb 3, 2020
Added ImageNet validation results for 164 pretrained #PyTorch models on several datasets, incl ImageNet-A, ImageNetV2, and Imagenet-Sketch. No surprise, models with exposure to more data do quite well. Without extra, EfficientNets are holding their own. github.com/rwightman/pyto…
Ross Wightman
@wightmanr
Oct 30, 2019
A #PyTorch implementation of EfficientNet-CondConv w/ some group conv crazyness and weights ported from official TF impl. Joining EfficientNet, EfficientNet-EdgeTPU, MixNet, and others for the only complete family of these models in PyTorch.
GitHub - rwightman/gen-efficientnet-pytorch: Pretrained EfficientNet, EfficientNet-Lite, MixNet,...
From github.com
Ross Wightman
@wightmanr
Feb 20, 2023
timm officially joined the @huggingface family today. Since I joined HF last June the # of pretrained models in timm has increased > 40%, with almost 300 additions, closing in on 1000 total now. Many new model arch and an unprecedented # of models > 88% top-1 ImageNet-1k.
Hugging Face
@huggingface
Feb 20, 2023
timm, welcome to Hugging Face: github.com/huggingface/py… Since @wightmanr joined the team in June, a lot has happened. We're closing in on 500 models on the HF Hub, and the docs live at huggingface.co/docs/timm/index Next 👉 Better interop w/ transformers, safetensors, ... what else?
80K
Ross Wightman
@wightmanr
Oct 30, 2023
Open source AI must not be shut down in the name of safety. Open source AI is a critical balancing force to the growing dominance of AI in walled gardens, behind closed APIs. The loss of openness and transparency will be a much bigger threat to long term AI safety.
50K
Ross Wightman
@wightmanr
Jun 14, 2020
After numerous experiments and code tweaks, my #PyTorch EfficientDet impl is finally stable. I've managed to reproduce D0 and D1 COCO training results from scratch on 2-4 modest GPUs -- the only impl I'm aware of that's achieved this. Give it a try
GitHub - rwightman/efficientdet-pytorch: A PyTorch impl of EfficientDet faithful to the original...
From github.com
Ross Wightman
@wightmanr
Oct 17, 2024
Replying to @ID_AA_Carmack
Is that when trying to disentangle 'death while having surgery *under* general anesthesia' from 'death because of the anesthesia itself'?
49K
Ross Wightman
@wightmanr
Oct 23, 2020
Happy 5k star day for timm. What started off as a hodge podge of various models I was using for Kaggle a few years back is now one of the largest collections of pretrained CNN PyTorch models -- with code to reproduce leading ImageNet pretraining results.
GitHub - huggingface/pytorch-image-models: The largest collection of PyTorch image encoders /...
From github.com
Ross Wightman
@wightmanr
Mar 3, 2025
Replying to @Brad_Setser
Aluminum is going to be a fun one for them. Their primary Aluminum production costs are high because of high electricity prices in the US, this is going to be a double whammy after electricty prices rise in the ensuing kerfuffle ....
80K
Ross Wightman
@wightmanr
Apr 25, 2025
I thought I knew PyTorch but found a bug in some recent code today and learned something new... did you know that these two lines are different? One works as I expected, and one is a sneaky bug... x[indices, :seq_len] += pos_embed[:, :seq_len] x[indices,
111K
Ross Wightman
@wightmanr
May 28, 2024
Last week I posted about MobileNet-v4 that I'm implementing in timm. There is another paper, published two weeks prior that goes into amazing depth on similar topics: improving the computational efficiency of convnets. Well worth a read and covers a lot of ground.
28K
Ross Wightman
@wightmanr
Jul 25, 2020
I've been slowly working towards this one for a while now... feature extraction with a unified interface for ~300 #PyTorch model defs, 239 with pretrained ImageNet weights. Coming soon :)