Recent trends in language modeling have focused on increasing performance through scaling, and have resulted in an environment where training language models is out of reach for most researchers...

Model-recycling - the best model per architecture. Comparing finetuned models from HF, as base models for future finetuning.