I trained a 124m param GPT2 model with Google's new 'Lion' optimizer found through genetic programming and saw a 37.5% decrease in number of steps needed to reach the same loss as Adam (unusable models due to how small this test is)
paper: arxiv.org/pdf/2302.06675…
are there any artists embracing ai weirdness? are there ai glitch artists i can follow? so many people are focusing on realism and it's exhausting because we are not there yet
I have the first proof of concept MoLora (Mixture of Experts / LoRA) done and working! Here's a colab notebook to inference it (keep in mind, it's not fully trained, but it is working!) Details below..
colab.research.google.com/#fileId=https:…
my biggest trick for getting chatgpt to stop being lazy is saying i "need the full script for post-processing" whatever that means, it buys it most of the time
