Skip to content

New SOTA attempt (val_bpb=1.2014)#52

Merged
0hq merged 1 commit intoopenai:mainfrom
spokane-way:seq4096
Mar 19, 2026
Merged

New SOTA attempt (val_bpb=1.2014)#52
0hq merged 1 commit intoopenai:mainfrom
spokane-way:seq4096

Conversation

@spokane-way
Copy link
Copy Markdown
Contributor

Across three seeds: 1.2014, 1.1995, 1.2032

Better Muon momentum, LR, warmdown

Copy link
Copy Markdown
Collaborator

@0hq 0hq left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good!

@0hq 0hq merged commit 78c24e2 into openai:main Mar 19, 2026
scottspace pushed a commit to scottspace/parameter-golf that referenced this pull request Mar 21, 2026
Co-authored-by: spokane-way <spokane@way>
leonardcser pushed a commit to leonardcser/parameter-golf that referenced this pull request Mar 21, 2026
Co-authored-by: spokane-way <spokane@way>
nedcut pushed a commit to nedcut/parameter-golf that referenced this pull request Mar 26, 2026
Co-authored-by: spokane-way <spokane@way>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants