Record: Loqui Auris — 10L + SWA + Standard TTT (val_bpb=1.1100)#595
Closed
LoquiAuris wants to merge 2 commits intoopenai:mainfrom
Closed
Record: Loqui Auris — 10L + SWA + Standard TTT (val_bpb=1.1100)#595LoquiAuris wants to merge 2 commits intoopenai:mainfrom
LoquiAuris wants to merge 2 commits intoopenai:mainfrom
Conversation
Contributor
|
As far as I can tell here, this proposed TTT scheme trains on the validation set by reporting the score on a doc after its weights have adapted to it, rendering this unsound for the purposes of this competition. Specifically, you use #442's TTT scheme, which was ruled out. |
Author
|
Hi Valerio,
Thank you for the clarification. Understood, I'll focus on architectural
improvements going forward.
Separate question: are compute grants still available? I submitted a
request through the form and wanted to confirm it's still being processed.
My current work is focused on non-TTT architecture optimizations (11L,
improved quantization, activation functions) and additional compute would
help with validation runs.
Best regards,
Eli Pancamo
…On Tue, Mar 24, 2026 at 10:13 AM valerio-oai ***@***.***> wrote:
*valerio-oai* left a comment (openai/parameter-golf#595)
<#595 (comment)>
As far as I can tell here, this proposed TTT scheme trains on the
validation set by reporting the score on a doc after its weights have
adapted to it, rendering this unsound for the purposes of this competition.
Specifically, you use #442
<#442>'s TTT scheme, which
was ruled out.
—
Reply to this email directly, view it on GitHub
<#595?email_source=notifications&email_token=A6XCGYPLIQJJ6MBKYEUJKVL4SKJWZA5CNFSNUABFM5UWIORPF5TWS5BNNB2WEL2JONZXKZKDN5WW2ZLOOQXTIMJRHA3DGNZXGYYKM4TFMFZW63VGMF2XI2DPOKSWK5TFNZ2KYZTPN52GK4S7MNWGSY3L#issuecomment-4118637760>,
or unsubscribe
<https://github.com/notifications/unsubscribe-auth/A6XCGYLEXX247QG5EWL62OD4SKJWZAVCNFSM6AAAAACW44FUROVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHM2DCMJYGYZTONZWGA>
.
You are receiving this because you authored the thread.Message ID:
***@***.***>
|
Contributor
|
Hi Eli, thanks for understanding, and don't take the closing of this PR as me trying to clip your wings! Compute grants are still available, they just take some time to come through :) |
Author
|
Valerio, Thanks for the info on the grants... And no worries, wings not
clipped, wings are stronger..
I'm glad the challenge is back to being based on the core architecture
where it really matters..
Kindest regards
Eli
…On Tue, Mar 24, 2026 at 11:46 AM valerio-oai ***@***.***> wrote:
*valerio-oai* left a comment (openai/parameter-golf#595)
<#595 (comment)>
Hi Eli, thanks for understanding, and don't take the closing of this PR as
me trying to clip your wings! Compute grants are still available, they just
take some time to come through :)
—
Reply to this email directly, view it on GitHub
<#595?email_source=notifications&email_token=A6XCGYOGR3BQ5ZVEP6MNWU34SKUWNA5CNFSNUABFM5UWIORPF5TWS5BNNB2WEL2JONZXKZKDN5WW2ZLOOQXTIMJRHEZTIMRSGMY2M4TFMFZW63VGMF2XI2DPOKSWK5TFNZ2KYZTPN52GK4S7MNWGSY3L#issuecomment-4119342231>,
or unsubscribe
<https://github.com/notifications/unsubscribe-auth/A6XCGYKB46U6Q4BSPAYMYF34SKUWNAVCNFSM6AAAAACW44FUROVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHM2DCMJZGM2DEMRTGE>
.
You are receiving this because you authored the thread.Message ID:
***@***.***>
|
RoyiRa
added a commit
to RoyiRa/parameter-golf
that referenced
this pull request
Mar 25, 2026
PR openai#595 achieves 1.1100 BPB with AdamW TTT (10ep, lr=5e-4). Add TTT_OPTIMIZER env var to switch between SGD (default) and AdamW. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Acknowledgments