Exploratory: PR315-derived candidate and looped-depth gate#453
Open
Divyesh-Thirukonda wants to merge 2 commits intoopenai:mainfrom
Open
Exploratory: PR315-derived candidate and looped-depth gate#453Divyesh-Thirukonda wants to merge 2 commits intoopenai:mainfrom
Divyesh-Thirukonda wants to merge 2 commits intoopenai:mainfrom
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Status
This PR is exploratory and is not claiming a new leaderboard record.
The attached
8xH100run for the main candidate is valid under the size cap, but it does not beat the existing PR315 frontier reference.What Is In This PR
Two record folders:
records/track_10min_16mb/2026-03-22_11L_XSA4_EMA_PartialRoPE_LNScale_Entropy_LongDocTTTrecords/track_non_record_16mb/2026-03-22_PR315_LoopedDepth_GateThe first folder captures a PR315-derived candidate with experimental codec and TTT branches behind flags. The second folder keeps looped-depth gate work separate from the primary path.
Official 8xH100 Result For The Main Candidate
From
train_seed42.log:step 4625on the600.037swallclock cap26152 MiBallocated /26526 MiBreserved15,733,011bytesfinal_quant_roundtrip_exact val_bpb = 1.16892776final_quant_sliding_window_exact val_bpb = 1.14586586This is worse than the checked-in PR315 reference (
1.1248sliding-windowval_bpb), so this PR should be treated as implementation and investigation work only.Next Step Outside This PR
The actual leaderboard path is now a separate exact-reproduction effort: recover PR315 throughput and score parity on the official image first, then make one change at a time.