SOTA Attempt: Paid prefix (val_bpb=1.0238)#168
SOTA Attempt: Paid prefix (val_bpb=1.0238)#168spokane-way wants to merge 4 commits intoopenai:mainfrom
Conversation
|
After some more discussion, mind moving this to non-record submissions? Would accept there as an example. Some context: This isn't really a valid submission. In a text-compression benchmark, this would make some sense, given your model always should be a superior compressor than just storing the text itself/compressing the text with a standard compression algorithm. But given the benchmark isn't compressing the training data, but evaluating the performance of your compressor on an unseed IID validation dataset, this submission is equivalent to 'training on the validation set' directly with your prefix. |
|
@ibarrajo No, I'd only accept this PR, which implemented this idea first, as an example to explain why this type of approach isn't valid. |
|
I see, thanks for the clarification |
8.75MB paid-prefix blob