GSM8K score mismatch between offical score and eka-eval

### Hi,

### Under the **Result and reporting section** in readme, the GSM8K score for Gemma 2B given is `42.3%`  but according to the official report of the Gemma 2B, GSM8K score is close to 17.7%. Check [here](https://huggingface.co/google/gemma-1.1-2b-it-GGUF) or [here](https://catalog.ngc.nvidia.com/orgs/nvidia/teams/nemo/models/gemma_2b_base).

### Or, Am I missing something?


---


<img width="1558" height="604" alt="Image" src="https://github.com/user-attachments/assets/dd8c0da2-5a91-4506-b270-ccca91afef99" />


<img width="970" height="822" alt="Image" src="https://github.com/user-attachments/assets/5b6d9e7e-548b-4612-bc75-7247f54cca98" />

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

GSM8K score mismatch between offical score and eka-eval #5

Hi,

Under the Result and reporting section in readme, the GSM8K score for Gemma 2B given is `42.3%` but according to the official report of the Gemma 2B, GSM8K score is close to 17.7%. Check here or here.

Or, Am I missing something?

Metadata

Assignees

Labels

Type

Fields

Projects

Milestone

Relationships

Development

GSM8K score mismatch between offical score and eka-eval #5

Description

Hi,

Under the Result and reporting section in readme, the GSM8K score for Gemma 2B given is 42.3% but according to the official report of the Gemma 2B, GSM8K score is close to 17.7%. Check here or here.

Or, Am I missing something?

Metadata

Metadata

Assignees

Labels

Type

Fields

Projects

Milestone

Relationships

Development

Issue actions

Under the Result and reporting section in readme, the GSM8K score for Gemma 2B given is `42.3%` but according to the official report of the Gemma 2B, GSM8K score is close to 17.7%. Check here or here.