Skip to content

GSM8K score mismatch between offical score and eka-eval #5

@gitgoap

Description

@gitgoap

Hi,

Under the Result and reporting section in readme, the GSM8K score for Gemma 2B given is 42.3% but according to the official report of the Gemma 2B, GSM8K score is close to 17.7%. Check here or here.

Or, Am I missing something?


Image Image

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions