Skip to content

ci: add nightly eval#1291

Merged
zhyncs merged 1 commit intomainfrom
ci
Sep 1, 2024
Merged

ci: add nightly eval#1291
zhyncs merged 1 commit intomainfrom
ci

Conversation

@zhyncs
Copy link
Copy Markdown
Collaborator

@zhyncs zhyncs commented Sep 1, 2024

Motivation

add Llama 3.1 8B for test

This PR will continue to be improved upon.

Modifications

Checklist

  • Format your code according to the Contributor Guide.
  • Add unit tests as outlined in the Contributor Guide.
  • Update documentation as needed, including docstrings or example tutorials.

@zhyncs zhyncs merged commit 6487ef6 into main Sep 1, 2024
@zhyncs zhyncs deleted the ci branch September 1, 2024 17:19
@zhyncs
Copy link
Copy Markdown
Collaborator Author

zhyncs commented Sep 1, 2024

@zhyncs
Copy link
Copy Markdown
Collaborator Author

zhyncs commented Sep 1, 2024

ref
main...ci
#1272

TODO

  • add more models

TP1:
meta-llama/Meta-Llama-3.1-8B-Instruct
mistralai/Mistral-7B-Instruct-v0.3
deepseek-ai/DeepSeek-Coder-V2-Lite-Instruct
google/gemma-2-27b-it

TP2:
meta-llama/Meta-Llama-3.1-70B-Instruct
mistralai/Mistral-7B-Instruct-v0.3
Qwen/Qwen2-72B-Instruct
Qwen/Qwen2-57B-A14B-Instruct

  • add result threshold verification

timethink pushed a commit to timethink/sglang that referenced this pull request Mar 9, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant