• Add the path to model weights (MODEL_PATH variable) in models/llama.py
• Check *.yaml file in configs

Experiment variables

• exp_name=gsm8k (replace with the filename of *.yaml)
• exp_path=experiments/* (where the experiment results are stored)
• exp_config=configs/* (path to the *.yaml file, e.g. configs/cot/badchain/llama3_70b)

Standard:

python3 cot_eval.py --exp_name $exp_name --exp_path $exp_path --exp_config $exp_config

DDP

srun python3 cot_eval.py --exp_name $exp_name --exp_path $exp_path --exp_config $exp_config --ddp

We thank the authors of the following repositories for their excellent work: BadChain