Description:
I encountered an error while using the bfcl evaluate command. It appears that underscores in the path are causing incorrect splitting, leading to a KeyError. Below are the details:
Steps to Reproduce:
Run the following command:
bfcl evaluate --model /mnt/dolphinfs/hdd_pool/sft_model/qwen --test-category simple
Original path: /mnt/dolphinfs/hdd_pool/sft_model/qwen
The command results in the following error:
KeyError: '/mnt/dolphinfs/hdd/pool/sft/model/qwen'
Expected Behavior:
The command should correctly interpret the provided model path without splitting underscores.
Actual Behavior:
The underscores in the path are being split incorrectly, causing a KeyError.
Environment:
Operating System: centos
Python Version: python3.11
BFCL Version: newest
Request for Help:
Could you please provide guidance on how to resolve this path splitting issue, or update the documentation to reflect the correct usage?
Description:
I encountered an error while using the bfcl evaluate command. It appears that underscores in the path are causing incorrect splitting, leading to a KeyError. Below are the details:
Steps to Reproduce:
Run the following command:
bfcl evaluate --model /mnt/dolphinfs/hdd_pool/sft_model/qwen --test-category simpleOriginal path: /mnt/dolphinfs/hdd_pool/sft_model/qwen
The command results in the following error:
KeyError: '/mnt/dolphinfs/hdd/pool/sft/model/qwen'Expected Behavior:
The command should correctly interpret the provided model path without splitting underscores.
Actual Behavior:
The underscores in the path are being split incorrectly, causing a KeyError.
Environment:
Operating System: centos
Python Version: python3.11
BFCL Version: newest
Request for Help:
Could you please provide guidance on how to resolve this path splitting issue, or update the documentation to reflect the correct usage?