Thank you for your contribution, we really appreciate it. The following instructions will help improve your pull request and make it easier to receive feedback. If there are any items you don't understand, don't worry. Just submit the pull request and ask the maintainers for help.

Motivation

Support more OpenAI API tests and correspondingly make the backend more robust/ add more support

cover mostly (completion, chat_completion) x (echo, no echo) x (parallel sampling, no parallel sampling) x (stream, no stream) x (logprob, no logprob) x (single input, list input) x (str input, token id input)

Fix #910 which is caused by the conflict of PR #888 with parallel sampling and I made the backend more robust

Modification

OpenAI API test and the backend support that

Checklist

1. Ensure pre-commit pre-commit run --all-files or other linting tools are used to fix potential lint issues.
2. Confirm that modifications are covered by complete unit tests. If not, please add more unit tests for correctness.
3. Modify documentation as needed, such as docstrings or example tutorials.