[docs] LLM prompting guide#26274
Conversation
|
The documentation is not available anymore as the PR was closed or merged. |
|
The first draft of the LLM prompting guide is ready for review, let me know if anything major is missing. cc @patrickvonplaten |
patrickvonplaten
left a comment
There was a problem hiding this comment.
Very nice! My main feedback is:
- Let's maybe not pass default parameters to make sure the pipeline call function stays simple. E.g. I think we can remove all the
num_return_sequences=1statements as well as theeos_token_id=...statements as the model should have that set as the default (see here). - For task that do pure classification (sentiment analysis) or NER where there is arguably one and only one answer and where the model only generates a few tokens, I think it'd be better to not set
do_sample=Trueand instead leave the sampling to be greedy - I don't think we want to introduce any randomness there. Maybe a 1-2 liner explaining the difference could also help there
|
|
||
| This is a wrong answer, it should be 12. In this case, this can be due to the prompt being too basic, or due to the choice | ||
| of model, after all we've picked the smallest version of Falcon. Reasoning is difficult for models of all sizes, but larger | ||
| models are likely to perform better. |
There was a problem hiding this comment.
Nice I like this!
| * "Lead" the output in the right direction by writing the first word (or even begin the first sentence for the model). | ||
| * Use advanced techniques like [Few-shot prompting](#few-shot-prompting) and [Chain-of-thought](#chain-of-thought) | ||
| * Test your prompts with different models to assess their robustness. | ||
| * Version and track the performance of your prompts. |
Co-authored-by: Patrick von Platen <patrick.v.platen@gmail.com>
LysandreJik
left a comment
There was a problem hiding this comment.
Looks great :)
Only left a few comments
Co-authored-by: Lysandre Debut <hi@lysand.re>
|
Feel free to merge when satisfied with it! |
|
@LysandreJik I'm happy with it, but I think we should wait for @gante to review it once he's back from vacation. |
|
Gently pinging @gante for a review :) |
gante
left a comment
There was a problem hiding this comment.
This is a great guide, with clear examples, great suggestions, and relevant caveats. Big thumbs up, thank you for writing this guide @MKhalusova 💛
What does this PR do?
This PR addresses part 2.2 ("Prompting" ) of the issue #24575
It adds an LLM Prompting Guide to the docs that covers the following topics:
Let me know, if there's anything missing that has to be included.