Motivation and Context

Large Language Models (LLMs) operate with a defined limit on the number of tokens they can process at once, referred to as the context window. Exceeding this limit can have significant cost and performance implications. Therefore, it is essential to manage the size of the input sent to the LLM, particularly when using chat completion models. This involves effectively managing chat history and implementing strategies to truncate it when it becomes too large.

Related to #8518

Description

Samples demonstrating how to manage chat history, the following strategies are implemented:

1. Sending Only the Last N Messages: Retain the last few messages, including the System Message.
2. Limiting Based on Maximum Token Count: Ensure the total token count remains within a specified limit.
3. Summarizing Older Messages: Create a summary of previous messages to maintain context while reducing token usage.

Contribution Checklist

• The code builds clean without any errors or warnings
• The PR follows the SK Contribution Guidelines and the pre-submission formatting script raises no violations
• All unit tests pass, and I have added new tests where possible
• I didn't break anyone 😄