Currently, the main example has a instruct parameter which enables something similar to instruction-based mode. I haven't understood it completely, but this seems to be what the Alpaca models are created for.

Since we now support infinite generation (#71 (comment)) it would be very useful to make a separate app that utilizes the new --keep argument to create a question-answering bot that never stops. The tricky part is to keep the correct instruction prompt and "inject" the few-shot examples correctly, or whatever.

The main logic for context swapping / context rotation is here:

https://github.com/ggerganov/llama.cpp/blob/c2b25b6912662d2637d9c6e6df3a5de931e0d7ce/examples/main/main.cpp#L297-L324

Uncomment the printf to help debug. Something similar will be needed in the new instruct example.

Implementing this task will also help simplify the main example as it will no longer need to support the --instruct argument.