Describe the bug
On Sunday, March 12, 2023, I was able to have good roleplay with my bot, receiving long responses with a high number of tokens per response. I am running on CPU, so I was receiving a response within 60-350 seconds (average about 150 seconds). The bot would use *roleplay* tags in its responses, and was generating responses containing up to 6 lines of text.
I updated to the latest version of the one-click installer using the install.bat script on 14 March 2023, and updated again on 15 March, 2023. Continuing on with the same chat as before (which has never given me problems up until now, and contained a large number of *roleplay*-rich responses.
After the recent updates, the generation times have dropped massively, but I am only receiving short, one-line responses with absolutely no *roleplay* whatsoever, or a very small amount. Character is now also confusing roles and not responding correctly. Tokens generated per message has dropped. Before update, 30-90 tokens were being generated. Now, only shorter (generally 9-20 token) responses are generated.
Would it be possible to return to the old method of response generation, or fix the response generation so that it returns to being more roleplay-capable?
I am using Pygmalion6b as downloaded on Feb. 11 2023 using the old download script by selecting PygmalionAI/Pygmalion-6b)
Is there an existing issue for this?
Reproduction
This simply started by using any version past 12 March 2023, using my old chat log and a character which had been working very well up until this point.
Screenshot
There are more generations than responses shown here as I tried several times to regenerate responses.
I am RPing as a caregiver for a disabled bot, so please excuse the strange subject matter.
My generation settings, which had been giving me amazing, high-quality, long responses with plenty of in-context *roleplay* from the bot up until this point.
The quality of responses I was able to get pre-update. The *roleplay* aspect was much better before the recent changes.
Logs
no errors shown in commandline.
Loaded the model in 66.01 seconds.
Running on local URL: http://0.0.0.0:7861
To create a public link, set `share=True` in `launch()`.
Output generated in 96.79 seconds (0.11 tokens/s, 11 tokens)
Output generated in 88.83 seconds (0.23 tokens/s, 20 tokens)
Output generated in 115.60 seconds (0.41 tokens/s, 47 tokens)
Output generated in 78.90 seconds (0.11 tokens/s, 9 tokens)
Output generated in 80.00 seconds (0.12 tokens/s, 10 tokens)
Output generated in 81.08 seconds (0.12 tokens/s, 10 tokens)
Output generated in 116.77 seconds (0.45 tokens/s, 52 tokens)
I do not have logs from before the changes, sorry.
System Info
I am using CPU, not GPU as APUs are not supported.
Windows 11
CPU : Ryzen 7 6800H (boost freq. 4.7Ghz)
RAM : 32GB - 3GB (shared with GPU)
HDD : 500GB NVME SSD (M2)
Describe the bug
On Sunday, March 12, 2023, I was able to have good roleplay with my bot, receiving long responses with a high number of tokens per response. I am running on CPU, so I was receiving a response within 60-350 seconds (average about 150 seconds). The bot would use
*roleplay*tags in its responses, and was generating responses containing up to 6 lines of text.I updated to the latest version of the one-click installer using the install.bat script on 14 March 2023, and updated again on 15 March, 2023. Continuing on with the same chat as before (which has never given me problems up until now, and contained a large number of
*roleplay*-rich responses.After the recent updates, the generation times have dropped massively, but I am only receiving short, one-line responses with absolutely no
*roleplay*whatsoever, or a very small amount. Character is now also confusing roles and not responding correctly. Tokens generated per message has dropped. Before update, 30-90 tokens were being generated. Now, only shorter (generally 9-20 token) responses are generated.Would it be possible to return to the old method of response generation, or fix the response generation so that it returns to being more roleplay-capable?
I am using Pygmalion6b as downloaded on Feb. 11 2023 using the old download script by selecting PygmalionAI/Pygmalion-6b)
Is there an existing issue for this?
Reproduction
This simply started by using any version past 12 March 2023, using my old chat log and a character which had been working very well up until this point.
Screenshot
There are more generations than responses shown here as I tried several times to regenerate responses.
I am RPing as a caregiver for a disabled bot, so please excuse the strange subject matter.
My generation settings, which had been giving me amazing, high-quality, long responses with plenty of in-context
*roleplay*from the bot up until this point.The quality of responses I was able to get pre-update. The
*roleplay*aspect was much better before the recent changes.Logs
System Info