You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
The decoder appears to be adding an extra space at the end of the string:
tokenizer.decode(tokenizer.encode('I can feel the magic, can you?').ids) produces 'I can feel the magic, can you? ' instead of 'I can feel the magic, can you?'
The decoder appears to be adding an extra space at the end of the string:
tokenizer.decode(tokenizer.encode('I can feel the magic, can you?').ids)produces'I can feel the magic, can you? 'instead of'I can feel the magic, can you?'