Example where pressing up will navigate history instead of moving the cursor to the desired spot:
Additionally I'd prefer up to not navigate history unless the cursor is at (0,0), otherwise it's too easy to accidentally trigger history when revising a prompt.