Skip to content

Fix Windows width calculation#215

Open
sophiajt wants to merge 1 commit intokkawakam:masterfrom
sophiajt:master
Open

Fix Windows width calculation#215
sophiajt wants to merge 1 commit intokkawakam:masterfrom
sophiajt:master

Conversation

@sophiajt
Copy link
Copy Markdown

This fixes the Windows prompt width calculation by using the same width calculation Unix uses. This code might have originally been in place to work around a Windows shortcoming, but it seems that current versions of Windows calculate width the same way Unix does

Before:

beforefix

After:

afterfix

@gwenn
Copy link
Copy Markdown
Collaborator

gwenn commented May 11, 2019

Sorry, I don't have access to a Windows machine right now (and only Windows 7 at work).
I guess your PR is fine but I don't understand why!
Because there should be no ansi escape sequence in prompt (only highlight_prompt should inject esc. seq.).
On unix platform, to keep backward compatibility, you can still use esc. seq. directly in prompt (but should not).

@sophiajt
Copy link
Copy Markdown
Author

Would the above fix not work for highlight_prompt?

@gwenn
Copy link
Copy Markdown
Collaborator

gwenn commented May 12, 2019

Your fix will work for highlight_prompt.
But prompt is not supposed to contain escape sequence (at least) on windows.
Could you please confirm that rustyline example was run without any modification ?
Thanks.

@sophiajt
Copy link
Copy Markdown
Author

Yes, the normal prompt works. This fix is for when people put ANSI sequences into their prompts.

@sophiajt
Copy link
Copy Markdown
Author

Basically it lets both kinds of prompts work correctly.

@sophiajt
Copy link
Copy Markdown
Author

oh I see what you're saying. You're talking about the screenshot. No, the PR doesn't change the default behavior of the example. I just changed it to make my testing a little easier.

@gwenn
Copy link
Copy Markdown
Collaborator

gwenn commented May 13, 2019

Ok,
Your screenshots are the result of rustyline example with the following modifications:
a prompt containing ansi escape sequences directly.
Right ?

My concerns are that:

  • ansi escape sequences are not supported yet on Windows < 10 => highlighting is disabled.
  • highlighting/colouring is (should be) disabled when terminal is not supported.
  • highlighting/colouring is (should be) disabled when stdout is not a tty.
  • highlighting may be disabled manually (https://docs.rs/rustyline/4.0.0/rustyline/config/enum.ColorMode.html).

When prompt already contains ansi escape sequences, it's not easy/efficient to remove them (when highlighting is disabled).

So I am not sure:

  • with your PR, behaviour is consistent on unix and Windows 10,
  • but highlighting cannot be disabled dynamically anymore.

@sophiajt
Copy link
Copy Markdown
Author

Just to be clear, the PR does not change the default prompt. The screenshot is only from my testing and is not part of the PR.

The PR only changes how the length is calculated. If there is no ANSI in the prompt, the length is calculated correctly.

@sophiajt
Copy link
Copy Markdown
Author

@gwenn - I'm not clear what you're asking me to change so that the PR can be accepted.

@gwenn
Copy link
Copy Markdown
Collaborator

gwenn commented May 14, 2019

@jonathandturner
I am so sorry.
With ConEmu (and Windows 7), without your PR but this :

diff --git a/src/tty/windows.rs b/src/tty/windows.rs
index 2a44206..aa47bbd 100644
--- a/src/tty/windows.rs
+++ b/src/tty/windows.rs
@@ -550,6 +550,7 @@ impl Term for Console {
                 let raw = original_stdstream_mode | wincon::ENABLE_VIRTUAL_TERMINAL_PROCESSING;
                 self.ansi_colors_supported =
                     unsafe { consoleapi::SetConsoleMode(self.stdstream_handle, raw) != 0 };
+                self.ansi_colors_supported = true;
             }
             Some(original_stdstream_mode)
         } else {

The prompt is displayed correctly:
unnamed

Could you please tell me:

  • the Windows version,
  • the console (cmd, powershell, conemu, ...)
  • rustyline version/sha

?

And if you have time, without your PR, add this line:

diff --git a/src/tty/windows.rs b/src/tty/windows.rs
index 2a4420689..ca0feeccf 100644
--- a/src/tty/windows.rs
+++ b/src/tty/windows.rs
@@ -398,6 +398,7 @@ impl Renderer for ConsoleRenderer {
             pos.col = 0;
             pos.row += 1;
         }
+        debug!(target: "rustyline", "orig: {:?}, s: {:?}, pos: {:?}", orig, s, pos);
         pos
     }

And send me the logs.
Thanks.

@sophiajt
Copy link
Copy Markdown
Author

I'm on cmd in Windows 10 (latest stable release). Using git sha which was the latest as of a few days ago aba9b21

The fix you suggested of forcing ansi doesn't fix the calculation.

Here are some sample logs:

     Running `target\debug\examples\example.exe`
[2019-05-15T12:46:27Z DEBUG rustyline::tty::windows] orig: Position { col: 0, row: 0 }, s: "\u{1b}[1;32m>>\u{1b}[0m ", pos: Position { col: 12, row: 0 }
[2019-05-15T12:46:27Z DEBUG rustyline::tty::windows] orig: Position { col: 12, row: 0 }, s: "", pos: Position { col: 12, row: 0 }
[2019-05-15T12:46:27Z DEBUG rustyline::tty::windows] orig: Position { col: 12, row: 0 }, s: "", pos: Position { col: 12, row: 0 }
>>          [2019-05-15T12:46:30Z DEBUG rustyline::keymap] Emacs command: SelfInsert(1, 't')
[2019-05-15T12:46:30Z DEBUG rustyline::undo] Changeset::insert(0, 't')
[2019-05-15T12:46:30Z DEBUG rustyline::tty::windows] orig: Position { col: 12, row: 0 }, s: "t", pos: Position { col: 13, row: 0 }
t[2019-05-15T12:46:30Z DEBUG rustyline::keymap] Emacs command: SelfInsert(1, 'e')
[2019-05-15T12:46:30Z DEBUG rustyline::undo] Changeset::insert(1, 'e')
[2019-05-15T12:46:30Z DEBUG rustyline::tty::windows] orig: Position { col: 12, row: 0 }, s: "te", pos: Position { col: 14, row: 0 }
e[2019-05-15T12:46:30Z DEBUG rustyline::keymap] Emacs command: SelfInsert(1, 's')
[2019-05-15T12:46:30Z DEBUG rustyline::undo] Changeset::insert(2, 's')
[2019-05-15T12:46:30Z DEBUG rustyline::tty::windows] orig: Position { col: 12, row: 0 }, s: "tes", pos: Position { col: 15, row: 0 }
s[2019-05-15T12:46:30Z DEBUG rustyline::keymap] Emacs command: SelfInsert(1, 't')
[2019-05-15T12:46:30Z DEBUG rustyline::undo] Changeset::insert(3, 't')
[2019-05-15T12:46:30Z DEBUG rustyline::tty::windows] orig: Position { col: 12, row: 0 }, s: "test", pos: Position { col: 16, row: 0 }

The column miscalculation is what this PR attempts to fix.

@gwenn
Copy link
Copy Markdown
Collaborator

gwenn commented May 15, 2019

Ok,

[2019-05-15T12:46:27Z DEBUG rustyline::tty::windows] orig: Position { col: 0, row: 0 }, s: "\u{1b}[1;32m>>\u{1b}[0m ", pos: Position { col: 12, row: 0 }

Is unexpected !
There should be no ansi escape sequence when calling calculate_position.

static PROMPT: &'static str = ">> ";
...
let readline = rl.readline(PROMPT);
...     let mut s = State::new(&mut stdout, prompt, helper, ctx);
...             let prompt_size = out.calculate_position(prompt, Position::default());

Expected:

[2019-05-15T16:26:05Z DEBUG rustyline::tty:: windows] orig: Position { col: 0, row: 0 }, s: ">> ", pos: Position { col: 3, row: 0 }

@sophiajt
Copy link
Copy Markdown
Author

@gwenn - as I mentioned earlier I am toggling the prompt to be the ansi prompt to test both non-ansi and ansi prompts.

The non-ansi version works fine (but it's not what I'm trying to fix). Example:

    Finished dev [unoptimized + debuginfo] target(s) in 0.92s
     Running `target\debug\examples\example.exe`
[2019-05-15T16:31:30Z DEBUG rustyline::tty::windows] orig: Position { col: 0, row: 0 }, s: ">> ", pos: Position { col: 3, row: 0 }
[2019-05-15T16:31:30Z DEBUG rustyline::tty::windows] orig: Position { col: 3, row: 0 }, s: "", pos: Position { col: 3, row: 0 }
[2019-05-15T16:31:30Z DEBUG rustyline::tty::windows] orig: Position { col: 3, row: 0 }, s: "", pos: Position { col: 3, row: 0 }
>> [2019-05-15T16:31:31Z DEBUG rustyline::keymap] Emacs command: SelfInsert(1, 'q')
[2019-05-15T16:31:31Z DEBUG rustyline::undo] Changeset::insert(0, 'q')
[2019-05-15T16:31:31Z DEBUG rustyline::tty::windows] orig: Position { col: 3, row: 0 }, s: "q", pos: Position { col: 4, row: 0 }
[2019-05-15T16:31:31Z DEBUG rustyline::tty::windows] orig: Position { col: 3, row: 0 }, s: "q", pos: Position { col: 4, row: 0 }
>> q[2019-05-15T16:31:33Z DEBUG rustyline::keymap] Emacs command: SelfInsert(1, 'i')
[2019-05-15T16:31:33Z DEBUG rustyline::undo] Changeset::insert(1, 'i')
[2019-05-15T16:31:33Z DEBUG rustyline::tty::windows] orig: Position { col: 3, row: 0 }, s: "qi", pos: Position { col: 5, row: 0 }
[2019-05-15T16:31:33Z DEBUG rustyline::tty::windows] orig: Position { col: 3, row: 0 }, s: "qi", pos: Position { col: 5, row: 0 }
>> qi            

@sophiajt
Copy link
Copy Markdown
Author

Are you saying that putting ansi in the default prompt is what is breaking it?

@sophiajt
Copy link
Copy Markdown
Author

I see. Sorry it took so long. I couldn't understand why it wouldn't be okay to have ANSI codes in the prompt.

As best as I can figure out now, you put two versions of the prompt in there: one with colors and one without. The one without colors is the one used for the calculation.

I went back to using a simple prompt and a colored prompt separately, and then did .color_mode(ColorMode::Forced) to used the colored prompt. This appears to work correctly with your fix.

     Running `target\debug\examples\example.exe`
[2019-05-15T16:41:18Z DEBUG rustyline::tty::windows] orig: Position { col: 0, row: 0 }, s: ">> ", pos: Position { col: 3, row: 0 }
[2019-05-15T16:41:18Z DEBUG rustyline::tty::windows] orig: Position { col: 3, row: 0 }, s: "", pos: Position { col: 3, row: 0 }
[2019-05-15T16:41:18Z DEBUG rustyline::tty::windows] orig: Position { col: 3, row: 0 }, s: "", pos: Position { col: 3, row: 0 }
>> [2019-05-15T16:41:19Z DEBUG rustyline::keymap] Emacs command: SelfInsert(1, 't')
[2019-05-15T16:41:19Z DEBUG rustyline::undo] Changeset::insert(0, 't')
[2019-05-15T16:41:19Z DEBUG rustyline::tty::windows] orig: Position { col: 3, row: 0 }, s: "t", pos: Position { col: 4, row: 0 }
[2019-05-15T16:41:19Z DEBUG rustyline::tty::windows] orig: Position { col: 3, row: 0 }, s: "t", pos: Position { col: 4, row: 0 }
>> t[2019-05-15T16:41:19Z DEBUG rustyline::keymap] Emacs command: SelfInsert(1, 'h')
[2019-05-15T16:41:19Z DEBUG rustyline::undo] Changeset::insert(1, 'h')
[2019-05-15T16:41:19Z DEBUG rustyline::tty::windows] orig: Position { col: 3, row: 0 }, s: "th", pos: Position { col: 5, row: 0 }
[2019-05-15T16:41:19Z DEBUG rustyline::tty::windows] orig: Position { col: 3, row: 0 }, s: "th", pos: Position { col: 5, row: 0 }
>> th[2019-05-15T16:41:19Z DEBUG rustyline::keymap] Emacs command: SelfInsert(1, 'i')
[2019-05-15T16:41:19Z DEBUG rustyline::undo] Changeset::insert(2, 'i')
[2019-05-15T16:41:19Z DEBUG rustyline::tty::windows] orig: Position { col: 3, row: 0 }, s: "thi", pos: Position { col: 6, row: 0 }
i[2019-05-15T16:41:20Z DEBUG rustyline::keymap] Emacs command: SelfInsert(1, 's')
[2019-05-15T16:41:20Z DEBUG rustyline::undo] Changeset::insert(3, 's')
[2019-05-15T16:41:20Z DEBUG rustyline::tty::windows] orig: Position { col: 3, row: 0 }, s: "this", pos: Position { col: 7, row: 0 }

@gwenn
Copy link
Copy Markdown
Collaborator

gwenn commented May 15, 2019

Yes, because there is no way to remove them on Windows < 10 where ansi esc seq are not supported, or if stdout is not a tty, or when the terminal is not supported.

And, I guess there is something wrong in rustyline because you should not have to force the color mode (on Windows 10). The bug should be here:

            // To enable ANSI colors (Windows 10 only):
            // https://docs.microsoft.com/en-us/windows/console/setconsolemode
            if original_stdstream_mode & wincon::ENABLE_VIRTUAL_TERMINAL_PROCESSING == 0 {
                let raw = original_stdstream_mode | wincon::ENABLE_VIRTUAL_TERMINAL_PROCESSING;
                self.ansi_colors_supported =
                    unsafe { consoleapi::SetConsoleMode(self.stdstream_handle, raw) != 0 };
            }

@gwenn
Copy link
Copy Markdown
Collaborator

gwenn commented May 15, 2019

I will try to have access to a windows 10 machine...
https://azure.microsoft.com/en-us/pricing/details/virtual-desktop/ ?

@sophiajt
Copy link
Copy Markdown
Author

sophiajt commented May 16, 2019

Since this PR fixes the issue where people use ANSI in their prompts, wouldn't it be okay to just accept it? It doesn't break people who don't use ANSI.

A use case I've seen is something like this:

let readline = rl.readline(&format!(
            "{}> ",
            Color::Green.paint(filepath.to_string())
        ));

Where you're also mixing in parameters into the prompt to display things like:

C:\Source> 

The most natural place to put this is when you write out the prompt. I'm not sure how else you would do it.

@gwenn
Copy link
Copy Markdown
Collaborator

gwenn commented May 16, 2019

Something like this.
I will try to see if I can fix the lifetime mismatch.

@gwenn
Copy link
Copy Markdown
Collaborator

gwenn commented May 16, 2019

But you are right. It's not user friendly.
https://python-prompt-toolkit.readthedocs.io/en/master/pages/reference.html?highlight=prompt#module-prompt_toolkit.formatted_text

Many places in prompt_toolkit can take either plain text, or formatted text. For instance the prompt() function takes either plain text or formatted text for the prompt.

@gwenn
Copy link
Copy Markdown
Collaborator

gwenn commented May 18, 2019

Ok, here is an example with a non static prompt.
Let me know if it looks awkward.

@gwenn
Copy link
Copy Markdown
Collaborator

gwenn commented May 19, 2019

Windows bug related to ansi colors is also fixed: #227.

@PaddiM8
Copy link
Copy Markdown

PaddiM8 commented Dec 10, 2020

Any way around this? Is this going to be fixed? :/

@mxrlbo
Copy link
Copy Markdown

mxrlbo commented Aug 31, 2022

Will there finally be a fix?

@gwenn
Copy link
Copy Markdown
Collaborator

gwenn commented Aug 31, 2022

As already explained, there should be no ansi escape seq directly in the prompt, only highlight_prompt may return ansi escape seq.
Or we need a PR to remove ansi escape seq if terminal doesn't support them...

@mxrlbo
Copy link
Copy Markdown

mxrlbo commented Aug 31, 2022

@gwenn You could also just allow the user to feed a usize into a readline function that places the cursor at the given distance from the beginning. That would allow for ansi codes to be used but doesn't affect non-ansi apps.

@gwenn
Copy link
Copy Markdown
Collaborator

gwenn commented Aug 31, 2022

@BlackBirdTV You mean that rustyline user should compute the size of the prompt ?
And cursor may not be at the end of the prompt by default because you can have an initial input...
Also final user may want no color but how to remove the ansi sequences from your already styled prompt ?

We may introduce a Prompt trait like:

trait Prompt {
  /// ANSI esc sequences stripped used colors are not supported or disabled.
  fn rawText(&self) -> &str;
  /// May contains ANSI esc sequences
  fn styledText(&self) -> &str;
  /// Display size
  //fn width(&self) -> usize;
}

with a default impl for &str.
And your impl would need something like strip-ansi-escapes for rawText.

@parasyte
Copy link
Copy Markdown

parasyte commented Mar 27, 2025

Since I hit this issue today, I spent some time to use ANSI codes in the prompt the supported way.

I wish to not clone the prompt string every time it is displayed, which is how #215 (comment) works. Unfortunately, I don't know what was supposed to appear in the "non-static example" in #215 (comment) because the link is broken.

My workaround is splitting the prompt into separate types: one for the plain prompt, and one rendered with ANSI codes.

#[derive(Debug)]
struct Prompt {
    count: usize,
    prompt: String,
}

impl Prompt {
    fn get(&self) -> &str {
        &self.prompt
    }
}

#[derive(Completer, Debug, Helper, Hinter, Validator)]
struct PromptHelper {
    colored_prompt: String,
}

impl PromptHelper {
    fn set_count(&mut self, count: usize, prompt: &mut Prompt) {
        prompt.count = count;
        prompt.prompt = format!("awesome app ({count})> ");
        self.colored_prompt = format!("{} ({count})> ", "awesome app".cyan());
    }
}

impl Highlighter for PromptHelper {
    fn highlight_prompt<'b, 's: 'b, 'p: 'b>(
        &'s self,
        _prompt: &'p str,
        _default: bool,
    ) -> Cow<'b, str> {
        Cow::Borrowed(&self.colored_prompt)
    }
}

This allows Editor::readline() to own the PromptHelper while simultaneously borrowing from Prompt. And I can update them together without getting out of sync:

while let Ok(line) = rl.readline(prompt.get()) {
    // Do stuff...

    // Conditionally update the prompt.
    if let (Some(helper), true) = (rl.helper_mut(), prompt.count != new_count) {
        helper.set_count(new_count, &mut prompt);
    }
}

The cloning isn't that bad. But the workaround isn't, either. I prefer the workaround; it just wasn't obvious to discover.

I see that some thought has been put into a trait for providing the two variants of the prompt string. That seems more reasonable. It's basically what I want!

I can pass ownership of my own Prompt struct to Editor and query it later with prompt_mut() to make changes. And because Editor knows how to get both variants, I don't need to pass any string references when I call rl.readline_with_prompt(), or whatever does the obvious thing.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants