Skip to content

Re-enable label colors for issue list#4106

Merged
mislav merged 11 commits intocli:trunkfrom
heaths:issue4079
Aug 23, 2021
Merged

Re-enable label colors for issue list#4106
mislav merged 11 commits intocli:trunkfrom
heaths:issue4079

Conversation

@heaths
Copy link
Contributor

@heaths heaths commented Aug 11, 2021

This reverts commit 930ee60
and uses the text.Truncate function from PR #3519 that correctly
measures ANSI escape codes.

This reverts commit 930ee60
and uses the text.Truncate function from PR cli#3519 that correctly
measures ANSI escape codes.
@heaths
Copy link
Contributor Author

heaths commented Aug 11, 2021

As noted here, we might consider a couple alternatives if returning a closure for truncating labels isn't desirable:

  1. Just keep the parenthesis the default color e.g., white.
  2. Don't print the parenthesis. The tsvTablePrinter doesn't, though I suspect the idea was to better delineate the label column from other columns for the ttyTablePrinter.

@heaths
Copy link
Contributor Author

heaths commented Aug 11, 2021

In case it comes up, this new text.Truncate function does still count graphemes the same way but supports ANSI escape codes. See my comment on my other PR I lifted this from for more information.

@heaths
Copy link
Contributor Author

heaths commented Aug 11, 2021

image

@heaths
Copy link
Contributor Author

heaths commented Aug 11, 2021

I updated the PR to colorize (if supported TTY) the label separators as well

Copy link
Contributor

@mislav mislav left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is a pretty great use of the new DisplayWidth and Truncate implementations that pay respect to ANSI sequences 😍

One thing that I would like us to also think about before enabling label color in other places in gh is what happens to labels whose color is very dark? Currently they will get rendered with dark color on a dark background. This, of course, affects the gh issue view command too, but if we roll out the colors to issue list as well then we've made the problem more visible.

@heaths
Copy link
Contributor Author

heaths commented Aug 11, 2021

I was considering that last night and thought maybe any R, G, and/or B component beyond a certain threshold could trigger a lighter background, but for people that have light backgrounds the reverse is a problem (and is today) as well. I did some searching and can't find a reliable cross-platform way of querying background colors or even default colors.

Could instead take a page from git itself and add a --no-color option where needed, or maybe even global since quite a few commands have color. Effectively, I imagine this would not disable TTY but instead set the iostreams.ColorScheme to one with colors disabled.

FWIW, the random color generator for labels seems to favor more pastels and may already have some limit that randomly-chosen colors are rarely dark. Of course, people can always set them themselves. For the Azure SDK, we actually use color codes with intent, but still tend to favor medium hues since they look good on light or dark background.

@heaths
Copy link
Contributor Author

heaths commented Aug 11, 2021

To note, I love the colored labels because my team uses colors with intent. I can better identify issues at a glance. I agree there's already a problem with colors in the console, but more and more CLIs across platforms are doing it to the point I doubt a significant number of people have light-colored terminal backgrounds. I imagine it would get more difficult to use various CLIs and few I've seen support user-defined color schemes.

@mislav
Copy link
Contributor

mislav commented Aug 11, 2021

I did some searching and can't find a reliable cross-platform way of querying background colors or even default colors.

For rendering Markdown we do query the terminal for whether it has light or dark background, but this is brittle and doesn't work with Dark mode on macOS (Dark mode Terminal.app reports that it has a light background even though the colors are inverted per Dark mode)

if termenv.HasDarkBackground() {

We also allow the user to override the dark/light style with GLAMOUR_STYLE, but it might be confusing if that environment variable starts affecting more than just Markdown rendering.

or maybe even global since quite a few commands have color.

You can already disable color per invocation with NO_COLOR=1 or CLICOLOR=0

FWIW, the random color generator for labels seems to favor more pastels and may already have some limit that randomly-chosen colors are rarely dark.

That's true! In my experience, most colors for existing labels render really nicely out of the box. I haven't tested with light backgrounds or on Windows, though. (PowerShell default background color is blue, if I recall correctly.)

@heaths
Copy link
Contributor Author

heaths commented Aug 11, 2021

PowerShell default background color is blue, if I recall correctly.

Yes, but with Terminal going in-box for Windows 11, I wonder how much it'll matter; though, I admit I don't remember if the default Windows PowerShell (PowerShell nee "PowerShell Core" defaults to black background) profile for Terminal is blue or black. Hopefully, too, more people are using PowerShell over Windows PowerShell since the latter is effectively in sustained engineering.

Still, as you noted: you already have label colors elsewhere. While issue list may make this more apparent, do you consider there's a significant number of users with completing backgrounds and label colors to omit label colors by default? If it's already a significant enough problem, those environment variables (I didn't know about those; neat!) are available to turn them off globally.

@heaths
Copy link
Contributor Author

heaths commented Aug 11, 2021

Just installed Terminal in Windows Sandbox and the default Windows PowerShell profile is black:

image

Still, for Windows 10 and 11, powershell.exe i.e., Windows PowerShell, does default to blue.

@mislav
Copy link
Contributor

mislav commented Aug 11, 2021

Still, for Windows 10 and 11, powershell.exe i.e., Windows PowerShell, does default to blue.

See that's tricky. I also want everyone to use Windows Terminal instead of powershell.exe and cmd.exe. But I don't know how many developers are actually doing that. Furthermore, I personally wasn't able to install Windows Terminal from the Windows Store for months (installing would fail without visible error) until I spent 1h on the phone with Windows Support this week.

@heaths
Copy link
Contributor Author

heaths commented Aug 11, 2021

It's a fair point, but as you pointed out previously, color is already a problem. Here's powershell.exe on my system with defaults (I used to change the default to black before switching to Terminal):

image

Some of the colors already don't pop, including the default grays, but still legible. My main point was that color from any CLI for any terminal can already be a problem that people in that predicament have probably already worked around, and gh has a couple workarounds available if necessary. The color, in general, adds a lot of value whether it's label colors, rendered markdown, or even highlighting columns of info e.g., green for "keys", bright white for titles, and gray for "other". Users can disable it if enabled, but the reverse isn't true if never implemented.

I'll leave it at that. 🙂 If you're still interested and have feedback specific to the implementation I'm happy to make changes. It's primarily just a revert, though I had to add HexToRGB back (I didn't check the blame for when it was deleted, since it was still there for my other PR) and ported the Truncate func.

@mislav
Copy link
Contributor

mislav commented Aug 11, 2021

I appreciate your contribution and thoughts. This will likely be accepted, I just need to ruminate about this for a bit and run it by my team. Thank you!

@mislav
Copy link
Contributor

mislav commented Aug 18, 2021

I have pushed updates to:

  • Remove (...) around labels
  • Avoid outputting RGB color to 256-only (no truecolor) terminals
  • Remove colored ,

Now I wonder should we have fallback color (other than default foreground color) for terminals that don't support truecolor 🤔

@heaths
Copy link
Contributor Author

heaths commented Aug 18, 2021

I'm using Windows Terminal and not seeing colors with your changes:

image

@mislav
Copy link
Contributor

mislav commented Aug 18, 2021

@heaths And before my changes you've seen label colors in Windows Terminal?

It looks like Windows Terminal doesn't set either TERM nor COLORTERM variables, so I'm wondering how it assumed that the terminal even supported 256 colors before my changes.

Copy link
Contributor Author

@heaths heaths left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

From https://github.com/gookit/color, I found a link to https://github.com/termstandard/colors (via a gist that recommends that repo) that has another way to detect true color, but not sure how well that will work in practice. It also makes mention that several terminals do approximate.

As mentioned in previous comments, people can always disable color entirely if it's a problem, which with existing commands it may have been. Or maybe true colors were approximated and they never noticed.

@heaths
Copy link
Contributor Author

heaths commented Aug 18, 2021

@heaths And before my changes you've seen label colors in Windows Terminal?

It looks like Windows Terminal doesn't set either TERM nor COLORTERM variables, so I'm wondering how it assumed that the terminal even supported 256 colors before my changes.

Checking out my last commit - 2433eeb - no, it didn't (I've been developing this in WSL2/Ubuntu since it's faster):

image

But if I do set $env:TERM='xterm-truecolor' I, expectedly, get the same color output - seemingly approximated, given that difference in the red "bug" label - as above:

image

That said, trying this out in cmd.exe (the shell and terminal) shows that colors are effectively ignored, and I wonder if all (major, at least) terminals do this. As long as ANSI escape sequences have been a thing, any unrecognized sequences are supposed to be ignored anyway:

image

@heaths
Copy link
Contributor Author

heaths commented Aug 18, 2021

For some context, here's been one discussion about general detection but TERM and color support has come up: microsoft/terminal#1040. @DHowett commented that Terminal is truecolor-compliant but that many apps don't properly detect it - the subject of the first issue I mentioned above - and don't send proper ANSI escape sequences. Would be nice if gh wasn't one of those apps.

As a compromise, would it make sense to detect other heuristics like checking for WT_SESSION as suggested in the first bug? I agree it's rather specific, but since Windows Terminal support it and has been fairly popular - and other popular terminals could be added as requested - check for that env var in IsTrueColorSupported()?

mislav added 2 commits August 18, 2021 21:07
- Assume 256-color by default on all Windows
- When in Windows Terminal, assume truecolor
It seems that in powershell.exe terminal, if a line has exactly as many
characters as available viewport width, it will wrap. So on Windows (but
not under Windows Terminal) assume that the available width for table
output is one character narrower.
@mislav
Copy link
Contributor

mislav commented Aug 18, 2021

Thanks for doing the research! I've pushed some changes—please review!

I see that some tests are broken on Windows. Will fix.

As long as ANSI escape sequences have been a thing, any unrecognized sequences are supposed to be ignored anyway:

You would think so! But in my experience, sending ANSI escape sequences that a terminal doesn't support can lead to visual artifacts. For example, Terminal.app being sent truecolor sequences:

Weirdly, p3 has an underline (as if it were a hyperlink) while p2 is bold.

Copy link
Contributor Author

@heaths heaths left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Mainly, WT_SESSION is still defined for WSL(2) because Windows Terminal defines it regardless of the shell or platform running within it. I think just adding a check for it in the old color.go would be best, but the availableWidth check makes an incorrect assumption and would affect linux shells as well.


func IsTrueColorSupported() bool {
// Windows Terminal supports true color.
return os.Getenv("WT_SESSION") != ""
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This isn't Windows-specific. Windows Terminal defines this environment variable in WSL2 as well, despite TERM being defined as xterm-256color. I think either how you had it is fine (other Windows terminals may define TERM, and hopefully WT will as well per the link I posted), or just add a check for WT_SESSION:

image

)

func (t *ttyTablePrinter) availableWidth() int {
if os.Getenv("WT_SESSION") == "" {
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As mentioned above, WT defines this. It's not specific to powershell.exe which, if run directly, doesn't run in WT (though it may in Windows 11 if WT is set as the default terminal).

cmd.exe, powershell.exe, and pwsh.exe by default run in conhost.exe, and this problem - which I can repro - appears to happen only there (though, to be frank, I don't have anything other than conhost and WT installed). But with WT, I've never seen this.

So given all that, the check for WT_SESSION for truecolor should be platform agnostic, but if you wanted to solve this problem here, perhaps check that the environment windir is defined byt not WT_SESSION. Short of getting the current process name - certainly another option - that would probably be easiest.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sorry, I misread this as it was checking for empty. This is probably fine then. Checking for windir in this case would be pointless. If there are any other terminals people are running in Windows they are only losing a single block char. Anything popular enough could always be added to this.

@heaths
Copy link
Contributor Author

heaths commented Aug 18, 2021

I don't know if you're already started, but I'm fixing up the PR (and the build break) and can push another commit.

@DHowett
Copy link

DHowett commented Aug 18, 2021

Please do not use WT_SESSION to detect the presence of a true-color terminal on Windows. All Windows versions starting with the release of 10.0.14393 ("Anniversary Update", released August 2016) support true color SGR on the console when ENABLE_VIRTUAL_TERMINAL_PROCESSING is enabled.

Detecting WT_SESSION as a signal to enable true color will result in a degraded experience for people who are using Windows Server and versions of Windows that do not support Terminal, as well as people who opt to not use it, undeservedly.

I know that I am asking for you to do nothing to detect color support on Windows, and that that will cause some trouble. However, there only one version of Windows 10 still in support that supports SGR but not true color SGR.

I'm curious -- is the problem actually that this library does not turn on ENABLE_VIRTUAL_TERMINAL_PROCESSING, and it is turned on by default in Terminal? That would also explain your line wrapping behavior...

Going forward, and in general, any VT feature that is supported by Windows Terminal will in the fullness of time become supported by the Windows Console. They share a VT parser.

@mislav
Copy link
Contributor

mislav commented Aug 18, 2021

Thanks for reviewing! Don't worry about fixing the break; I will push a fix tomorrow. I now recognize that t.availableWidth() was the wrong abstraction level for fixing the wrapping issue. I will revert that change.

@DHowett Yeah, in retrospect I feel queasy about relying on WT_SESSION for anything. I will undo that as well, and likely just assume that on Windows we have truecolor support.

I'm not familiar with ENABLE_VIRTUAL_TERMINAL_PROCESSING. What is that, and is the responsibility of the app to turn it on or of the user?

@heaths
Copy link
Contributor Author

heaths commented Aug 18, 2021

@DHowett thanks for the recommendations, but I don't think gh is limited to Windows 10. Lots of developers still on Windows 7. I get that Windows only supports Windows 10 now, but I don't see why that should impact external tools - or even some Microsoft first-party tools that support older versions where the customer are, like Visual Studio.

@mislav you can call the Windows-native GetConsoleMode to determine if ENABLE_VIRTUAL_TERMINAL_PROCESSING (0x4) is set. You can also call SetConsoleMode to enable that for the output buffer, given the handle of stdout.

I agree if there's a better way it's worth doing. As for assuming truecolor - basically going back to the original color.go, it's seemingly been working in gh issue view. Maybe it's worth fixing if and when people actually find a problem with it. Seems no one has noticed yet, or not many are using gh issue view.

@DHowett
Copy link

DHowett commented Aug 18, 2021

Thanks for the info on the console modes -- you beat me to it!

I get that Windows only supports Windows 10 now, but I don't see why that should impact external tools

That's not what I'm recommending 😄. After all, Terminal has to support Windows 7 as well! We're well aware of the price we pay for compatibility.

I think the most correct thing to do¹ is to try to enable ENABLE_VIRTUAL_TERMINAL_PROCESSING, which will enlighten the entire console for VT sequences and fix the newline behavior, and switch to only using VT when you were successful. When the mode cannot be set, you are on a version of Windows that doesn't support VT.

¹ There is a small portion of Windows devices running 10.0.10240 that will accept this flag but do not support true color SGR. That's why it's only "most" correct, not completely correct! 😁

@heaths
Copy link
Contributor Author

heaths commented Aug 18, 2021

Thanks, @DHowett. I'm working on these changes now.

@heaths
Copy link
Contributor Author

heaths commented Aug 18, 2021

I found mattn/go-isatty#59 that may be a request for this already. Don't know if it's worth waiting to confirm or just implement locally, though a TODO in code may be sufficient for now. The brunt of the code necessary is arleady there, and gh already uses mattn/go-isatty.

@heaths
Copy link
Contributor Author

heaths commented Aug 18, 2021

There's a number of ways to do this. I see from here that we're already using mattn/go-colorable to call GetConsoleMode to see if ENABLE_VIRTUAL_TERMINAL_PROCESSING is enabled. We first just need to set it, and I think that would take care of most of it.

Talking with @DHowett offline, in this case probably the only reason we care if it's enabled (and can be enabled) is to know if we can use 256color or truecolor. Reading the environment variables is still worth it since other terminals - like cygwin - could be in use on Windows.

@heaths
Copy link
Contributor Author

heaths commented Aug 18, 2021

FWIW, mattn/go-colorable can also enable VT processing, but if it fails it assumes color is enabled. Given we actually detect VT support and assuming from that, based on conversations with @DHowett, that truecolor is supported (except on a small, old range of Windows 10, in which case the ANSI sequences should simply be ignored), I don't think that default assumption is desirable in this case.

@heaths
Copy link
Contributor Author

heaths commented Aug 18, 2021

In the course of testing on Windows 10 and linux (WSL2/Ubuntu 18.04), I found that - unrelated to this change (occurs with gh 1.14) - CLICOLOR_FORCE=1 gh issue list | less or redirecting to a file does not emit ANSI escape sequences. A brief glance through the code path makes it seem like it should.

@mislav
Copy link
Contributor

mislav commented Aug 19, 2021

@heaths Thanks for the updates on this PR! It looks great. I will review in depth and merge on Monday.

CLICOLOR_FORCE=1 gh issue list | less or redirecting to a file does not emit ANSI escape sequences. A brief glance through the code path makes it seem like it should.

Even though that environment variable forces color across all gh, the "is the output a tty?" bit is still off if you've piped stdout to anywhere. In that case, commands like gh issue list that ordinarily output a table switch to outputting TSV. The TSV mode, unlike the table output, doesn't have any colorization features. I think that makes sense because you want to use TSV for machine consumption, and colors would just get in your way in a lot of cases.

We're adding an environment variable that will force tty-style output even when redirected: #4146

@heaths
Copy link
Contributor Author

heaths commented Aug 19, 2021

the "is the output a tty?" bit is still off if you've piped stdout to anywhere

Makes sense. It wasn't really a problem - just unexpected. I can't think of any reason - apart from maybe debugging an output template with color or something e.g., piping to xxd - you'd want to emit ANSI color sequences when piping. Well, maybe for a pager like less, but it seems not even shell commands like ls do that. 🤷‍♂️

@mislav
Copy link
Contributor

mislav commented Aug 19, 2021

I can't think of any reason - apart from maybe debugging an output template with color or something e.g., piping to xxd - you'd want to emit ANSI color sequences when piping.

Exactly. And when piping to a program that will ultimately render the content to a terminal, like less -R or fzf --ansi, you will be able to explicitly tell gh to keep tty-style output via the new environment variable. That will automatically turn on color output again (unless explicitly disabled with CLICOLOR=0).

if stdoutIsTTY {
// also enables truecolor support for NewColorable below
if err := enableVirtualTerminalProcessing(os.Stdout); err == nil {
hasTrueColor = true
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This looks like it unconditionally considers that every non-Windows terminal supports truecolor. We will rely on TERM and COLORTERM variables instead. I'll push an update

Copy link
Contributor

@mislav mislav left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @heaths for the hard work and patience and @DHowett for advice! 🙇

@mislav mislav merged commit 88af63d into cli:trunk Aug 23, 2021
@heaths heaths deleted the issue4079 branch August 24, 2021 10:53
abdfnx added a commit to secman-archive/gh-api that referenced this pull request Aug 30, 2021
* Re-enable label colors for issue list
* Drop parentheses wrapping issue labels
* Support ANSI escape codes in TablePrinter cells
* Switch to a Truncate implementation that correctly measures ANSI escape codes
* Only output RGB color if terminal has truecolor capabilities
* Enable `ENABLE_VIRTUAL_TERMINAL_PROCESSING` on Windows - fixes wrapping issues with full lines and allows truecolor rendering
@stalkerg
Copy link

What about the same thing but for pr?

@mislav
Copy link
Contributor

mislav commented Dec 12, 2022

@stalkerg Sorry, right now gh pr list doesn't print labels because it wouldn't fit in a narrow terminal. We have no plans to change it soon.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants