Fix broken pipe in command #1680

Lomadriel · 2019-03-07T05:58:00Z

Fix #916.
As I said in #916, the bug was:

The child replaces STDIN, STDERR and STDOUT by the pipe.
The child closes the unused pipes
Since the command (cmd &) does a fork the wait function returns immediately while the process is still alive.
The destructor of the command closes the pipe (parent side)
The new process writes into STDOUT (alias the closed pipe)
The new process get a SIGPIPE and is killed.

I modified the class command, to create "two classes", one that redirects the output with pipes, and one that doesn't redirect output.
I refactored the rest of the code in consequence.

EDIT: At the moment, users still need to use cmd & to avoid blocking polybar. It may be interesting to add a parameter to the constructor to detach the child.

EDIT: Ref #1717

codecov-io · 2019-03-07T06:48:09Z

Codecov Report

Merging #1680 into master will increase coverage by 0.50%.
The diff coverage is 63.72%.

@@            Coverage Diff            @@
##           master   #1680      +/-   ##
=========================================
+ Coverage    9.40%   9.90%   +0.50%     
=========================================
  Files         181     182       +1     
  Lines       10871   10922      +51     
=========================================
+ Hits         1022    1082      +60     
+ Misses       9849    9840       -9

Flag	Coverage Δ
#unittests	`9.90% <63.72%> (+0.50%)`	⬆️

Impacted Files	Coverage Δ
include/components/types.hpp	`0.00% <ø> (ø)`
include/modules/script.hpp	`0.00% <ø> (ø)`
src/adapters/net.cpp	`0.00% <0.00%> (ø)`
src/components/controller.cpp	`0.00% <0.00%> (ø)`
src/modules/ipc.cpp	`0.00% <0.00%> (ø)`
src/modules/script.cpp	`0.00% <0.00%> (ø)`
src/utils/process.cpp	`21.95% <0.00%> (+0.07%)`	⬆️
src/utils/command.cpp	`52.47% <62.50%> (+14.10%)`	⬆️
include/utils/command.hpp	`66.66% <100.00%> (ø)`
tests/unit_tests/utils/command.cpp	`100.00% <100.00%> (ø)`
... and 5 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 0dffca9...79ea0ce. Read the comment docs.

patrick96

The patch works great. I had just one comment about how the pipes for IGNORED commands are connected that I wanted to discuss below.

Please also rebase so that we can test this against master.

patrick96 · 2020-05-01T14:17:16Z

src/utils/command.cpp

+
+  if (process_util::in_forked_process(m_forkpid)) {
+    setpgid(m_forkpid, 0);
+    process_util::exec_sh(m_cmd.c_str());


Here, the forked process inherits polybar's stdin, stdout, and stderr. So any output will become part of polybar's output. Is this intended?

For one, it makes sense that people can see the output of their script for debugging, but the polybar output may get spammed with command outputs. I'm not sure what the best approach is here.

I think this is the only point that still needs to be addressed.
I have thought about this a bit and I believe processes spawned like this should have their stdout and stderr redirected to /dev/null. If they don't do that, they will forever output in the shell running polybar (even after polybar is closed). This is especially annoying for long-running programs.

I think voiding the output like this is enough:

int fd_null = open("/dev/null", O_WRONLY); dup2(fd_null, STDOUT_FILENO); close(fd_null);

And the same for stderr (with error checks of course).

patrick96 · 2020-05-01T14:31:30Z

tests/unit_tests/utils/command.cpp

+    auto cmd = command_util::make_command<output_policy::IGNORED>("echo polybar > /dev/null");
+    int status = cmd->exec();
+
+    ASSERT_EQ(status, EXIT_SUCCESS);


Please use EXPECT_* everywhere so that we don't see only a part of the test fail. Use ASSERT_* if the rest of the code would otherwise have undefined behavior (for example for NULL checks).

patrick96 · 2020-05-01T15:02:42Z

EDIT: At the moment, users still need to use cmd & to avoid blocking polybar. It may be interesting to add a parameter to the constructor to detach the child.

It may be worth reconsidering the interface we expose for executing commands.
For many use-cases we don't need all the functionality of a command instances. In most cases we want to just execute a command and either forget about it (ignore output, fork into background, don't call wait) or execute it synchronously and get its output and exit code. Here having to create a command object is a bit overkill.

The only place command is really needed is in the script module and there the interface it provides is not wide enough, it still has to manually deal with pipe file descriptors.

I believe that we could do without the command class. All the use cases except the script module could be implemented in two functions in process.cpp and the script module can either do the forking and redirecting pipes itself or we create a well defined interface that covers everything the script module needs.

For spawning commands in the background without blocking polybar, we can take an approach similar to what jgmenu does:

https://github.com/johanmalm/jgmenu/blob/e34a804799a4dcafe4346cd7a9eec213013c5fae/src/spawn.c#L41-L69

Lomadriel · 2020-05-01T19:55:38Z

I agree, the only place where we should use the command object is in indeed in the script module.

Lomadriel · 2020-05-01T19:59:43Z

Rebased on master

patrick96 · 2020-05-06T19:43:02Z

src/utils/command.cpp

+
+  if (process_util::in_forked_process(m_forkpid)) {
+    setpgid(m_forkpid, 0);
+    process_util::exec_sh(m_cmd.c_str());


I think this is the only point that still needs to be addressed.
I have thought about this a bit and I believe processes spawned like this should have their stdout and stderr redirected to /dev/null. If they don't do that, they will forever output in the shell running polybar (even after polybar is closed). This is especially annoying for long-running programs.

I think voiding the output like this is enough:

int fd_null = open("/dev/null", O_WRONLY); dup2(fd_null, STDOUT_FILENO); close(fd_null);

And the same for stderr (with error checks of course).

patrick96 · 2020-05-06T19:44:20Z

I agree, the only place where we should use the command object is in indeed in the script module.

We should maybe plan a refactor after this PR is merged. It will rip out a lot of redundant code and should make running commands a lot cleaner.

patrick96

One final nitpick (I think)

src/utils/process.cpp

patrick96

Awesome. Great work!

Now we can finally resolve this bug 😃

Lomadriel · 2020-05-09T12:30:15Z

🎉

Shell commands triggered from action tags used to block polybar until they finished. Since we are not actually interested in the output of the commands, it makes sense to run them completely detached from polybar and have polybar not block when executing these commands. Now the spawned child processes no longer get killed when polybar exits. This is fine because polybar is not responsible for these processes since they were explicitly started by the user through click commands. Ref: polybar#770 Ref: polybar#1680

Shell commands triggered from action tags used to block polybar until they finished. Since we are not actually interested in the output of the commands, it makes sense to run them completely detached from polybar and have polybar not block when executing these commands. Now the spawned child processes no longer get killed when polybar exits. This is fine because polybar is not responsible for these processes since they were explicitly started by the user through click commands. Ref: #770 Ref: #1680

Breaking Changes: * The new config parser imposes some restrictions on which characters can be used in section and key names. Users shouldn't be affected by this unless they use one of the following characters in any section or key name: `"'=;#[](){}:.$\%` Please consult [`man 5 polybar`](https://polybar.readthedocs.io/en/latest/man/polybar.5.html) for a full reference of the new config rules. * `internal/temperature`: The first and last ramp element are now only used for `base-temperature` and below and `warn-temperature` and above respectively. This only slightly changes the ranges for which the different ramp levels are responsible for. (#2197) Changelog **Deprecations** * `[settings]`: `throttle-input-for` has been removed. It wasn't a useful option and could cause certain actions (e.g. click commands) to be ignored. (#2117) * All action names used by modules to handle click and scroll events are depercated (#1907). This change mainly affects users of the menu module. Please read the [documentation](https://polybar.readthedocs.io/en/latest/user/actions.html) for instructions on how to migrate. **New Config Options** The `include-directory` key can be used the same as `include-file` and includes all regular files in the given directory. In labels: * `label-NAME-minlen`, `label-NAME-alignment` can be used to pad labels with spaces to the right (alignment set to `left`), left (alignment set to `right`), and both sides (alignment set to `center`). In `internal/backlight`: * `enable-scroll` enables changing the brightness through scrolling. In `internal/github`: * `format-offline` is used when the module cannot connect to the server. * `label-offline` can be used in `format-offline`. * `api-url` can be used to connect to a custom github enterprise instance In `internal/pulseaudio`: * `click-right` and `click-middle` can be used to execute commands when right/middle clicking. * `%decibels%` token can be used in `label-volume` and `label-muted` to show the volume in decibels. **Changes To The Build System** * Allow users to specify python executable when building. (polybar/xpp#27, #2125) * The i3ipcpp submodule no longer rebuilds jsoncpp and just uses whatever version is available. (#2015, polybar/i3ipcpp#9) **Features** * New commandline argument: `-M` / `--list-all-monitors`. Will display all available monitors (even cloned ones). * New log level: `notice`. Used as the default and is used for non-warning messages the user should nevertheless be aware of. (#2027) * config: * New config parser (#1377) * `include-directory` key (#2196), see #1946 * Add error message when an entire section is missing. (#2195) * `-minlen` and `-alignment` properties for labels. (#1546) * Make the `seperator` key in the bar section a label. (#1918) * Better color validation. (#1290) * timer modules: Schedule module updates to be aligned with the update interval. For example, the date module now updates on the minute instead of in the middle of a minute if the interval is set to 60 seconds. (#2123), see #2064 * `custom/menu`: Multiple menu modules per bar (#1907) * `internal/backlight`: Support for changing the brightness through scrolling. This may require additional changes to the system running polybar. (#1957) * `internal/github`: * `format-offline` for when the module cannot connect to the server (#1825), see #1815 * Support for github enterprise instances. (#1841), see #1812 * `internal/network`: Support `Gbit/s` for `%linkspeed%` token. (#2055) * `internal/pulseaudio`: * `click-right` and `click-middle` keys (#1941) * `%decibels%` token (#1894), see #1886 * `internal/xworkspaces`: Proper implementation for `label-occupied`. (#822), see #874, #1444, #1033 **Fixes** * Polybar not executing commands that produce output. (#1680), see #916 * Polybar froze until click commands finished executing (#2248) * Polybar not properly working with mirrored monitors. (#1823), see #1192 and #1794 * Unstable animation framerate (#1683), see #568 * Multiple modules of the same type caused click events not to be delivered to the rigth one (#1907), see #1172 * config: * Seemingly unrelated error messages when BOM character appears in config. (#2166), see #2075 * Fall back to next possible config location if config file doesn't exist (except if `--config` was used). (#2026), see #2016 * iconset: `fuzzy-match` chose first match, even if exact match was available. (#2042), see #2041 * `custom/menu`: Spacing issue (#1656) * `internal/alsa`: Volume didn't go over 100% (#2184), see #2173 * `internal/backlight`: Use amdgpu workaround for all devices starting with `amdgpu_bl`. (#2122) * `internal/battery`: Battery not marked as full if over `full-at` percent. (#2019), see #1622 * `internal/cpu`: More accurate cpu load calculation. (#1955) * `internal/github`: Outdated GitHub API authentication. (#2029), see #2002 * `internal/memory`: Use the correct size prefixes (#2211), see #2023 * `internal/network`: * Wrong up- and downspeed values for non-integer intervals. (#2219) * tun/tap interfaces didn't query their IP addresses. (#1990), see #1986 * Don't crash module if linkspeed cannot be queried. (#1772), see #1211 * `internal/temperature`: * `format-warn` was not used if the temperature was *exactly* `warn-temperature`. (#1897) * `internal/xworkspaces`: * Multi-monitor issue (#1929), see #1764, #1849 * Module sometimes showed too many workspaces (#1984), see #1983 * build: * xpp submodule doesn't work with python 3.9 (polybar/xpp#26) * CMake 3.17+ developer warnings (#2089, polybar/xpp#24, polybar/xpp#25) * gtest compilation failure (#1993), see google/googletest#2678 * Compilation issue in GCC 6. (#1953)

Breaking Changes: * The new config parser imposes some restrictions on which characters can be used in section and key names. Users shouldn't be affected by, this unless they use one of the following characters in any section or key name: `"'=;#[](){}:.$\%` Please consult [`man 5 polybar`](https://polybar.readthedocs.io/en/latest/man/polybar.5.html) for a full reference of the new config rules. * `internal/temperature`: The first and last ramp element are now only used for `base-temperature` and below and `warn-temperature` and above respectively. This only slightly changes the ranges for which the different ramp levels are responsible for. (#2197) Changelog **Deprecations** * `[settings]`: `throttle-input-for` has been removed. It wasn't a useful option and could cause certain actions (e.g. click commands) to be ignored. (#2117) * All action names used by modules to handle click and scroll events are deprecated (#1907). This change mainly affects users of the menu module. Please read the [documentation](https://polybar.readthedocs.io/en/latest/user/actions.html) for instructions on how to migrate. **New Config Options** The `include-directory` key can be used the same as `include-file` and includes all regular files in the given directory. In labels: * `label-NAME-minlen`, `label-NAME-alignment` can be used to pad labels with spaces to the right (alignment set to `left`), left (alignment set to `right`), and both sides (alignment set to `center`). In `internal/backlight`: * `enable-scroll` enables changing the brightness through scrolling. In `internal/github`: * `format-offline` is used when the module cannot connect to the server. * `label-offline` can be used in `format-offline`. * `api-url` can be used to connect to a custom github enterprise instance In `internal/pulseaudio`: * `click-right` and `click-middle` can be used to execute commands when right/middle clicking. * `%decibels%` token can be used in `label-volume` and `label-muted` to show the volume in decibels. **Changes To The Build System** * Allow users to specify python executable when building. (polybar/xpp#27, #2125) * The i3ipcpp submodule no longer rebuilds jsoncpp and just uses whatever version is available. (#2015, polybar/i3ipcpp#9) **Features** * New commandline argument: `-M` / `--list-all-monitors`. Will display all available monitors (even cloned ones). * New log level: `notice`. Used as the default and is used for non-warning messages the user should nevertheless be aware of. (#2027) * config: * New config parser (#1377) * `include-directory` key (#2196), see #1946 * Add error message when an entire section is missing. (#2195) * `-minlen` and `-alignment` properties for labels. (#1546) * Make the `seperator` key in the bar section a label. (#1918) * Better color validation. (#1290) * timer modules: Schedule module updates to be aligned with the update interval. For example, the date module now updates on the minute instead of in the middle of a minute if the interval is set to 60 seconds. (#2123), see #2064 * `custom/menu`: Multiple menu modules per bar (#1907) * `internal/backlight`: Support for changing the brightness through scrolling. This may require additional changes to the system running polybar. (#1957) * `internal/github`: * `format-offline` for when the module cannot connect to the server (#1825), see #1815 * Support for github enterprise instances. (#1841), see #1812 * `internal/network`: Support `Gbit/s` for `%linkspeed%` token. (#2055) * `internal/pulseaudio`: * `click-right` and `click-middle` keys (#1941) * `%decibels%` token (#1894), see #1886 * `internal/xworkspaces`: Proper implementation for `label-occupied`. (#822), see #874, #1444, #1033 **Fixes** * Polybar not executing commands that produce output. (#1680), see #916 * Polybar froze until click commands finished executing (#2248) * Polybar not properly working with mirrored monitors. (#1823), see #1192 and #1794 * Unstable animation framerate (#1683), see #568 * Multiple modules of the same type caused click events not to be delivered to the rigth one (#1907), see #1172 * config: * Seemingly unrelated error messages when BOM character appears in config. (#2166), see #2075 * Fall back to next possible config location if config file doesn't exist (except if `--config` was used). (#2026), see #2016 * iconset: `fuzzy-match` chose first match, even if exact match was available. (#2042), see #2041 * `custom/menu`: Spacing issue (#1656) * `internal/alsa`: Volume didn't go over 100% (#2184), see #2173 * `internal/backlight`: Use amdgpu workaround for all devices starting with `amdgpu_bl`. (#2122) * `internal/battery`: Battery not marked as full if over `full-at` percent. (#2019), see #1622 * `internal/cpu`: More accurate cpu load calculation. (#1955) * `internal/github`: Outdated GitHub API authentication. (#2029), see #2002 * `internal/memory`: Use the correct size prefixes (#2211), see #2023 * `internal/network`: * Wrong up- and downspeed values for non-integer intervals. (#2219) * tun/tap interfaces didn't query their IP addresses. (#1990), see #1986 * Don't crash module if linkspeed cannot be queried. (#1772), see #1211 * `internal/temperature`: * `format-warn` was not used if the temperature was *exactly* `warn-temperature`. (#1897) * `internal/xworkspaces`: * Multi-monitor issue (#1929), see #1764, #1849 * Module sometimes showed too many workspaces (#1984), see #1983 * build: * xpp submodule doesn't work with python 3.9 (polybar/xpp#26) * CMake 3.17+ developer warnings (#2089, polybar/xpp#24, polybar/xpp#25) * gtest compilation failure (#1993), see google/googletest#2678 * Compilation issue in GCC 6. (#1953)

Lomadriel changed the title ~~fix(command): broken pipe when ignoring output.~~ Fix broken pipe in command Mar 7, 2019

Lomadriel mentioned this pull request Mar 19, 2019

xkill won't launch from click-commands or menu-x-x-exec commands. #1702

Closed

patrick96 mentioned this pull request Mar 29, 2019

custom/text module cannot click to execute 'chromium-browser &' #1717

Closed

patrick96 added this to the 3.5.0 milestone Jun 28, 2019

Lomadriel added the bug label Jul 27, 2019

patrick96 requested changes May 1, 2020

View reviewed changes

fix(command): broken pipe when ignoring output.

4c639ca

Lomadriel force-pushed the fix/command_output branch from 3191728 to 5f1493b Compare May 1, 2020 19:59

add command unit tests

f7a9410

Lomadriel force-pushed the fix/command_output branch from 5f1493b to f7a9410 Compare May 1, 2020 20:15

patrick96 requested changes May 6, 2020

View reviewed changes

Lomadriel force-pushed the fix/command_output branch from 41511d6 to 636c212 Compare May 6, 2020 20:30

patrick96 requested changes May 6, 2020

View reviewed changes

src/utils/process.cpp Outdated Show resolved Hide resolved

Redirect process output to dev null

79ea0ce

Lomadriel force-pushed the fix/command_output branch from 636c212 to 79ea0ce Compare May 8, 2020 11:11

patrick96 approved these changes May 8, 2020

View reviewed changes

patrick96 merged commit f016b99 into polybar:master May 8, 2020

patrick96 mentioned this pull request Nov 25, 2020

controller: Detach shell commands from polybar #2248

Merged

patrick96 mentioned this pull request Nov 29, 2020

Release 3.5.0 #2253

Merged

28 tasks

Uh oh!

Fix broken pipe in command #1680

Fix broken pipe in command #1680

Uh oh!

Conversation

Lomadriel commented Mar 7, 2019 • edited by patrick96 Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

codecov-io commented Mar 7, 2019 • edited by codecov bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

patrick96 left a comment

Choose a reason for hiding this comment

Uh oh!

patrick96 May 1, 2020

Choose a reason for hiding this comment

Uh oh!

patrick96 May 6, 2020

Choose a reason for hiding this comment

Uh oh!

patrick96 May 1, 2020

Choose a reason for hiding this comment

Uh oh!

patrick96 commented May 1, 2020

Uh oh!

Lomadriel commented May 1, 2020

Uh oh!

Lomadriel commented May 1, 2020

Uh oh!

patrick96 May 6, 2020

Choose a reason for hiding this comment

Uh oh!

patrick96 commented May 6, 2020

Uh oh!

patrick96 left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

patrick96 left a comment

Choose a reason for hiding this comment

Uh oh!

Lomadriel commented May 9, 2020

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Lomadriel commented Mar 7, 2019 •

edited by patrick96

Loading

codecov-io commented Mar 7, 2019 •

edited by codecov bot

Loading