Skip to content

Conversation

@patrick96
Copy link
Member

@patrick96 patrick96 commented Aug 12, 2018

This is the next step to merge #1237 in stages.

Currently there are barely any restrictions on how the config can be
written. This causes things like config files with DOS line endings to
not be parsed properly (#1366) because polybar splits by \n and when
parsing section headers, it can't deal with the \r at the end of the
line and thus doesn't recognize any section headers.

With this PR we introduce some rules as to what characters are allowed
in section names and keys.
Note: When talking about spaces I refer to any character for which
isspace() returns true.

The rules are as follows:

  • A section name or a key name cannot contain any spaces as well as any
    of there characters:"'=;#[](){}:.$\%
  • Spaces at the beginning and end of lines are always ignored when
    parsing
  • Comment lines start with ; or # and last for the whole line. The
    whole line will be ignored by the parser. You cannot start a comment at
    the end of a line.
  • Section headers have the following form [HEADER_NAME]
  • Key-value lines look like this:
    KEY_NAME{SPACES}={SPACES}VALUE_STRING where {SPACES} represents any
    number of spaces. VALUE_STRING can contain any characters. If it is
    surrounded with double quotes ("), those quotes will be removed,
    this can be used to add spaces to the beginning or end of the value
  • Empty lines are lines with only spaces in them
  • If the line has any other form, it is a syntax error

This will introduce the following breaking changes because of how
underdefined the config syntax was before:

  • key = "" will get treated as an empty string instead of the literal string ""
  • Any section or key name with forbidden characters will now be syntax
    errors.
  • Certain strings will be forbidden as section names: self, root, BAR. Because they have a special meaning inside references and so a section [root] can never be referenced.

This replaces the current parser implementation with a new more robust
one that will later be expanded to also check for dependency cycles and allow for values that contain references mixed with other strings.

This PR also now expands the config paths given over the command line so that --config=~/.config/polybar/config resolves properly.

EDIT:
Closes #1032
Closes #1694

@patrick96 patrick96 mentioned this pull request Aug 12, 2018
19 tasks
@patrick96 patrick96 added this to the 3.3.0 milestone Aug 12, 2018
@patrick96 patrick96 changed the title [WIP] config_parser: Introduce stricter syntax convention config_parser: Introduce stricter syntax convention Aug 13, 2018
@patrick96 patrick96 force-pushed the feature/strict-config branch from b45e8da to 28bea57 Compare August 13, 2018 16:09
@patrick96
Copy link
Member Author

patrick96 commented Aug 13, 2018

I think this is ready for first review now. I know this is quite a big PR, so we should take our time reviewing it. I think I have already outlined the motivation behind this and what this PR does quite well, please say, if I need to provide any more info.

With my configs, I see no difference between using this PR and master when displaying the bar.
I will myself also test the following things:

  • Does the parser keep the same behaviour regarding multiple inheritance described in Nested inherit no longer works? #412. This is just for consistency that the behaviour stay broken in the same way as before
  • Do ${xrdb references work properly
  • Does include-file correctly detect circular dependencies

@patrick96 patrick96 requested a review from NBonaparte August 13, 2018 16:15
@patrick96 patrick96 changed the title config_parser: Introduce stricter syntax convention config_parser: Introduce stricter syntax conventions Aug 13, 2018
Copy link
Member

@NBonaparte NBonaparte left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Did a cursory review of the code to get trivial stuff out of the way, will do a more in-depth review soon.

Should we have a style guide of some sort to prevent inconsistencies?

@patrick96
Copy link
Member Author

According to the .clang-format file we should already be using the google style guide. I have created a style guide page on the wiki where we can write down the style guidelines we come up with.

I have now gone over the whole PR again and mainly improved the comments as well as fixed the ${xrdb reference detection.

I have tested the three things I listed above and now they all work as expected.

@patrick96
Copy link
Member Author

patrick96 commented Oct 5, 2018

Before we merge this there are two more things we need to confirm didn't change in this PR:

Does inherit support multiple inheritance?

It doesn't in this PR because key names have to be unique. If it did in the current version it would have weird behaviour because polybar doesn't preserve the order of the keys and if there are conflicts they would be resolved differently depending on the order.

In any case it would be a nice to have. Maybe let inherit be a space separated list of section names and polybar resolves them one after the other, if there is a key conflict the key of the first inherited section is used because of how inheritance works. This would be something to be implemented later though, not in this PR and I can already foresee problems because when doing the cyclic dependency check, the graph edges don't have any order on them, but we'll deal with that once we get to it
EDIT: The order of the graph edges shouldn't be a problem since the purpose of the cyclic dependency check for inherit is totally unrelated to and is done before resolving ${...} references.

Does include-file support ${...} references?

It doesn't in this PR because it is not really treated as a key but more like an #include directive. Allowing it would introduce some really weird semantics and hugely complicated code.
EDIT: It is probably possible to support external references (xrdb, env, file) just not references inside the config. Also the current plan is to separate external and internal references, so we could have some code reuse here.

@patrick96 patrick96 modified the milestones: 3.3.0, 3.4.0 Oct 13, 2018
@NBonaparte NBonaparte self-requested a review November 1, 2018 00:18
@NBonaparte
Copy link
Member

@patrick96 This still has to be reviewed, correct? I forgot about this because it wasn't assigned to me.

@patrick96
Copy link
Member Author

Yeah this needs to be reviewed. I forgot to request a review again.

I will also need to check the two things mentioned above, though there won't probably be any surprises there.

patrick96 added a commit to patrick96/polybar that referenced this pull request Nov 4, 2018
NBonaparte pushed a commit that referenced this pull request Nov 5, 2018
@patrick96 patrick96 force-pushed the feature/strict-config branch from 7f377bd to 81dd405 Compare November 17, 2018 09:53
patrick96 added a commit to patrick96/polybar that referenced this pull request Nov 17, 2018
@patrick96 patrick96 force-pushed the feature/strict-config branch 2 times, most recently from 659e6e5 to caa363a Compare November 17, 2018 10:19
@patrick96
Copy link
Member Author

My findings:

Does inherit support multiple inheritance?
No. It doesn't in both implementations, it always throws an error. This was to be expected since both implementations first parse the config into a section-map before doing inheritance.

Does include-file support ${...} references?
No. Both implementations use file_util::expand only.

Does inherit support ${...} references?
Yes. Both implementation use the same code for doing inheritance: copy_inherited.

I'm pretty sure that I have now checked that the functionality didn't change with this PR (except for the syntax rules).

Feel free to criticize. My C++ is still not that good.

@patrick96
Copy link
Member Author

Merge conflicts are now fixed :)

First step in the config_parser develoment. Only tests functions that
are easily testable without many outside dependencies. Integration tests
will follow.
@patrick96
Copy link
Member Author

Thanks 😊

If you want me to split this up into multiple smaller PRs so that we can review smaller chunks, let me know.

Copy link
Member

@NBonaparte NBonaparte left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Overall functionality looks OK.

All the string arguments for the config_parser functions should be const string& instead of string since they're not being modified.

config::make_type result = config::make(m_file, m_barname);

// Cast to non-const to set sections, included and xrm
config& m_conf = const_cast<config&>(result);
Copy link
Member

@NBonaparte NBonaparte Mar 25, 2019

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We keep getting away with this, we should probably fix this some time

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, terrible practice. Also in the cpu module 😄

I think I will change that in future PRs and make config something that is constructed just like a normal struct and after that is const in the entire program

Copy link
Member

@Lomadriel Lomadriel Aug 6, 2019

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I have a suggestion for this.
How we have suppressed this in the cpu module.
Maybe we should make config::make private and modifies it to return a non-const ref and add friend class config_parser into the config class then we add a config::get or something like that which returns the singleton (const& version). What do you think of that?

In case we want to remove the singleton, we could also store the config into bar and so we can access it from bar into the modules.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think removing the singleton would be good. The singleton provides two things: it restricts a the class to a single instance and it gives easy access to it. We only need the easy access since technically there could be multiple config instances. So moving the config into an object the code already has access to (bar_settings or similar) is a good idea.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I can do that once your PR is merged if you want.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

On second though, it may be beneficial to just pass the config to wherever it's needed since it's only really needed in the modules, controller.cpp, renderer.cpp, screen.cpp, and tray_manager.cpp. Though we could probably remove it from screen.cpp since it only needs access to the screenchange-reload setting.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah that's also a possibility.

@patrick96 patrick96 force-pushed the feature/strict-config branch from c42ac8c to 047c57c Compare March 27, 2019 14:47
@patrick96 patrick96 requested a review from NBonaparte March 28, 2019 22:24
Copy link
Member

@Lomadriel Lomadriel left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This new parser seems really nice.
My remarks are more about the code quality.

Misc remarks:

  1. Maybe string filepath() const; and string section() const; should return a const ref to let the caller decide if a copy is needed.
  2. There are some missing m_ for some attributes in the config_parser class.

Some of my comments might be excessive optimization but it may improve performance/code readability/code safety so it could be worth it! (and I like code :P, sorry :( )

@patrick96
Copy link
Member Author

Thanks! I'll try to address the issues. Keep 'em coming, you're definitely more experienced with C++ especially when it comes to move semantivs :)

@patrick96 patrick96 modified the milestones: 3.4.0, 3.5.0 Jun 28, 2019
@patrick96 patrick96 force-pushed the feature/strict-config branch from 047c57c to 59ba3e6 Compare July 5, 2019 16:16
@patrick96
Copy link
Member Author

Maybe string filepath() const; and string section() const; should return a const ref to let the caller decide if a copy is needed.

section actually constructs a string so we can't return a reference since it would be on the stack.

Thanks a lot for the review. Once this is merged I think I'll try to refactor it again a little to more decouple things.

Copy link
Member

@Lomadriel Lomadriel left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

section actually constructs a string so we can't return a reference since it would be on the stack.

Oops didn't see that. sry

Other things that can't be include in the review:

  • utils/file.hpp (line 9) & utils/math.hpp (line 10) aren't used in config.cpp
  • cstring (line 2) isn't used in string.cpp

I missed things in my first review but we are pretty close to the end :D.

EDIT: I don't know why my review has been separated in two reviews 😞 but please also take a look at the review above.

EDIT 2: You new code doesn't seems to respect the clang-tidy maybe you should run it on your files ;)
I don't know which IDE/code editor you are using but maybe you should install a plugin to automatically run clang-format on the modified code.

@patrick96 patrick96 force-pushed the feature/strict-config branch from 8c52e63 to 198883e Compare August 5, 2019 22:28
@codecov codecov bot deleted a comment from codecov-io Aug 5, 2019
@patrick96
Copy link
Member Author

Done! I ran all changed files through clang-format and it changed all the indentation in config_parser.hpp but that's not too bad.

The only thing missing seems to be your suggestion about using a template parameter instead of std::function.

@patrick96 patrick96 requested a review from Lomadriel August 5, 2019 22:38
@Lomadriel
Copy link
Member

Lomadriel commented Aug 6, 2019

Except for my suggestion about using a template parameter everything LGTM.
Great job :)

EDIT: I also added a suggestion for removing the const_cast but it could be modified in a future PR.

Copy link
Member

@Lomadriel Lomadriel left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@patrick96 patrick96 force-pushed the feature/strict-config branch from 732dc6c to 198883e Compare August 6, 2019 17:24
@patrick96 patrick96 merged commit 56e2499 into polybar:master Aug 6, 2019
@patrick96 patrick96 deleted the feature/strict-config branch August 6, 2019 17:41
patrick96 added a commit that referenced this pull request Nov 29, 2020
Breaking Changes:

* The new config parser imposes some restrictions on which characters can be
  used in section and key names. Users shouldn't be affected by this unless they
  use one of the following characters in any section or key name: `"'=;#[](){}:.$\%`
  Please consult [`man 5 polybar`](https://polybar.readthedocs.io/en/latest/man/polybar.5.html) for a full reference of the new config rules.
* `internal/temperature`: The first and last ramp element are now only used for
  `base-temperature` and below and `warn-temperature` and above respectively.
  This only slightly changes the ranges for which the different ramp levels are
  responsible for.
  (#2197)

Changelog

**Deprecations**

* `[settings]`: `throttle-input-for` has been removed. It wasn't a useful option
  and could cause certain actions (e.g. click commands) to be ignored. (#2117)

* All action names used by modules to handle click and scroll events are
  depercated (#1907).  This change mainly affects users of the menu module.
  Please read the [documentation](https://polybar.readthedocs.io/en/latest/user/actions.html) for instructions on how to migrate.

**New Config Options**

The `include-directory` key can be used the same as `include-file` and includes
all regular files in the given directory.

In labels:

* `label-NAME-minlen`, `label-NAME-alignment` can be used to pad labels with
  spaces to the right (alignment set to `left`), left (alignment set to
  `right`), and both sides (alignment set to `center`).

In `internal/backlight`:

* `enable-scroll` enables changing the brightness through scrolling.

In `internal/github`:

* `format-offline` is used when the module cannot connect to the server.
* `label-offline` can be used in `format-offline`.
* `api-url` can be used to connect to a custom github enterprise instance

In `internal/pulseaudio`:

* `click-right` and `click-middle` can be used to execute commands when
  right/middle clicking.
* `%decibels%` token can be used in `label-volume` and `label-muted` to show the
  volume in decibels.

**Changes To The Build System**

* Allow users to specify python executable when building. (polybar/xpp#27,
  #2125)
* The i3ipcpp submodule no longer rebuilds jsoncpp and just uses whatever
  version is available. (#2015, polybar/i3ipcpp#9)

**Features**

* New commandline argument: `-M` / `--list-all-monitors`.
    Will display all available monitors (even cloned ones).
* New log level: `notice`.
    Used as the default and is used for non-warning messages the user should
    nevertheless be aware of. (#2027)
* config:
    * New config parser (#1377)
    * `include-directory` key (#2196), see #1946
    * Add error message when an entire section is missing. (#2195)
    * `-minlen` and `-alignment` properties for labels. (#1546)
    * Make the `seperator` key in the bar section a label. (#1918)
    * Better color validation. (#1290)
* timer modules: Schedule module updates to be aligned with the update interval.
  For example, the date module now updates on the minute instead of in the
  middle of a minute if the interval is set to 60 seconds. (#2123), see #2064
* `custom/menu`: Multiple menu modules per bar (#1907)
* `internal/backlight`: Support for changing the brightness through scrolling.
  This may require additional changes to the system running polybar. (#1957)
* `internal/github`:
    * `format-offline` for when the module cannot connect to the server (#1825),
    see #1815
    * Support for github enterprise instances. (#1841), see #1812
* `internal/network`: Support `Gbit/s` for `%linkspeed%` token. (#2055)
* `internal/pulseaudio`:
    * `click-right` and `click-middle` keys (#1941)
    * `%decibels%` token (#1894), see #1886
* `internal/xworkspaces`: Proper implementation for `label-occupied`. (#822),
  see #874, #1444, #1033

**Fixes**

* Polybar not executing commands that produce output. (#1680), see #916
* Polybar froze until click commands finished executing (#2248)
* Polybar not properly working with mirrored monitors. (#1823), see #1192 and
  #1794
* Unstable animation framerate (#1683), see #568
* Multiple modules of the same type caused click events not to be delivered to
  the rigth one (#1907), see #1172
* config:
    * Seemingly unrelated error messages when BOM character appears in config.
    (#2166), see #2075
    * Fall back to next possible config location if config file doesn't exist
      (except if `--config` was used). (#2026), see #2016
* iconset: `fuzzy-match` chose first match, even if exact match was available.
  (#2042), see #2041
* `custom/menu`: Spacing issue (#1656)
* `internal/alsa`: Volume didn't go over 100% (#2184), see #2173
* `internal/backlight`: Use amdgpu workaround for all devices starting with
  `amdgpu_bl`. (#2122)
* `internal/battery`: Battery not marked as full if over `full-at` percent.
  (#2019), see #1622
* `internal/cpu`: More accurate cpu load calculation. (#1955)
* `internal/github`: Outdated GitHub API authentication. (#2029), see #2002
* `internal/memory`: Use the correct size prefixes (#2211), see #2023
* `internal/network`:
    * Wrong up- and downspeed values for non-integer intervals. (#2219)
    * tun/tap interfaces didn't query their IP addresses. (#1990), see #1986
    * Don't crash module if linkspeed cannot be queried. (#1772), see #1211
* `internal/temperature`:
    * `format-warn` was not used if the temperature was *exactly*
      `warn-temperature`. (#1897)
* `internal/xworkspaces`:
    * Multi-monitor issue (#1929), see #1764, #1849
    * Module sometimes showed too many workspaces (#1984), see #1983
* build:
    * xpp submodule doesn't work with python 3.9 (polybar/xpp#26)
    * CMake 3.17+ developer warnings (#2089, polybar/xpp#24, polybar/xpp#25)
    * gtest compilation failure (#1993), see google/googletest#2678
    * Compilation issue in GCC 6. (#1953)
@patrick96 patrick96 mentioned this pull request Nov 29, 2020
28 tasks
patrick96 added a commit that referenced this pull request Dec 1, 2020
Breaking Changes:

* The new config parser imposes some restrictions on which characters can be
  used in section and key names. Users shouldn't be affected by, this unless
  they use one of the following characters in any section or key name: `"'=;#[](){}:.$\%`
  Please consult [`man 5 polybar`](https://polybar.readthedocs.io/en/latest/man/polybar.5.html) for a full reference of the new config rules.
* `internal/temperature`: The first and last ramp element are now only used for
  `base-temperature` and below and `warn-temperature` and above respectively.
  This only slightly changes the ranges for which the different ramp levels are
  responsible for.
  (#2197)

Changelog

**Deprecations**

* `[settings]`: `throttle-input-for` has been removed. It wasn't a useful option
  and could cause certain actions (e.g. click commands) to be ignored. (#2117)

* All action names used by modules to handle click and scroll events are
  deprecated (#1907). This change mainly affects users of the menu module.
  Please read the [documentation](https://polybar.readthedocs.io/en/latest/user/actions.html) for instructions on how to migrate.

**New Config Options**

The `include-directory` key can be used the same as `include-file` and includes
all regular files in the given directory.

In labels:

* `label-NAME-minlen`, `label-NAME-alignment` can be used to pad labels with
  spaces to the right (alignment set to `left`), left (alignment set to
  `right`), and both sides (alignment set to `center`).

In `internal/backlight`:

* `enable-scroll` enables changing the brightness through scrolling.

In `internal/github`:

* `format-offline` is used when the module cannot connect to the server.
* `label-offline` can be used in `format-offline`.
* `api-url` can be used to connect to a custom github enterprise instance

In `internal/pulseaudio`:

* `click-right` and `click-middle` can be used to execute commands when
  right/middle clicking.
* `%decibels%` token can be used in `label-volume` and `label-muted` to show the
  volume in decibels.

**Changes To The Build System**

* Allow users to specify python executable when building. (polybar/xpp#27,
  #2125)
* The i3ipcpp submodule no longer rebuilds jsoncpp and just uses whatever
  version is available. (#2015, polybar/i3ipcpp#9)

**Features**

* New commandline argument: `-M` / `--list-all-monitors`.
    Will display all available monitors (even cloned ones).
* New log level: `notice`.
    Used as the default and is used for non-warning messages the user should
    nevertheless be aware of. (#2027)
* config:
    * New config parser (#1377)
    * `include-directory` key (#2196), see #1946
    * Add error message when an entire section is missing. (#2195)
    * `-minlen` and `-alignment` properties for labels. (#1546)
    * Make the `seperator` key in the bar section a label. (#1918)
    * Better color validation. (#1290)
* timer modules: Schedule module updates to be aligned with the update interval.
  For example, the date module now updates on the minute instead of in the
  middle of a minute if the interval is set to 60 seconds. (#2123), see #2064
* `custom/menu`: Multiple menu modules per bar (#1907)
* `internal/backlight`: Support for changing the brightness through scrolling.
  This may require additional changes to the system running polybar. (#1957)
* `internal/github`:
    * `format-offline` for when the module cannot connect to the server (#1825),
    see #1815
    * Support for github enterprise instances. (#1841), see #1812
* `internal/network`: Support `Gbit/s` for `%linkspeed%` token. (#2055)
* `internal/pulseaudio`:
    * `click-right` and `click-middle` keys (#1941)
    * `%decibels%` token (#1894), see #1886
* `internal/xworkspaces`: Proper implementation for `label-occupied`. (#822),
  see #874, #1444, #1033

**Fixes**

* Polybar not executing commands that produce output. (#1680), see #916
* Polybar froze until click commands finished executing (#2248)
* Polybar not properly working with mirrored monitors. (#1823), see #1192 and
  #1794
* Unstable animation framerate (#1683), see #568
* Multiple modules of the same type caused click events not to be delivered to
  the rigth one (#1907), see #1172
* config:
    * Seemingly unrelated error messages when BOM character appears in config.
    (#2166), see #2075
    * Fall back to next possible config location if config file doesn't exist
      (except if `--config` was used). (#2026), see #2016
* iconset: `fuzzy-match` chose first match, even if exact match was available.
  (#2042), see #2041
* `custom/menu`: Spacing issue (#1656)
* `internal/alsa`: Volume didn't go over 100% (#2184), see #2173
* `internal/backlight`: Use amdgpu workaround for all devices starting with
  `amdgpu_bl`. (#2122)
* `internal/battery`: Battery not marked as full if over `full-at` percent.
  (#2019), see #1622
* `internal/cpu`: More accurate cpu load calculation. (#1955)
* `internal/github`: Outdated GitHub API authentication. (#2029), see #2002
* `internal/memory`: Use the correct size prefixes (#2211), see #2023
* `internal/network`:
    * Wrong up- and downspeed values for non-integer intervals. (#2219)
    * tun/tap interfaces didn't query their IP addresses. (#1990), see #1986
    * Don't crash module if linkspeed cannot be queried. (#1772), see #1211
* `internal/temperature`:
    * `format-warn` was not used if the temperature was *exactly*
      `warn-temperature`. (#1897)
* `internal/xworkspaces`:
    * Multi-monitor issue (#1929), see #1764, #1849
    * Module sometimes showed too many workspaces (#1984), see #1983
* build:
    * xpp submodule doesn't work with python 3.9 (polybar/xpp#26)
    * CMake 3.17+ developer warnings (#2089, polybar/xpp#24, polybar/xpp#25)
    * gtest compilation failure (#1993), see google/googletest#2678
    * Compilation issue in GCC 6. (#1953)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Remove the error in which polybar won't run if modules have a space after the bracket

3 participants