Skip to content

pgwire/hba: fix the HBA parser#43779

Merged
craig[bot] merged 1 commit intocockroachdb:masterfrom
knz:20200107-fix-hba-parser
Jan 7, 2020
Merged

pgwire/hba: fix the HBA parser#43779
craig[bot] merged 1 commit intocockroachdb:masterfrom
knz:20200107-fix-hba-parser

Conversation

@knz
Copy link
Copy Markdown
Contributor

@knz knz commented Jan 7, 2020

Previously, the HBA rule parser would not properly support the pg
split format for separate address and netmask.

When the 4th column of a rule in
the setting server.host_based_authentication.configuration is
an IP address, the 5th column is expected to be an IP netmask.

Prior to this patch, the HBA parser would not recognize the netmask
syntax and instead expect a simple number of bits set (which is not
valid pg syntax).

This patch fixes this and adds the corresponding tests.

Additionally, this patch replaces the HBA rule parser by one inspired
from pg's own implementation, so as to ease future maintainance.

Release note (bug fix): When the 4th column of a rule in the setting
server.host_based_authentication.configuration is an IP address
without a mask length (e.g. 1.2.3.4 instead of 1.2.0.0/16),
CockroachDB now properly interprets the 5th column as an IP netmask,
as per https://www.postgresql.org/docs/current/auth-pg-hba-conf.html.

@knz knz requested a review from madelynnblue January 7, 2020 15:05
@cockroach-teamcity
Copy link
Copy Markdown
Member

This change is Reviewable

@knz knz force-pushed the 20200107-fix-hba-parser branch 9 times, most recently from 4f73ef2 to 0727897 Compare January 7, 2020 17:15
@knz knz force-pushed the 20200107-fix-hba-parser branch from 0727897 to 33b6119 Compare January 7, 2020 18:31
}{
{r: rule{`[ \t\r,]*` /*******/, func(l *lex) (bool, error) { return false, nil }}},
{r: rule{`#.*$` /************/, func(l *lex) (bool, error) { return false, nil }}},
{r: rule{`[^", \t\r,]+,?` /**/, func(l *lex) (bool, error) { l.checkComma(); l.Value = l.lexed; return true, nil }}},
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

comma appears twice here

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

wow nicely spotted. Fixed.

Previously, the HBA rule parser would not properly support the pg
split format for separate address and netmask.

When the 4th column of a rule in
the setting `server.host_based_authentication.configuration` is
an IP address, the 5th column is expected to be an IP netmask.

Prior to this patch, the HBA parser would not recognize the netmask
syntax and instead expect a simple number of bits set (which is not
valid pg syntax).

This patch fixes this and adds the corresponding tests.

Additionally, this patch replaces the HBA rule parser by one inspired
from pg's own implementation, so as to ease future maintainance.

Release note (bug fix): When the 4th column of a rule in the setting
`server.host_based_authentication.configuration` is an IP address
without a mask length (e.g. `1.2.3.4` instead of `1.2.0.0/16`),
CockroachDB now properly interprets the 5th column as an IP netmask,
as per https://www.postgresql.org/docs/current/auth-pg-hba-conf.html.
@knz knz force-pushed the 20200107-fix-hba-parser branch from 33b6119 to 87e150d Compare January 7, 2020 21:49
@knz
Copy link
Copy Markdown
Contributor Author

knz commented Jan 7, 2020

TFYR!

bors r+

craig bot pushed a commit that referenced this pull request Jan 7, 2020
43764: storage: change command registration to declare if ro/rw r=irfansharif a=irfansharif

This PR reworks parts of #26916. It's not strictly necessary, but is a
quality of life improvement. There are also miscellaneous cleanups
around TeeEngine, RocksDB benchmarks, etc.

Previous to this we relied on engine.NewReadOnly's runtime panics when
receiving writes to ensure that read-only commands do not accidentally
write data. By changing the command registration process to have
commands declare upfront whether they are read-only or read-write, we
can plumb in engine.Reader or engine.ReadWriter respectively. It's nice
rely on the type system to assert these guarantees.

NB: There's still the unaddressed wart that is replica read code paths
operating off an engine.ReadWriter. Changing this out would require a
refactor of replica evaluation code paths, which I'm not doing now. This
PR however paves the way for that.

Release note: None


43779: pgwire/hba: fix the HBA parser r=knz a=knz

Previously, the HBA rule parser would not properly support the pg
split format for separate address and netmask.

When the 4th column of a rule in
the setting `server.host_based_authentication.configuration` is
an IP address, the 5th column is expected to be an IP netmask.

Prior to this patch, the HBA parser would not recognize the netmask
syntax and instead expect a simple number of bits set (which is not
valid pg syntax).

This patch fixes this and adds the corresponding tests.

Additionally, this patch replaces the HBA rule parser by one inspired
from pg's own implementation, so as to ease future maintainance.

Release note (bug fix): When the 4th column of a rule in the setting
`server.host_based_authentication.configuration` is an IP address
without a mask length (e.g. `1.2.3.4` instead of `1.2.0.0/16`),
CockroachDB now properly interprets the 5th column as an IP netmask,
as per https://www.postgresql.org/docs/current/auth-pg-hba-conf.html.

Co-authored-by: irfan sharif <irfanmahmoudsharif@gmail.com>
Co-authored-by: Raphael 'kena' Poss <knz@thaumogen.net>
@craig
Copy link
Copy Markdown
Contributor

craig bot commented Jan 7, 2020

Build succeeded

@craig craig bot merged commit 87e150d into cockroachdb:master Jan 7, 2020
@knz knz deleted the 20200107-fix-hba-parser branch January 7, 2020 22:23
craig bot pushed a commit that referenced this pull request Jan 7, 2020
43726: pgwire: clarify the special cases as pre-defined HBA rules r=knz a=knz

First commit from #43779.
This PR is needed ahead of fixing #31113, which will require supporting `local` HBA rules.

Prior to this patch, the authentication code was relatively complex to
understand:

- the code path to use the HBA configuration would only be activated
  if the cluster setting was set. There was a special case if it
  wasn't. The special case was in the per-conn auth logic.
- the special case for the root login escape was also present in the
  per-conn auth logic.
- every time the cluster setting would be updated, the result of
  parsing the setting was cached as-is, and the textual strings
  re-interpreted over and over upon every connection.

This complexity can be interpreted as a defect, for the main reason
that **the per-conn auth code should be as simple as possible** to
facilitate security audits and future maintainance. Additionally, an
argument could be made that the per-conn re-interpretation of the
result was a performance mishap.

This patch aims to simplify the per-conn auth logic by folding all the
special casing as pre-defined rules in the HBA configuration:

- the default logic (when the cluster setting is empty or invalid)
  becomes a predefined HBA configuration with just two rules:

         host all root all cert
         host all all  all cert-password

- each time the config is loaded from a cluster setting, the
  root escape is implemented by force-inserting `host all root all
  cert` at the start of the configuration.

With this in place, the auth logic can be simplified to always
and exclusively use the HBA rules.

This special casing can also be inspected in the output of
`/debug/hba_conf`.

Additionally, this patch optimizes the code by pre-normalizing upfront
when the setting is updated. Normalizing includes:

- unicode-normalizing and case-folding usernames, since the username
  upon new connection is also normalized and case-folded.
- expanding lists of multiple usernames into multiple rules, so
  that the checking code can be simplified to only check one username
  per rule.

Release note (security): The authentication code for new SQL
connections has been simplified to always use the HBA configuration
defined per `server.host_based_authentication.configuration`.

The format of this file generally follows that of `pg_hba.conf` as
defined here:
https://www.postgresql.org/docs/current/auth-pg-hba-conf.html.

Upon each configuration change, CockroachDB auto-magically inserts the
entry `host all root all cert` as a first rule, to ensure the root
user can always log in with a valid client certificate.

If the configuration is set to empty, or found to be invalid in the
cluster setting, the following default configuration is automatically
used:

         host all root all cert
         host all all  all cert-password

At any moment the current configuration on each node can be inspected
using the `/debug/hba_conf` URL on the HTTP endpoint.

The list of valid authentication methods is currently:

- `cert`, for certificate-based authentication over a SSL connection
  exclusively;
- `cert-password`, which allows either cert-based or password-based
  authentication over a SSL connection;
- `password` for password-based authentication over a SSL connection;
- `gss` for Kerberos-based authentication over a SSL connection,
  enabled when running a CCL binary and an Enterprise license.

In effect CockroachDB treats all the `host` rules as `hostssl`, and
behaves as per a default of `hostnossl all all all reject`.

It is not currently possible to define authentication rules over
non-SSL connections: as of this writing, non-SSL connections are only
possible when running with `--insecure`, and on insecure nodes all the
authentication logic is entirely disabled.

This behavior remains equivalent to previous CockroachDB versions,
and this change is only discussed here for clarity.


Co-authored-by: Raphael 'kena' Poss <knz@thaumogen.net>
@knz
Copy link
Copy Markdown
Contributor Author

knz commented Jan 9, 2020

(found while working on #31113)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants