Skip to content

Document Clickhouse installation and configuration (experimental)#1228

Closed
ghost wants to merge 7 commits into
developfrom
unknown repository
Closed

Document Clickhouse installation and configuration (experimental)#1228
ghost wants to merge 7 commits into
developfrom
unknown repository

Conversation

@ghost

@ghost ghost commented Dec 4, 2023

Copy link
Copy Markdown

Purpose

Document Clickhouse database engine installation and configuration for Zonemaster-Backend. This is an experimental feature.

Context

Relates to zonemaster/zonemaster-backend#1094

Changes

Add a Clickhouse section to docs/public/installation/zonemaster-backend.md and docs/public/configuration/backend.md.

How to test this PR

Documentation, review the changes.

@tgreenx tgreenx added the A-Documentation Area: Documentation only. label Dec 4, 2023
@tgreenx tgreenx added this to the v2023.2 milestone Dec 4, 2023
matsduf
matsduf previously requested changes Dec 4, 2023
Comment thread docs/public/configuration/backend.md Outdated
Comment thread docs/public/configuration/backend.md Outdated
Comment thread docs/public/installation/zonemaster-backend.md Outdated
cpanm --notest Net::Statsd
```

## 10.2 Installation with Clickhouse

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I suggest that we have the same model as with the other databases, per OS. In this case "currently not available" for other than Debian/Ubuntu.

Comment on lines +831 to +832
Currently there are no instructions for cleaning up a Clickhose database.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
Currently there are no instructions for cleaning up a Clickhose database.
Currently there are no instructions for cleaning up a Clickhouse database.

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done

[Zonemaster::LDNS]: https://github.com/zonemaster/zonemaster-ldns/blob/master/README.md


Cleaning up the database]

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
Cleaning up the database]

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done


### 9.3. Clickhouse (FreeBSD)

There is not yet any specific installation instructions for FreeBSD. In most

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
There is not yet any specific installation instructions for FreeBSD. In most
There is no specific installation instructions for FreeBSD yet. In most

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done


### 9.1. Clickhouse (Rocky Linux)

There is not yet any specific installation instructions for Rocky Linux. In most

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
There is not yet any specific installation instructions for Rocky Linux. In most
There is no specific installation instructions for Rocky Linux yet. In most

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done

@matsduf matsduf dismissed their stale review December 5, 2023 22:36

With the proposed updates from @tgreenx I am ready to approve.

matsduf
matsduf previously approved these changes Dec 6, 2023
tgreenx
tgreenx previously approved these changes Dec 6, 2023

@tgreenx tgreenx left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A few comments after I went through here again while reviewing zonemaster/zonemaster-backend#1094

Comment thread docs/public/installation/zonemaster-backend.md Outdated
Comment thread docs/public/installation/zonemaster-backend.md
Comment thread docs/public/installation/zonemaster-backend.md Outdated
Comment thread docs/public/installation/zonemaster-backend.md Outdated
@ghost ghost dismissed stale reviews from tgreenx and matsduf via 3a49896 December 7, 2023 16:08

@tgreenx tgreenx left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

One more thing I found out when following the installation instructions on my machine.

Comment thread docs/public/installation/zonemaster-backend.md Outdated
tgreenx
tgreenx previously approved these changes Dec 11, 2023
matsduf
matsduf previously approved these changes Dec 11, 2023

@marc-vanderwal marc-vanderwal left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The instructions are incomplete for Rocky Linux and FreeBSD, but I’ve filled in the gaps for both operating systems. I haven’t tested the FreeBSD instructions however; only Rocky Linux.

Comment thread docs/public/configuration/backend.md Outdated
Comment on lines +217 to +218
The [MySQL interface][Clickhouse MySQL interface]'s port the Clickhouse server
is listening on.

@marc-vanderwal marc-vanderwal Dec 12, 2023

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The way it’s worded can seem like it’s a typo. Can I suggest:

The port the Clickhouse server is listening on, for connections using the MySQL-compatible protocol.

Comment on lines +729 to +730
There is no specific installation instructions for Rocky Linux yet. In most
parts the instructions for Debian/Ubuntu can be followed.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

When I read this, it feels like “sorry, I’ve couldn’t be bothered to write instructions for Rocky Linux, go figure it out on your own using that other OS’s documentation.”

Doing so means repeating oneself, that’s true, but other sections in this document also repeat the instructions with slight variations where it’s needed.

In a nutshell:

> **Note**: The Clickhouse MySQL interface requires a [double SHA1
> password](https://clickhouse.com/docs/en/operations/settings/settings-users#password_double_sha1_hex)
> ```
> PASSWORD=$(base64 < /dev/urandom | head -c8); echo "$PASSWORD"; echo -n "$PASSWORD" | sha1sum | tr -d '-' | xxd -r -p | sha1sum | tr -d '-'

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Using openssl might be a bit less hacky. I also suggest avoiding echo -n:

Suggested change
> PASSWORD=$(base64 < /dev/urandom | head -c8); echo "$PASSWORD"; echo -n "$PASSWORD" | sha1sum | tr -d '-' | xxd -r -p | sha1sum | tr -d '-'
> PASSWORD=$(openssl rand -base64 12); echo "$PASSWORD"; printf "%s" "$PASSWORD" | openssl sha1 -binary | openssl sha1 | awk '{ print $2 }'

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The oneliner is a copy/paste from the Clickhouse documentation.

Comment on lines +779 to +780
There is no specific installation instructions for FreeBSD yet. In most parts
the instructions for Debian/Ubuntu can be followed.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There is a FreeBSD port for Clickhouse, so installing it is a pkg install -y clickhouse away.

You will also need to do pkg install -y p5-DBD-MySQL and adjust the sed command so that it modifies /usr/local/etc/zonemaster/backend_config.ini instead of /etc/zonemaster/backend_config.ini.

@marc-vanderwal

Copy link
Copy Markdown
Contributor

Good news though: I am happy to report that Clickhouse works on Rocky Linux 9 by following the instructions, where I filled in the gaps.

@ghost ghost dismissed stale reviews from matsduf and tgreenx via 3a096c9 December 12, 2023 09:32
@ghost

ghost commented Dec 12, 2023

Copy link
Copy Markdown
Author

I updated the documentation document to integrate other OS. I've changed the section layout a little to avoid repetitions.

@matsduf matsduf left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just some minor comments, else it looks good.

Comment thread docs/public/installation/zonemaster-backend.md Outdated
Comment thread docs/public/installation/zonemaster-backend.md Outdated
matsduf
matsduf previously approved these changes Dec 13, 2023

@tgreenx tgreenx left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Postponed, as the corresponding feature (zonemaster/zonemaster-backend#1094) has been postponed as well

@tgreenx tgreenx modified the milestones: v2023.2, v2024.1 Dec 20, 2023
@matsduf matsduf dismissed their stale review December 20, 2023 14:18

Not ready yet.

@ghost ghost closed this by deleting the head repository Feb 9, 2024
This pull request was closed.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

A-Documentation Area: Documentation only.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants