Updates the authorization model for batches and adds support for non-authorized, small batches by matsduf · Pull Request #1406 · zonemaster/zonemaster

matsduf · 2025-07-11T15:39:41Z

Purpose

This PR, firstly, moves away from authorization of batch creation by using username/password created through RPC API to using a token configured in the configuration file. Secondly, is supports the creation of small batches without authorization. In the latter case using undelegated data etc will not be supported.

When small batches without authorization can be created, that function is to be added to GUI.

Updates configuration/backend.md:
- Updates table of contents
- Deprecates "enable_add_batch_job"
- Deprecates "enable_add_api_user"
- Redefines "enable_batch_create" to replace "enable_add_batch_job"
- Defines "enable_batch_create" and "enable_add_batch_job" to be conflicting.
- Removes experimental "enable_user_create"
- Defines "batch_api_key" for a token to authorize batch creation.
- Defines "max_batch_size_non_auth" to set to number of domains in batches created without batch_api_key.
- Editorial updates.
Updates rpcapi-reference.md
- Updates table of contents
- Defines data type "Batch API key"
- Defines method "conf_max_batch_size_non_auth"
- Deprecates methods "add_api_user" and "add_batch_job"
- Redefines method "batch_create" to replace "add_batch_job"
- Removes experimental "user_create"
- Editorial updates
- Deprecates "batch id" and introduces "hash batch id" which is the same kind of "hashed ID" as the test ID.
Removal of deprecated parts:
- Parts deprecated by this PR are to be removed by release v2026.2.

When the changes in this PR are approved the following changes are to be created:

Updated docs/public/using/backend/Using-Zonemaster-Backend-for-batch-testing.md
Implement the updates in Backend.

Context

Simplify authorization model zonemaster-backend#1223 (batch authorization model)
New batch id format zonemaster-backend#1154 ("hash format" on batch ID)
Anonymous batch creation zonemaster-backend#1153 (batch creation without token)

How to test this PR

Review the changes.

The purpose of the update is, firstly, to move away from authorization of batch creation by using username/password created through RPC API to using a token configured in the configuration file. Secondly, to support the creation of small batches without authorization. In the latter case using undelated data etc will not be supported. * Updates configuration/backend.md: * Updates table of contents * Deprecates "enable_add_batch_job" * Deprecates "enable_add_api_user" * Redefines "enable_batch_create" to replace "enable_add_batch_job" * Defines "enable_batch_create" and "enable_add_batch_job" to be conflicting. * Removes experimental "enable_user_create" * Defines "batch_create_token" for a token to authorize batch creation. * Defines "batch_create_max_length_non_auth" to set to number of domains in batches created without token. * Editorial updates. * Updates rpcapi-reference.md * Updates table of contents * Defines data type "Token" * Defines method "conf_batch_create_max_size_non_auth" * Deprecates methods "add_api_user" and "add_batch_job" * Redefines method "batch_create" to replace "add_batch_job" * Removes experimental "user_create" * Editorial updates

Co-authored-by: Marc van der Wal <103426270+marc-vanderwal@users.noreply.github.com>

matsduf · 2025-09-04T21:34:30Z

Renames some methods and keys. No change of logic.

Co-authored-by: Mattias Päivärinta <mattias@paivarinta.se>

Co-authored-by: Marc van der Wal <103426270+marc-vanderwal@users.noreply.github.com>

marc-vanderwal · 2026-01-20T15:22:59Z

+Multiple `batch_api_key` is supported. All configured keys will be equal in
+authorization. For each key `batch_api_key` is repeated on a new line in the
+configuration file with the "Batch API Key" as value.


So does that mean that something like this will work?

[RPCAPI] batch_api_key = "NotSoSecret" batch_api_key = "NotSoSecretEither" batch_api_key = "LikewiseNotASecret"

Do think the language should be clarified? Do you think it is a good or bad idea?

I think it’s a good idea because it helps avoid having shared secrets between more than two components.

Yes, maybe you could add an example like the one I gave in my previous comment in the text to make the syntax clearer.

@marc-vanderwal, I like examples because it helps to understand. In that file we do not have much examples, but we have several examples in the zonemaster-backend:share/config.ini file. Do you agree that it would be better to put your examples in the ini file? It will be updated in an upcoming PR.

I think it’s best to have this example both in the documentation and in the example configuration file. The syntax is a bit unorthodox because configuration keys are usually unique within a file. It should be crystal clear for users that yes, you can have multiple batch_api_key variables and that it makes all API keys work, not just the first or last one.

Updated as suggested.

MichaelTimbert · 2026-05-18T13:22:26Z

+
+### API method: `get_batch_job_result`
+
+*Deprecated. To be removed with release v2025.2* Replaced by


Suggested change

*Deprecated. To be removed with release v2025.2* Replaced by

*Deprecated. To be removed with release v2026.2* Replaced by

Already removed. I have to rebase the PR.

MichaelTimbert · 2026-05-18T13:31:02Z

+#### `"result"`
+
+An object with the following properties:
+
+* `"batch_id"`: A [*hash batch id*][Hash batch id].
+


It is confusing with the example. do we get

"result": "B-c45a3f8256c4a155"

or

"result": { "batch_id": "B-c45a3f8256c4a155" }

Updates the specification. It should use the same model as "start_domain_test" and "add_batch_job".

Co-authored-by: MichaelTimbert <110017095+MichaelTimbert@users.noreply.github.com>

MichaelTimbert

Look good for me.

matsduf added this to the v2025.2 milestone Jul 11, 2025

matsduf requested review from MichaelTimbert, marc-vanderwal, mattias-p, tgreenx and tolvmannen July 11, 2025 15:39

matsduf added T-Feature Type: New feature in software or test case description A-Documentation Area: Documentation only. labels Jul 11, 2025

matsduf mentioned this pull request Jul 11, 2025

Simplify authorization model zonemaster/zonemaster-backend#1223

Open

matsduf added 2 commits July 14, 2025 11:51

Adds the same kind of "hash id" for batches as for tests.

2cc7a6c

Moves removal of parts deprecated from v2026.2 to v2026.1.

1bf0947

matsduf force-pushed the update-batch-auth branch from 9231d6e to 1bf0947 Compare July 14, 2025 10:02

marc-vanderwal reviewed Aug 7, 2025

View reviewed changes

Comment thread docs/public/using/backend/rpcapi-reference.md Outdated

Update docs/public/using/backend/rpcapi-reference.md

707e609

Co-authored-by: Marc van der Wal <103426270+marc-vanderwal@users.noreply.github.com>

matsduf requested a review from marc-vanderwal August 10, 2025 14:17

matsduf added the RC-Features Release category: Features. label Aug 10, 2025

marc-vanderwal previously approved these changes Aug 11, 2025

View reviewed changes

Renames "token" to batch_api_key (Batch API key)

d066e58

matsduf dismissed marc-vanderwal’s stale review via d066e58 September 4, 2025 20:59

matsduf added 3 commits September 4, 2025 23:03

Updated the order of the keys to have it sorted

057701a

Renames "batch id type 2" to "hash batch id"

a0de915

Renames batch_create_max_size_non_auth to max_batch_size_non_auth

9aeac75

matsduf requested review from marc-vanderwal and mattias-p and removed request for mattias-p September 4, 2025 21:34

mattias-p reviewed Sep 8, 2025

View reviewed changes

matsduf and others added 2 commits September 8, 2025 19:51

Apply suggestions from review

ed9ccfc

Co-authored-by: Mattias Päivärinta <mattias@paivarinta.se>

Update docs/public/using/backend/rpcapi-reference.md

9b3f420

Co-authored-by: Mattias Päivärinta <mattias@paivarinta.se>

matsduf and others added 2 commits October 27, 2025 17:31

Apply suggestion from @marc-vanderwal

f22a5e6

Co-authored-by: Marc van der Wal <103426270+marc-vanderwal@users.noreply.github.com>

Apply suggestion from @marc-vanderwal

be04bd4

Co-authored-by: Marc van der Wal <103426270+marc-vanderwal@users.noreply.github.com>

matsduf requested a review from marc-vanderwal October 27, 2025 16:33

Changes yes/no to true/false as values on new bolean key

0502cac

matsduf mentioned this pull request Nov 7, 2025

Deprecate yes/no boolean values #1438

Merged

matsduf added 2 commits November 7, 2025 11:52

Moved deprecation to v2026.2

d7ceda8

Corrected

40f94c7

matsduf modified the milestones: v2025.2, v2026.1 Nov 13, 2025

marc-vanderwal reviewed Jan 20, 2026

View reviewed changes

Comment thread docs/public/configuration/backend.md Outdated

matsduf modified the milestones: v2026.1, v2026.2 Feb 2, 2026

matsduf added 2 commits April 9, 2026 16:35

Merge branch 'develop' into update-batch-auth

e5bfefc

Reverted the specification of "polling_interval" to the current language

8572f09

matsduf requested review from marc-vanderwal and mattias-p and removed request for mattias-p April 9, 2026 15:01

Adds example of multiple Batch API Keys, based on review comment

7b44cc8

matsduf requested review from marc-vanderwal and removed request for marc-vanderwal May 5, 2026 11:36

MichaelTimbert reviewed May 19, 2026

View reviewed changes

Apply suggestions from code review

db77b51

Co-authored-by: MichaelTimbert <110017095+MichaelTimbert@users.noreply.github.com>

MichaelTimbert previously approved these changes May 20, 2026

View reviewed changes

Updates after review comments

3ce44c6

matsduf dismissed MichaelTimbert’s stale review via 3ce44c6 May 20, 2026 08:13

Removes deprecated method.

324f301

matsduf requested a review from MichaelTimbert May 20, 2026 08:43

matsduf mentioned this pull request May 21, 2026

add_api_user localhost check can be bypassed via spoofed X-Forwarded-For #1485

Closed

marc-vanderwal approved these changes May 27, 2026

View reviewed changes


		### API method: `get_batch_job_result`

		Deprecated. To be removed with release v2025.2 Replaced by

Uh oh!

Conversation

matsduf commented Jul 11, 2025 • edited by tgreenx Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Purpose

Context

How to test this PR

Uh oh!

Uh oh!

matsduf commented Sep 4, 2025

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

MichaelTimbert left a comment

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

matsduf commented Jul 11, 2025 •

edited by tgreenx

Loading