Skip to content

RPCAPI: fix parameter validation in job_status and user_create with JSON::Validator ⩾ 5.03#1109

Merged
marc-vanderwal merged 6 commits into
zonemaster:developfrom
marc-vanderwal:bugfix/1105
Nov 2, 2023
Merged

RPCAPI: fix parameter validation in job_status and user_create with JSON::Validator ⩾ 5.03#1109
marc-vanderwal merged 6 commits into
zonemaster:developfrom
marc-vanderwal:bugfix/1105

Conversation

@marc-vanderwal

@marc-vanderwal marc-vanderwal commented Jun 1, 2023

Copy link
Copy Markdown
Contributor

Purpose

This PR fixes a bug where, on systems having JSON::Validator version 5.03 or later installed, calls to some RPCAPI methods like job_status or user_create failed to have their parameters validated against their respective schemas. It affects all methods whose schemas are directly constructed by JSON::Validator::Joi and where at least one parameter is required, such as job_status and user_create.

The offending code took a $method_schema, which could either be a raw Perl hash containing a JSON schema or an opaque object made with JSON::Validator::Joi, and tried to check whether the schema specified any keys as required. It did so by checking whether the required key exists in $method_schema.

This works fine if $method_schema is a Perl hash. But when it’s a JSON::Validator::Joi object, we are touching an implementation detail of that class, which could sooner or later break!

And indeed, it broke: between JSON::Validator version 5.02 and 5.03, an implementation detail was changed, causing issue #1105 on systems where JSON::Validator version 5.03 or later is installed. The change seems to have been introduced by commit 788fd42 on JSON::Validator’s Github repo.

Adding the unit tests also revealed a few more problems, which this pull request also addresses:

  • a bug causing a 500 Internal Server Error if we sent the RPCAPI endpoint anything but an object;
  • a bug causing a 500 Internal Server Error if the JSONRPC object contains anything but an object in the params key;
  • a bug causing the params object not to be validated if the corresponding method has no mandatory arguments;
  • and the lack of type enforcement of the id key.

Context

Fixes #1105.

Changes

  • Add unit tests to exercise the jsonrpc_validate method in Zonemaster::Backend::RPCAPI;
  • Ensure schemas made with JSON::Validator::Joi are compiled before examining them;
  • Add checks before attempting to dereference a ref as a hashref;
  • Ensure the id member of the JSON-RPC object is of the correct type;
  • Revisit the validation logic so as to always validate the params member if it exists.

How to test this PR

The newly-introduced unit tests should pass.

To test the fix for #1109 manually:

  1. Install JSON::Validator version 5.03 or later on your system;
  2. Start Zonemaster::Backend;
  3. Perform the following query using curl:
curl -s -H "Content-Type: application/json" \
    -d '{ "jsonrpc": "2.0", "method": "job_status", "params": {}, "id": "testing"}' \
     http://localhost:5000/ | jq

The output should be equivalent to the following JSON object:

{
  "jsonrpc": "2.0",
  "id": "testing",
  "error": {
    "message": "Invalid method parameter(s).",
    "code": "-32602",
    "data": [
      {
        "path": "/test_id",
        "message": "Missing property"
      }
    ]
  }
}

Also try passing data that is valid JSON but of the incorrect type as JSON-RPC objects, or as params keys.

@marc-vanderwal marc-vanderwal added T-Bug Type: Bug in software or error in test case description V-Minor Versioning: The change gives an update of minor in version. labels Jun 1, 2023
@marc-vanderwal marc-vanderwal added this to the v2023.2 milestone Jun 1, 2023
@marc-vanderwal marc-vanderwal requested review from a user, hannaeko, matsduf, mattias-p and tgreenx June 1, 2023 10:21
@marc-vanderwal marc-vanderwal linked an issue Jun 1, 2023 that may be closed by this pull request

@ghost ghost left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Subtle bug indeed. Nicely catch. The unit test file should be added to the MANIFEST. Beside that, this looks good to me.

@marc-vanderwal

Copy link
Copy Markdown
Contributor Author

I am not quite satisfied with the coverage and the style of the .t file. Addressing both helped me find even more bugs in the same bit of code.

mattias-p
mattias-p previously approved these changes Jun 5, 2023

@mattias-p mattias-p left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Great unit test! Great write ups in commit messages! Just great!

It's unfortunate that we need to configure a database just to test the validation, but we can fix that later. I'll create an issue.

Related to zonemaster#1105.

It seems that we are missing a proper test for the validation routine of
a JSONRPC object. This commit introduces one.

Some of the tests are disabled because they would either fail or crash
completely. They will be enabled as the corresponding bugs are fixed.
Related to zonemaster#1105.

A subtle bug could cause RPCAPI methods, whose parameters were validated
against JSON schemas generated with JSON::Validator::Joi, to skip
validation of parameters against that schema entirely.

The offending code took a $method_schema, which could either be a raw
Perl hash containing a JSON schema or an opaque object made with
JSON::Validator::Joi, and tried to check whether the schema specified
any keys as required. It did so by checking whether the 'required' key
exists in $method_schema.

This works fine if $method_schema is a Perl hash. But when it’s a
JSON::Validator::Joi object, we are touching an implementation detail of
that class, which could sooner or later break!

And indeed, it broke: between JSON::Validator version 5.02 and 5.03, an
implementation detail was changed, causing issue zonemaster#1105 on systems where
JSON::Validator version 5.03 or later is installed. The change seems to
have been introduced by commit 788fd42 [1] on JSON::Validator’s Github
repo [2].

[1]: jhthorsen/json-validator@788fd42
[2]: https://github.com/jhthorsen/json-validator
Calling the RPCAPI endpoint with a number or a string instead of null or
an object caused a “500 Internal Server Error” to be returned, instead
of a propre RPCAPI error. This was due to the offending code failing to
check whether we were passed a hashref before attempting to dereference
it and checking the existence of an “id” key in it.
According to the specification, IDs must be strings, numbers (although
numbers with fractional parts are discouraged) or null (also
discouraged, but permissible). We might as well enforce it.
There was another situation where JSON schema enforcement could be
skipped: when a method only has optional parameters and the “params” key
in the JSONRPC object was set to something, the value of that key was
not validated.

The offending code skipped validation altogether if none of the method’s
parameters is required. What was probably meant is that if none of the
method’s parameters are required, then the absence of a “params” key in
the JSONRPC object is acceptable. But a method requiring no parameters
doesn’t mean that it has no parameters at all (e.g. when a method has
default values for all its parameters), so validation should still be
done on a “params” key if it’s there.

The rewritten code implements a different, but more correct logic: if
the “params” key exists, validate its value and return an error if it
fails validation; if the “params” key does not exist and the method
requires at least one parameter, then return an error; else return
success.
@marc-vanderwal marc-vanderwal merged commit 8ebe174 into zonemaster:develop Nov 2, 2023
@ghost

ghost commented Jan 10, 2024

Copy link
Copy Markdown

v2023.2 release testing

Tested based on the "How to test" section and works as expected.

@ghost ghost added the S-ReleaseTested Status: The PR has been successfully tested in release testing label Jan 10, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

S-ReleaseTested Status: The PR has been successfully tested in release testing T-Bug Type: Bug in software or error in test case description V-Minor Versioning: The change gives an update of minor in version.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Insufficient parameter validation in job_status method

3 participants