RPCAPI: fix parameter validation in job_status and user_create with JSON::Validator ⩾ 5.03#1109
Merged
Merged
Conversation
c1a4047 to
b18dfa1
Compare
ghost
reviewed
Jun 1, 2023
ghost
left a comment
There was a problem hiding this comment.
Subtle bug indeed. Nicely catch. The unit test file should be added to the MANIFEST. Beside that, this looks good to me.
b18dfa1 to
6b49045
Compare
Contributor
Author
|
I am not quite satisfied with the coverage and the style of the |
mattias-p
previously approved these changes
Jun 5, 2023
mattias-p
left a comment
Member
There was a problem hiding this comment.
Great unit test! Great write ups in commit messages! Just great!
It's unfortunate that we need to configure a database just to test the validation, but we can fix that later. I'll create an issue.
Related to zonemaster#1105. It seems that we are missing a proper test for the validation routine of a JSONRPC object. This commit introduces one. Some of the tests are disabled because they would either fail or crash completely. They will be enabled as the corresponding bugs are fixed.
Related to zonemaster#1105. A subtle bug could cause RPCAPI methods, whose parameters were validated against JSON schemas generated with JSON::Validator::Joi, to skip validation of parameters against that schema entirely. The offending code took a $method_schema, which could either be a raw Perl hash containing a JSON schema or an opaque object made with JSON::Validator::Joi, and tried to check whether the schema specified any keys as required. It did so by checking whether the 'required' key exists in $method_schema. This works fine if $method_schema is a Perl hash. But when it’s a JSON::Validator::Joi object, we are touching an implementation detail of that class, which could sooner or later break! And indeed, it broke: between JSON::Validator version 5.02 and 5.03, an implementation detail was changed, causing issue zonemaster#1105 on systems where JSON::Validator version 5.03 or later is installed. The change seems to have been introduced by commit 788fd42 [1] on JSON::Validator’s Github repo [2]. [1]: jhthorsen/json-validator@788fd42 [2]: https://github.com/jhthorsen/json-validator
Calling the RPCAPI endpoint with a number or a string instead of null or an object caused a “500 Internal Server Error” to be returned, instead of a propre RPCAPI error. This was due to the offending code failing to check whether we were passed a hashref before attempting to dereference it and checking the existence of an “id” key in it.
According to the specification, IDs must be strings, numbers (although numbers with fractional parts are discouraged) or null (also discouraged, but permissible). We might as well enforce it.
There was another situation where JSON schema enforcement could be skipped: when a method only has optional parameters and the “params” key in the JSONRPC object was set to something, the value of that key was not validated. The offending code skipped validation altogether if none of the method’s parameters is required. What was probably meant is that if none of the method’s parameters are required, then the absence of a “params” key in the JSONRPC object is acceptable. But a method requiring no parameters doesn’t mean that it has no parameters at all (e.g. when a method has default values for all its parameters), so validation should still be done on a “params” key if it’s there. The rewritten code implements a different, but more correct logic: if the “params” key exists, validate its value and return an error if it fails validation; if the “params” key does not exist and the method requires at least one parameter, then return an error; else return success.
6b49045 to
8da793d
Compare
mattias-p
approved these changes
Jun 5, 2023
ghost
approved these changes
Jul 13, 2023
matsduf
approved these changes
Jul 13, 2023
v2023.2 release testingTested based on the "How to test" section and works as expected. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Purpose
This PR fixes a bug where, on systems having JSON::Validator version 5.03 or later installed, calls to some RPCAPI methods like
job_statusoruser_createfailed to have their parameters validated against their respective schemas. It affects all methods whose schemas are directly constructed by JSON::Validator::Joi and where at least one parameter is required, such asjob_statusanduser_create.The offending code took a
$method_schema, which could either be a raw Perl hash containing a JSON schema or an opaque object made with JSON::Validator::Joi, and tried to check whether the schema specified any keys as required. It did so by checking whether therequiredkey exists in$method_schema.This works fine if
$method_schemais a Perl hash. But when it’s a JSON::Validator::Joi object, we are touching an implementation detail of that class, which could sooner or later break!And indeed, it broke: between JSON::Validator version 5.02 and 5.03, an implementation detail was changed, causing issue #1105 on systems where JSON::Validator version 5.03 or later is installed. The change seems to have been introduced by commit 788fd42 on JSON::Validator’s Github repo.
Adding the unit tests also revealed a few more problems, which this pull request also addresses:
paramskey;paramsobject not to be validated if the corresponding method has no mandatory arguments;idkey.Context
Fixes #1105.
Changes
jsonrpc_validatemethod inZonemaster::Backend::RPCAPI;idmember of the JSON-RPC object is of the correct type;paramsmember if it exists.How to test this PR
The newly-introduced unit tests should pass.
To test the fix for #1109 manually:
The output should be equivalent to the following JSON object:
{ "jsonrpc": "2.0", "id": "testing", "error": { "message": "Invalid method parameter(s).", "code": "-32602", "data": [ { "path": "/test_id", "message": "Missing property" } ] } }Also try passing data that is valid JSON but of the incorrect type as JSON-RPC objects, or as
paramskeys.