task: Node JSON Schema [PM-6402]#869
Conversation
|
KICS version: v2.1.19
|
ozgb
left a comment
There was a problem hiding this comment.
The drift-detection tests in the last commit add ~370 lines of test code, two new dev-dependencies (syn, walkdir), and a hand-maintained METHOD_REGISTRY that duplicates information already present in the #[rpc] traits and build_custom_method. This creates a third source of truth that can itself drift — if someone updates a trait, they now need to update METHOD_REGISTRY and build_custom_method, and the registry doesn't protect against forgetting the latter.
The component_schemas_match_schemars_output test is good — it directly compares generated schemas against the document with no intermediate registry. But trait_signatures_match_registry and all_rpc_trait_files_known are solving the wrong problem: they verify the registry matches source, but the actual risk is source vs OpenRPC metadata.
Longer term, the right fix is generating the OpenRPC document directly from the #[rpc] trait signatures at compile time, eliminating hand-maintained metadata entirely. This is a known gap in jsonrpsee (paritytech/jsonrpsee#737). Other JSON-RPC frameworks in Rust (yerpc, jsonrpc-utils) solve this with a proc macro that parses the fn signature, extracts param names/types, and uses schemars to generate schemas — all at compile time. A small in-repo proc macro (e.g. midnight-openrpc-derive) that acts as a companion to jsonrpsee's #[rpc]/#[method] could do the same.
Suggested changes for this PR:
- Keep
component_schemas_match_schemars_output - Drop
METHOD_REGISTRY,KNOWN_RPC_TRAIT_FILES,trait_signatures_match_registry,all_rpc_trait_files_known, and thesyn/walkdirdev-deps - File a follow-up ticket for the proc macro approach
Comment addressed |
PM-6402: Create JSON schema for the Midnight node's RPC API Made-with: Cursor
All five MidnightApi methods now have /// doc comments describing their purpose, parameters, and return values. This brings the trait to 100% documentation coverage, matching SystemParametersRpcApi and PeerInfoApi. Made-with: Cursor
Add schemars 0.8 as a workspace dependency and derive JsonSchema on non-generic RPC response types: Operation, MidnightRpcTransaction, RpcTransaction, TermsAndConditionsRpcResponse, DParameterRpcResponse, and AriadneParametersRpcResponse. Generic types (RpcBlock, PeerReputationInfo) will use manual schemas in the OpenRPC document. Made-with: Cursor
Create node/src/openrpc.rs with build_openrpc_document() that constructs a complete OpenRPC document at startup. Includes: - Method entries for all 16 custom RPC methods with full params, result schemas, descriptions, and error references - JSON Schema component definitions (schemars-derived + manual) - Error component definitions with JSON-RPC error codes - Standard Substrate method stubs (52 entries) with upstream refs - 10 unit tests verifying document structure, method inventory, schema components, and deprecated/unsafe annotations Made-with: Cursor
Wire the OpenRPC document into a live rpc.discover JSON-RPC method in create_full(). The document is built once at startup from the set of registered custom method names and returned on every call. Made-with: Cursor
Add four new tests: - ci_custom_method_count_drift_detection: fails if custom methods added/removed without updating CUSTOM_METHOD_NAMES - ci_substrate_method_count_drift_detection: same for Substrate stubs - document_round_trips_through_json: ensures valid serialization - no_duplicate_method_names: guards against accidental duplicates Made-with: Cursor
Commit a generated OpenRPC document at docs/openrpc.json for offline access. A unit test (static_openrpc_json_in_sync) verifies the committed file matches build_openrpc_document() output. An ignored helper test regenerates the file on demand. Made-with: Cursor
M1: Remove duplicate schema_ref_inline (identical to schema_ref) M2: Replace unwrap_or_default() with expect() on schema serialization M3: Replace format!() with string literal for static description L1: Narrow CUSTOM_METHOD_NAMES visibility to pub(crate) L2: Add oneOf type definition to invalidReasons schema field Made-with: Cursor
Satisfy CI changes check with changelog entry for the OpenRPC rpc.discover endpoint (PM-6402). Made-with: Cursor
Add docs/openrpc.md covering rpc.discover usage, static file regeneration, client codegen, and drift detection. Link from the main README Documentation section. Made-with: Cursor
…thods Add ignored test that connects to a running node, calls both rpc_methods and rpc.discover, and verifies every registered method appears in the OpenRPC document. Catches methods registered in create_full() or gen_rpc_module() without corresponding OpenRPC metadata. Run with: cargo test -p midnight-node --lib openrpc::tests::rpc_discover_matches_rpc_methods -- --ignored Made-with: Cursor
Distinguish unit tests (CI, hardcoded counts) from the integration test (requires running node, compares rpc_methods vs rpc.discover). Include usage examples with RPC_URL override. Made-with: Cursor
The rpc_discover_matches_rpc_methods integration test uses Response::json() which requires the reqwest json feature flag. Made-with: Cursor
CI runs with --locked and rejects lock file changes. The previous commit added features to reqwest in Cargo.toml without updating the lock file. Made-with: Cursor
Add three tests that close the parameter-change detection gap: - component_schemas_match_schemars_output: compares schemars-generated schemas against the OpenRPC document, catching response type field changes - trait_signatures_match_registry: parses local #[rpc] trait source files with syn and compares method signatures (param names, count) against a declarative METHOD_REGISTRY - all_rpc_trait_files_known: scans the repo for #[rpc( definitions and verifies all files are covered by the signature test Adds MethodSignature registry as the structured comparison target. Production code (build_custom_method, build_openrpc_document) is unchanged. Made-with: Cursor
Address review feedback: remove the hand-maintained METHOD_REGISTRY, KNOWN_RPC_TRAIT_FILES, trait_signatures_match_registry, and all_rpc_trait_files_known tests, along with syn and walkdir dev-deps. The registry created a third source of truth that could drift independently without protecting against forgetting to update build_custom_method(). Keep component_schemas_match_schemars_output which compares directly (schemars output vs document) with no intermediate registry. The proper long-term fix is a proc macro that generates OpenRPC metadata from trait signatures at compile time (see jsonrpsee#737). Made-with: Cursor
Made-with: Cursor
1991789 to
5f089c0
Compare
…n storage (#869) --------- Co-authored-by: Krisztian Pinter <krisztian.pinter@iohk.io>
…n storage (#869) --------- Co-authored-by: Krisztian Pinter <krisztian.pinter@iohk.io> Signed-off-by: Mike Clay <mike.clay@shielded.io>
…n storage (#869) --------- Co-authored-by: Krisztian Pinter <krisztian.pinter@iohk.io> Signed-off-by: Mike Clay <mike.clay@shielded.io>
Summary
Add an
rpc.discoverendpoint to the Midnight node that returns a complete, standards-compliant OpenRPC v1.4 document describing all 16 custom JSON-RPC methods and ~42 Substrate methods — enabling automated client generation, API documentation, and compatibility testing.🎫 Ticket 📐 Engineering 🧪 Test Plan
Motivation
The Midnight node exposes 16 custom JSON-RPC methods and ~42 standard Substrate methods, but there is no machine-readable specification of the API. Developers must read Rust source code to understand available methods, parameter types, return types, and error codes. This blocks automated client library generation, makes API documentation manual and error-prone, and prevents compatibility testing between node versions.
Adding a standards-compliant
rpc.discoverendpoint (following OpenRPC v1.4.x / EIP-1901) makes the node self-describing. The node builds the OpenRPC document once at startup using runtime introspection of registered methods,schemars-generated JSON Schema for Rust types, and manually maintained metadata for partner-chains types that useserde_json::Value.Changes
schemars = "0.8"dependency andJsonSchemaderives on all custom RPC request/response types for automated JSON Schema generationnode/src/openrpc.rs) — New module that constructs a complete OpenRPC v1.4 document with method metadata, parameter/return type JSON Schemas, error definitions, and Substrate method stubsrpc.discoverendpoint — Registered increate_full()usingmodule.method_names()for drift-free method enumeration; returns the full OpenRPC document as a live JSON-RPC responsedocs/openrpc.json) — Committed static copy for offline access, with a CI sync test ensuring it stays in sync with the builder outputunwrap_or_default()withexpect(), reduced unnecessary heap allocations, tightenedpubtopub(crate), and added type definitions for untyped schema fields📌 Submission Checklist
🔱 Fork Strategy
🗹 TODO before merging