Motivation

The primary motivation for the database implementation described in this PR is optimize key fragment availability. In order to support this requirement, each Ursula node must store a local database of decryption key fragments, which must be indexed, support concurrency, be capable of handling large numbers of records, and be easily backed up. The database must also be transparently integrated into the node's python runtime and support configuration of the database by CLI.

Requirements

• Add the ability to store, retrieve, and delete DKFs in a database on each individual Ursula node.
• Ensure that the database supports concurrent read/write operations and can handle large numbers of records.
• Allow for easy backup of the database (e.g. to a file, JSON, etc.).
• Integrate the database transparently into node operations in Python.

Data Model

The database model is defined as follows:

Ritual(Id, timestamp, fragment)
ID | timestamp | fragment |
42| 13234134  | 0101010101

Additional Details

• The database is a lightweight persistent key/value store.
• The data stored in the data base is security-sensitive.
• Nodes may participate in multiple concurrent DKG rituals while the database is running.
• Create implies finalization in of a DKG ritual, Retrieve implies a request for threshold decryption.
• Read/Write/Delete operations will only be permitted when the protocol's economic guarantees and user-defined conditions are satisfied (authorization, expiration, payment).

API

• A nucypher.datastore package that provides a simple and light database API:
  • nucypher.datastore.datastore
    • Datastore
      • initialize (clssmethod alternate constructor)
      • restore (clssmethod alternate constructor)
      • store_fragment
      • retrieve
      • delete_fragment
  • nucypher.datastore.models
    • Ritual(id, timestamp, fragment)

Initialize and Restore

datastore = datastore.Datastore.initialize(path)
datastore = datastore.Datastore.restore(path)

Usage

datastore = datastore.store_fragment(id, timestamp, fragment)
datastore = datastore.retrieve_fragment(id)
datastore = datastore.delete_fragment(id)

Internal Integrations

• Initialization occurs when a new ursula is initialized or upgraded from a previous version
  • A CLI command/option to allow node operators to configure the database (backup location) during nucypher ursula init|config.
  • initialize integration with nucypher ursula init|config
• Writes will occur when a DKG ritual is finalized
  • store_fragment integration with RitualTracker event handling callbacks.
• Reads will occur when theshold decryption is requested
  • retrieve_fragment integration with nucypher.network.server's threshold decryption endpoint.
• Deletions occur on DKG-Revoke
• Integration of the datastore into Ursula class.
• Integration of datastore into UrsulaConfig class.

Implications

• nodes will need to reconfigure
• backwards incompatibility
• bump configuration file version
• nucypher/nucypher-ops updates to support databases