there is a variety of storage schemes which provide different ways of encoding, exchanging and storing information about user data to achieve the required levels of reliability at different costs. Lets define this variety as StorageScheme (SS). dditionally, each SS may provide additional parameters that may affect the encoding and/or exchange/storage of user data

up until now we have only one (default) SS - Replication. Each user object is stored as is, but in multiple copies (REP N). In particular, single-copy replication involves a trivial singular scheme that is not explicitly distinguished

erasure coding seems to be a separate SS since it fundamentally differs from replication. Obviously, we must keep previous scheme untouched. Additional schemes will give the user freedom of choice to solve storage problems more efficiently

Activation

Container is the best mechanism of grouping objects by common props in NeoFS. Replication is configured per container and applies to all its objects. Being one of the schemes, the most logical thing would be to extend this approach to EC

we may define enum of storage schemes, and specification through container attribute. Let it be StorageScheme. Being a default scheme, Replication corresponds to the absent attribute. Other algorithms are set in value

for Reed-Solomon codes (d, p) w/ d and p data and parity blocks correspondingly, the attribute is "StorageScheme": "RS(d,p)". Per-value attributes - 3 for each RS, d and p - is also an option, but take more data

with this, the storage system will split and store data without going into the details of the rationale behind such a scheme. User wants, user gets. Full control over data life. At the same time, in practice, the selection of parameters will most likely cause difficulties. It is necessary to take into account the cluster topology (which also changes while the container is immutable), the volume/value of data and expectations for the space consumed relative to the desired fault tolerance. As a compromise, it will be needed to provide standard options covering practical use cases

Encoding

being stored in containers with "StorageScheme": "RS(d,p)", original user data is split (*) into data and parity blocks (aka shards) of the same size

(*) NeoFS has no arch limit on the user object's size. Since the entire block of coded data is required for EC, this operation will be applied to slices of the original data. This means that in general the original data is first split according to the current scheme, and only then each slice is subjected to EC and written to the storage

for each received block of code, an object (*) is formed, the payload of which is the code. These objects are placed according to REP 1 policy, i.e. in 1 copy with HRW-by-ID distribution. This will allow the blocks to be distributed across nodes as much as possible and, in the limit, obtain equivalence between node failure and code block loss

Storage

to a first approximation, code blocks are stored as regular objects. Each physically stored objects in container in EC is a block of some code

Recovery (active)

to recover data using RS algorithm, we need to know the following:

size of the original data (for alignment recover)
idx of each block in the code
whether particular block it is corrupted or not

storing blocks as objects, 2 is supported automatically. For 1, we may persist the following metadata:

size of the encoded data, i.e. binary-encoded user root or split object, header+payload
number of data and parity blocks
block checksums
block order

Recovery (background)

Policer checks whether individual objects are stored according to the container policy or not. For Replication containers, error correction is achieved by rewriting the replica of the object. At the same time, for REP 1 containers recovery is either not needed or impossible

for EC containers, error detection comes down to detecting missing blocks that need to be restored. However, if the blocks are stored in one instance of an object, then to restore the missing blocks, it is necessary to create a new object that is the restored code block. The object will be created by some SN. Such behavior will entail a number of related nuances

Support erasure codes in object service #526

Description

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions