-
Notifications
You must be signed in to change notification settings - Fork 963
Closed
Labels
1.0Issues and PRs required or helping to stabilize the APIIssues and PRs required or helping to stabilize the APIAPI breakThis PR requires a version bump for the next releaseThis PR requires a version bump for the next releasebrainstorm
Description
I just realized push-based consensus decoding could solve a bunch of issues:
- Work with APIs that don't enable pull-based decoding
- Enables trivial
asyncdeserialization - Potentially enable allocation-free decoding (this is somewhat orthogonal but I guess good to think about together)
- Reduce the complexity of no-std feature more problematic than anticipated #758 to just having a simple generic
Writetrait
Design draft (owned decoding only):
pub trait Decodable {
type Decoder: Decoder<Item = Self>;
/// Decodes `Self` by reading from a buffered reader.
#[cfg(any(feature = "std", feature = "core"))]
fn from_reader<R: BufRead>(mut reader: R) -> Result<Self, IoError<Self::Error>> {
let mut deserializer = Self::Decoder::default();
while let Ok(buf) = reader.fill_buf()? {
if buf.is_empty() {
break;
}
let consumed = deserializer.put_bytes(buf).map_err(IoError::Decode)?;
reader.consume(consumed);
if consumed < buf.len() {
break;
}
}
deserializer.end().map_err(Into::into)
}
/// Decodes `Self` from given slice containing *full* data.
///
/// The slice must **not** be partial - use `Decoder` API to handle partial slices.
/// Returns the number of bytes consumed as the second field of the tuple on success.
fn from_slice(bytes: &[u8]) -> Result<(Self::Item, usize), DecodeError<Self::Error> {
let mut decoder = Self::Decoder::default();
let consumed = decoder.put_bytes(bytes).map_err(DecodeError::Decode)?;
let item = decoder.end()?;
Ok((item, consumed))
}
}
pub trait Decoder: Default {
type Item;
type Error; // perhaps we should require our own error trait?
/// Add these bytes into deserializer state.
///
/// Returns the number of consumed bytes on success, error if data was corrupted.
/// If the returned number is less than `buf.len()` then this decoder finished decoding and `end()` should be called.
/// It will no longer accept any bytes.
///
/// ## Panics
///
/// The function should panic if `bytes` is empty.
fn put_bytes(&mut self, bytes: &[u8]) -> Result<usize, Self::Error>;
/// Finishes decoding the item.
///
/// Returns decoded item or `UnexpectedEndError` if decoding didn't finish.
fn end(self) -> Result<Self::Item, UnexpectedEndError>;
}
// not entirely correct macro, just to give you an idea
macro_rules! fixed_size_decodable {
($type:ty, $decoder:ident, $len:expr, $conversion_method:ident) => {
pub struct $decoder {
// could be MaybeUninit in the future, this should be well-contained
// or model it with an enum
buf: [u8; $len],
len: usize,
}
impl $crate::Decoder for $decoder {
fn put_bytes(&mut self, bytes: &[u8]) -> Result<usize, Self::Error> {
let dst = &mut self.buf[self.len..];
let to_copy = dst.len().min(bytes.len());
dst[..to_copy].copy_from_slice(&bytes[..to_copy]);
self.len += to_copy;
Ok(to_copy)
}
fn end(self) -> Result<Self, UnexpectedEndError> {
if self.len == $len {
Ok($type::$conversion_method(self.buf))
} else {
Err(UnexpectedEndError::new(self.len, $len))
}
}
}
}
}
fixed_size_decodable!(u32, U32Decoder, 4, from_be_bytes);I also envision MinLenDecodable and ExactLenDecodable traits to simplify decoding for some types where they'd only have to implement conversions from byte arrays.
Reactions are currently unavailable
Metadata
Metadata
Assignees
Labels
1.0Issues and PRs required or helping to stabilize the APIIssues and PRs required or helping to stabilize the APIAPI breakThis PR requires a version bump for the next releaseThis PR requires a version bump for the next releasebrainstorm