IDevice writes may return non-zero error code due to underlying errors and leave the device range in an arbitrary state. Currently, FasterLog only reports those exceptions to callers of CommitAsync through returned Tasks. Meanwhile, these exceptions are essentially ignored as later ranges can still commit.

While some use cases call for continued operation despite data loss, many other cases could benefit from stronger failure guarantees and potential ways to fix errors.

Proposal for new FasterLog error behavior:

• On failure of device range (x, y), all commits that has tail larger than x will produce a FasterException (concurrent and later commit calls)
• In-memory operations are not impacted throughout IDevice error handling
• Upon receiving the FasterException, clients are free to perform custom error handling to mask or repair the IDevice failure. There will be an explicit call MarkLastDeviceFailureResolved() that will clear the errors status of FasterLog and allow future commits to succeed, potentially retrying ranges larger than x

For users that want to ignore errors:

• There will be a special ForceCommitUntil(addr) method that ignores unresolved errors. Using this up to range y before issuing MarkLastDeviceFailureResolved() will essentially imitate current behavior
• Unfortunately, this will be breaking for existing use cases that depends on loose error handling semantics

Implementation Details:

• On a non-zero error code, modify callback to not update FlushedUntilAddress
• Quiesce the system by stopping to accept new commit requests and waiting for outstanding flush requests to finish. Repair internal data structure to throw away any flushed information beyond y
• Expose exception to user . Wait for MarkLastDeviceFailureResolved() call
• For ForceCommitUntil(addr), regardless of fast commit mode, must wait for commit metadata to become persistent.