Skip to content

make writes more reliable through cache #1936

@remusb

Description

@remusb

This is to track future work on cache to make uploads and general writes through cache more fault tolerant.

Ideas I have so far:

  1. cache will add the files in the storage and make it available for reading once that is done
  2. in the background, it will make use of rclone's syncing functionality to reliably upload the file to where it was originally intended to be

A couple of things I can think of:

  1. Cache will now be responsible of data. Before this, data would be manipulated based on user actions only but now it's actually going to do the backup on the cloud provider on its own.
  2. Naturally the temporary storage for files yet to be uploaded must be independent of the cache storage. This is to ensure that 1 is met (partially) and that it can resume them if rclone crashes for some reason.
  3. 2 might double the size of a file that it takes on disk. It's safer but more costly. Need to think about this a bit more
  4. Folder merge, overwrites, etc. All these cases that require input would not be possible in the background. At the same time I would like cache to throttle the backups to not put pressure on the cloud provider.

Metadata

Metadata

Assignees

Type

No type

Projects

No projects

Milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions