Today I watched a friend use wormhole send on a large (multi-GB) file over a relatively slow link, such that the transfer required several hours. About halfway through, the sending machine reported an error, probably due to a lost connection:
Traceback (most recent call last):
File "/usr/local/lib/python2.7/dist-packages/wormhole/cli/cli.py", line 99, in _dispatch_command
yield maybeDeferred(command)
Exception: Consumer asked us to stop producing
ERROR: Consumer asked us to stop producing
We don't know whether this was a transient network problem, or the simply the person at the far end closing the lid on their laptop.
What it made us think is that it'd be handy to have a way to resume an interrupted transfer. It'd be easier for files than for directories, but the basic idea would be:
- when a transfer begins, write a record to some
~/.wormhole/ file, maybe ~/.wormhole/in-progress/RANDOM.json. The record should include a "Wormhole Seed", the code that was used, the pathname of the source/dest file, and the intended size
- if the transfer finishes normally, delete the record
wormhole receive should do tab-completion against codes from the in-progress records, as well as the channels reported by the rendezvous server. Selecting an in-progress code means "please resume that previous connection". That will use the Seed to build a new connection, use the record to locate the previously-transferred contents, and ask the sender to start sending from a particular byte offset.
The goal is to allow "up-arrow" (aka !!, aka "re-run the exact previous command") to mean "please resume the previous transfer". I'm not entirely sure this would work well, so we might need a special flag for it (wormhole send --resume?), and we could remind the user about it if/when a transfer fails (Network connection lost: run "wormhole receive --resume 4-pink-snowmen" to resume).
We wouldn't actually re-use the wormhole code in this case (since re-using a PAKE code doubles an attacker's chances of guessing it). The code would just be shorthand for referring to the same file (or the same act of transferrence) as before. The Wormhole Seed would provide the cryptographic pieces needed to establish a new connection safely (and note that Seeds can be reused multiple times, since they start with a full-strength random key).
I think the temporary file data wants to live directly next to the target location (so it will be on the same filesystem), but we discussed multiple places for the metadata that remembers the incomplete transfer. We could either put it right next to the target file, or we could put them in a shared ~/.wormhole/SOMETHING/ directory.
Putting it next to the target file has the benefit of reducing the uncertainly of what directory should be used for the receive: the target file will always be in the CWD of the command, regardless of whether it's the first time executed or the second.
The downside is that it isn't clear which tempfiles might be magic-wormhole interrupted-file reminder records, and which are unrelated. We could use a naming convention to find them, and/or a checksum to exclude other files, but it felt cleaner to me to put all such records in a single ~/.wormhole/ directory, where their purpose would be unambiguous.
We also discussed just writing the metadata into the end of the temporary file, so there'd only be one tempfile left lying around. But I was uncomfortable with mingling user data and metadata.
Today I watched a friend use
wormhole sendon a large (multi-GB) file over a relatively slow link, such that the transfer required several hours. About halfway through, the sending machine reported an error, probably due to a lost connection:We don't know whether this was a transient network problem, or the simply the person at the far end closing the lid on their laptop.
What it made us think is that it'd be handy to have a way to resume an interrupted transfer. It'd be easier for files than for directories, but the basic idea would be:
~/.wormhole/file, maybe~/.wormhole/in-progress/RANDOM.json. The record should include a "Wormhole Seed", the code that was used, the pathname of the source/dest file, and the intended sizewormhole receiveshould do tab-completion against codes from the in-progress records, as well as the channels reported by the rendezvous server. Selecting an in-progress code means "please resume that previous connection". That will use the Seed to build a new connection, use the record to locate the previously-transferred contents, and ask the sender to start sending from a particular byte offset.The goal is to allow "up-arrow" (aka
!!, aka "re-run the exact previous command") to mean "please resume the previous transfer". I'm not entirely sure this would work well, so we might need a special flag for it (wormhole send --resume?), and we could remind the user about it if/when a transfer fails (Network connection lost: run "wormhole receive --resume 4-pink-snowmen" to resume).We wouldn't actually re-use the wormhole code in this case (since re-using a PAKE code doubles an attacker's chances of guessing it). The code would just be shorthand for referring to the same file (or the same act of transferrence) as before. The Wormhole Seed would provide the cryptographic pieces needed to establish a new connection safely (and note that Seeds can be reused multiple times, since they start with a full-strength random key).
I think the temporary file data wants to live directly next to the target location (so it will be on the same filesystem), but we discussed multiple places for the metadata that remembers the incomplete transfer. We could either put it right next to the target file, or we could put them in a shared
~/.wormhole/SOMETHING/directory.Putting it next to the target file has the benefit of reducing the uncertainly of what directory should be used for the receive: the target file will always be in the CWD of the command, regardless of whether it's the first time executed or the second.
The downside is that it isn't clear which tempfiles might be magic-wormhole interrupted-file reminder records, and which are unrelated. We could use a naming convention to find them, and/or a checksum to exclude other files, but it felt cleaner to me to put all such records in a single
~/.wormhole/directory, where their purpose would be unambiguous.We also discussed just writing the metadata into the end of the temporary file, so there'd only be one tempfile left lying around. But I was uncomfortable with mingling user data and metadata.