Skip to content

Large files uploaded via crypt remote to B2 are missing SHA1SUMs #1767

@tjanez

Description

@tjanez

What is your rclone version (eg output from rclone -V)

rclone v1.38
- os/arch: linux/amd64
- go version: go1.9

Which OS you are using and how many bits (eg Windows 7, 64 bit)

Fedora 26 (Workstation Edition), 64 bit

Which cloud storage system are you using? (eg Google Drive)

B2 Backblaze

The command you were trying to run (eg rclone copy /tmp remote:tmp)

An example of a large file uploaded via crypt remote to B2 that misses SHA1SUM:

$ rclone mkdir b2:crypted-missing-sha1sum-test
$ rclone config
Configure new crypted remote named crypted-missing-sha1sum-test:

Remote config
--------------------
[crypted-missing-sha1sum-test]
remote = b2:crypted-missing-sha1sum-test
filename_encryption = off
password = *** ENCRYPTED ***
password2 = *** ENCRYPTED ***
--------------------

$ ls -lh ~/Dist/Fedora-Workstation-Live-x86_64-26-1.5.iso 
-rw-rw-r--. 1 tadej tadej 1.5G Jul  6 00:34 /home/tadej/Dist/Fedora-Workstation-Live-x86_64-26-1.5.iso
$ rclone copy -v ~/Dist/Fedora-Workstation-Live-x86_64-26-1.5.iso crypted-missing-sha1sum-test:
2017/10/22 15:06:20 INFO  : Encrypted drive 'crypted-missing-sha1sum-test:': Modify window is 1ms
2017/10/22 15:06:20 INFO  : Encrypted drive 'crypted-missing-sha1sum-test:': Waiting for checks to finish
2017/10/22 15:06:20 INFO  : Encrypted drive 'crypted-missing-sha1sum-test:': Waiting for transfers to finish
2017/10/22 15:07:20 INFO  : 
Transferred:   383.938 MBytes (6.041 MBytes/s)
Errors:                 0
Checks:                 0
Transferred:            0
Elapsed time:      1m3.5s
Transferring:
 *     Fedora-Workstation-Live-x86_64-26-1.5.iso: 25% done, 8.578 MBytes/s, ETA: 2m9s

[... output trimmed... ]

2017/10/22 15:23:43 INFO  : Fedora-Workstation-Live-x86_64-26-1.5.iso: Copied (new)
2017/10/22 15:23:43 INFO  : 
Transferred:   1.456 GBytes (1.424 MBytes/s)
Errors:                 0
Checks:                 0
Transferred:            1
Elapsed time:    17m26.8s
$ rclone sha1sum b2:crypted-missing-sha1sum-test
                                          Fedora-Workstation-Live-x86_64-26-1.5.iso.bin
$ rclone sha1sum crypted-missing-sha1sum-test:
                                          Fedora-Workstation-Live-x86_64-26-1.5.iso

An example of the same large file uploaded directly to B2 remote that has SHA1SUM:

$ rclone mkdir b2:missing-sha1sum-test
$ ls -lh ~/Dist/Fedora-Workstation-Live-x86_64-26-1.5.iso 
-rw-rw-r--. 1 tadej tadej 1.5G Jul  6 00:34 /home/tadej/Dist/Fedora-Workstation-Live-x86_64-26-1.5.iso
$ rclone copy -v ~/Dist/Fedora-Workstation-Live-x86_64-26-1.5.iso b2:missing-sha1sum-test/
2017/10/22 13:37:22 INFO  : B2 bucket missing-sha1sum-test: Modify window is 1ms
2017/10/22 13:37:23 INFO  : B2 bucket missing-sha1sum-test: Waiting for checks to finish
2017/10/22 13:37:23 INFO  : B2 bucket missing-sha1sum-test: Waiting for transfers to finish
2017/10/22 13:37:37 INFO  : 
Transferred:   7.346 MBytes (467.081 kBytes/s)
Errors:                 0
Checks:                 0
Transferred:            0
Elapsed time:       16.1s
Transferring:
 *     Fedora-Workstation-Live-x86_64-26-1.5.iso:  0% done, 1.299 MBytes/s, ETA: 19m2s

[... output trimmed... ]

2017/10/22 13:54:56 INFO  : Fedora-Workstation-Live-x86_64-26-1.5.iso: Copied (new)
2017/10/22 13:54:56 INFO  : 
Transferred:   1.456 GBytes (1.413 MBytes/s)
Errors:                 0
Checks:                 0
Transferred:            1
Elapsed time:    17m35.3s

$ rclone sha1sum b2:missing-sha1sum-test
65880b7e61f995df0009ecf556d736552526d2e0  Fedora-Workstation-Live-x86_64-26-1.5.iso

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions