Skip to content

most of time for backup is cost in scan(unreasonable) #1160

@gluthgao

Description

@gluthgao

Output of restic version

debug log file /Users/gluth/go/src/github.com/restic/restic/log1.txt
debug enabled
restic 0.7.1
compiled with go1.8.1 on darwin/amd64

How did you run restic exactly?

Run the back up of local NAS to remote NAS
./restic -r remoteNAS backup / NAS

What backend/server/service did you use?

NAS

Expected behavior

We should not scan the file system first which cost long time, i have lots of file around 15TB,
the total time should be spend on the backup data.

Actual behavior

the scan cost around 6 hours !!!!

Steps to reproduce the behavior

lots of file system like NAS, then do the backup

Do you have any idea what may have caused this?

scan file system cost more time is normally since the fs.walk is single thread travel model which will cost long time.

Do you have an idea how to solve the issue?

Do we necessary to scan the file system to get the total file/dir number + size?
also some file may not have permission to read or read failure that's hard to get the accurate size.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions