Skip to content

Faster SCP Copying for Local to Remote directories#18

Merged
caldempsey merged 3 commits intomasterfrom
callum/unw-192-1
Jun 7, 2023
Merged

Faster SCP Copying for Local to Remote directories#18
caldempsey merged 3 commits intomasterfrom
callum/unw-192-1

Conversation

@caldempsey
Copy link
Copy Markdown
Contributor

This PR ensures we use the archiving process when copying directories from the local host to remote.

image

  1. If it does, assume we are doing local to remote (as the first argument would use the 'sess' header otherwise.
  2. DRY up the Copy arg to Scp args so we can prune the remote dirpath in a way that's re-usable, there's no guarantee that's the first SCP arg, but there is a guarantee it is the second one post validation.
@caldempsey caldempsey added the needs-extensive-testing This PR needs more extensive E2E testing before merging in label Jun 4, 2023
@caldempsey caldempsey requested a review from noorvir June 4, 2023 20:57
@caldempsey caldempsey removed the needs-extensive-testing This PR needs more extensive E2E testing before merging in label Jun 5, 2023
cmd/copy.go Outdated
} else {
err := copySourceSCP(scpArgs[0], scpArgs[1], publicKeyPath)
if err != nil {
ui.Infof("❎ Unsuccessful copy %s => %s", scpArgs[0], scpArgs[1])
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The emoji here should be ❌

cmd/copy.go Outdated
err := copySourceSCP(scpArgs[0], scpArgs[1], publicKeyPath)

// If the first argument of the SCP args points towards a logical directory assume local => remote
pathInfo, _ := os.Stat(scpArgs[0])
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This shouldn't ignore the error. The dereference below will cause a panic if the file doesn't exist. Also, we probably shouldn't assume the direction. We know that the remote path will always be prefixed by sess:. Let's use that info to determine the direction.

Copy link
Copy Markdown
Contributor Author

@caldempsey caldempsey Jun 6, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah I'll clean this up, it's a bit hacky

@caldempsey caldempsey requested a review from noorvir June 6, 2023 15:53
@caldempsey caldempsey merged commit 215c1df into master Jun 7, 2023
}

func isLocalDirToRemoteCopy(args []string) bool {
if strings.Contains(args[0], "sess:") {
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This should not access slice elements without checking the length. This will cause a panic if the slice is empty. This is why I think it's better not to break logic up too much.

The args check should happen in the cobra command handler where we can be sure the slice has an expected length. We should extract the args from the slice there and pass them to this function as a string.

Comment on lines +70 to +74
if strings.Contains(args[1], "sess:") {
return false
}

pathInfo, err := os.Stat(args[1])
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same issue as above accessing slice elements. This it prone to panics. The args parsing should happen in the cobra handler entry points.

Comment on lines +19 to +41
exec, err := getTargetExec(cmd, args)
if err != nil {
return err
}

if exec == nil {
return fmt.Errorf("At least one remote host must be specified")
}
if exec.Connection == nil {
return fmt.Errorf("Target session must have an active connection")
}
if exec.SSHKey.PublicKey == nil && config.SSHPublicKeyPath == "" {
return fmt.Errorf("Failed to identify public key, check your Unweave config file or specify it manually")
}

scpArgs, err := formatPaths(exec, args)
if err != nil {
ui.Infof("❌ Unsuccessful copy: %s", err.Error())
}
ui.Infof(fmt.Sprintf("🔄 Copying %s => %s", scpArgs[0], scpArgs[1]))

switch {
case isLocalDirToRemoteCopy(args):
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The args should be pars in this function and not passed to handlers as a slice. Here, we know the exact number of args expected because cobra validated them for us. If you pass them down to smaller private functions as a slice, we lose the information about the slice length. Another function could later call the isLocalDirToRemoteCopy function with an empty slice an cause a panice.

Instead, something like this is much safer and clearer to understand:

isLocalDirToRemoteCopy(pathA, pathB string)

@noorvir noorvir deleted the callum/unw-192-1 branch June 7, 2023 09:11
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants