TCL, expect: Multiple files w/ SCP

Question

I was able to transfer files with scp and expect, now I tried to upload several files at once:

#!/usr/bin/expect -f
# Escapes spaces in a text
proc esc text {
    return [regsub -all {\ } $text {\\&}]
}

# Uploads several files to a specified server
proc my_scp_multi {ACCOUNT SERVER PW files newfolder} {
    set timeout 30
    send_user -- "\n"
    spawn scp $files $ACCOUNT@$SERVER:[esc $newfolder]
    match_max 100000
    # Look for password prompt
    expect {
    -re ".*Connection closed.*" {
        sendError "\n\n\nUpload failed!\nPlease check the errors above and start over again.\nThis is most likely induced by too many wrong password-attempts and will last quite a time!"
    }
    -re ".*Permission denied.*" {
        sendError "\n\n\nUpload failed!\nPlease check the errors above and start over again.\nYou entered most likely a wrong password!"
    }
    -re ".*Are.*.*yes.*no.*" {
        send "yes\n"
        exp_continue
        #look for the password prompt
    }
    -re ".*sword.*" {
        # Send password aka $PW
        send -- "$PW\r"
        # send blank line (\r) to make sure we get back to gui
        send -- "\r\n"
        exp_continue
    }

    send_user -- "Upload successful!\n"
    }

    set timeout -1
}

When I want to upload several files, the sh command is: scp $a $b $c user@server:$folder, so I called my_scp_multi "ACCOUNT" "SERVER" "PW" "~/testfileA ~/testfileB ~/testfileC" "~/test/". Which also produces this output:

spawn scp ~/testfileA ~/testfileB ~/testfileC user@server:~/test/
user@server's password: 
~/testfileA ~/testfileB ~/testfileC: No such file or directory

It seems to see "~/testfileA ~/testfileB ~/testfileC" as one file. But when I copy-paste scp ~/testfileA ~/testfileB ~/testfileC user@server:~/test/ to the console it works fine!

What am I doing wrong? I've tried "\"~/testfileA\" \"~/testfileB\" \"~/testfileC\"" and such things, but nothing did work at all.

Any ideas or suggestions?

---

EDITS

P.S.: I'm transferring rather small files. Building up a connection is the biggest part of the transfer. This is the reason I want it to be done in ONE scp.

P.P.S.: I played around a little and came up with:

my_scp_multi3 "user" "server" "pw" "~/a\ b/testfileA, ~/a\\ b/testfileB, ~/a\\\ b/testfileC" "~/test"

with your first solution but {*}[split $files ","] and

my_scp_multi2 "user" "server" "pw" "~/a b/testfileA" "~/a\ b/testfileB" "~/a\\ b/testfileC" "~/test"

with your second solution. This prints:

~/a b/testfileA: No such file or directory
~/a\ b/testfileB: No such file or directory
~/a\ b/testfileC: No such file or directory

and

~/a b/testfileA: No such file or directory
~/a b/testfileB: No such file or directory
~/a\ b/testfileC: No such file or directory

(BTW: I of course moved the files :) )

---

Thanks to all the answers, here my Solution:

using \n \0 (nullbyte) as separator, because it is the only symbol except / and \ which may not be used in filenames.

#!/usr/bin/expect -f

    # Escapes spaces in a text
    proc esc text {
        return [regsub -all {\ } $text {\\&}]
    }

# Returns the absolute Filepath
proc makeAbsolute {pathname} {
    file join [pwd] $pathname
}

proc addUploadFile {files f} {
    if {$files != ""} {
        set files "$files\0"
    }
    return "$files[makeAbsolute $f]"
}

#Counts all files from an upload-list
proc countUploadFiles {s} {
        set rc [llength [split $s "\0"]] 
        incr rc -1
        return $rc
 }

# Uploads several files from a list (created by addUploadFile) to a specified server
proc my_scp_multi {ACCOUNT SERVER PW files newfolder} {
    foreground blue
    set nFiles [countUploadFiles $files]
    set timeout [expr $nFiles * 60]
        send_user -- "\n"
        spawn scp -r {*}[split $files "\0"] $ACCOUNT@$SERVER:[esc $newfolder]
        match_max 100000
        # Look for password prompt
        expect {
        -re ".*Connection closed.*" {
            sendError "\n\n\nUpload failed!\nPlease check the errors above and start over again.\nThis is most likely induced by too many wrong password-attempts and will last quite a time!"
        }
        -re ".*Permission denied.*" {
            sendError "\n\n\nUpload failed!\nPlease check the errors above and start over again.\nYou entered most likely a wrong password!"
        }
        -re ".*Are.*.*yes.*no.*" {
            send "yes\n"
            exp_continue
            #look for the password prompt
        }
        -re ".*sword.*" {
            # Send password aka $PW
            send -- "$PW\r"
            # send blank line (\r) to make sure we get back to gui
            send -- "\r\n"
            exp_continue
        }

        send_user -- "Upload successful!\n"
        }

        set timeout -1
    }



set fls [addUploadFile "" "a b/testfileA"]
set fls [addUploadFile $fls "a b/testfileB"]
set fls [addUploadFile $fls "a b/testfileC"]

my_scp_multi "user" "server" "pw" $fls "~/test"

Answer 1

You could spawn a shell and then run the scp command instead:

spawn bash
send "scp $files $ACCOUNT@$SERVER:[esc $newfolder]\r"

This allows for glob expansion but adds extra housekeeping as you will need to trap when the scp process is completed, as you still have a shell running. You could add below to your expect block:

-re "100%" {
    if { $index < $count } {
        set index [expr $index + 1]
        exp_continue
    }
}

Where index is the # of file being transferred and count the nr of files.

Answer 2

You should be using SSH public key authentication instead of typing in the password with expect. When it's set up properly, scp will work without any human input of passwords while keeping the system very secure. You will be free from all the troubles with expect.

• How do I setup Public-Key Authentication?
• http://www.ece.uci.edu/~chou/ssh-key.html

If there's some reason why you cannot use pubkey, you may find sftp useful because it accepts a batch command file as -b batchfile. See man 1 sftp Not a very good solution when expect can actually split the arguments

Answer 3

You don't want to send the filenames as a single string. Either do this:

spawn scp {*}[split $files] $ACCOUNT@$SERVER:[esc $newfolder]

And continue to quote the filenames:

my_scp_multi "ACCOUNT" "SERVER" "PW" "~/testfileA ~/testfileB ~/testfileC" "~/test/"

or do this:

proc my_scp_multi {ACCOUNT SERVER PW args} {
    set timeout 30
    send_user -- "\n"
    set files [lrange $args 0 end-1]
    set newfolder [lindex $args end]
    spawn scp {*}$files $ACCOUNT@$SERVER:[esc $newfolder]

And then do not quote the filenames

my_scp_multi "ACCOUNT" "SERVER" "PW" ~/testfileA ~/testfileB ~/testfileC "~/test/"

The splat ({*}) splits the list up into it's individual elements so the spawn command sees several words, not a single word. See http://tcl.tk/man/tcl8.5/TclCmd/Tcl.htm