CODEWITHSUNDEEP
linuxexpectfile-transferscp
Hasan Rumman
Hasan Rumman

Reputation: 597

SCP is not transferring whole data when used with expect

I'm trying to transfer data(archived/compressed) from ServerA to ServerB and check size of data in both servers if all data is transferred. My script is given below.

Below script decompresses data in ServerB and checkes data sizes(to be used later)

#!/bin/bash
#Decompresses file and checks sizes
cd /home
touch listB
tar -zxvf datafiles.tar.gz
du -sh datafiles* > listB

Below script creates data files, archive/compress and then sends to ServerB

#!/bin/bash
#File process and transfer

#Remote server IP and access
read -p "Enter ServerB IP:" serverBIP
read -p "Enter ServerB Port:" serverBSSHPort
read -p "Enter ServerB Password:" serverBPass

startTime=$(date +%s)

#Processing data
cd /home/
seq 10000000 > /home/datafile1
seq 11110000 > /home/datafile2
mkdir datafiles
cp -r datafile1 datafile2 datafiles
tar -czvf datafiles.tar.gz datafiles
du -sh datafiles* > listA

checker=/home/checker
data=datafiles.tar.gz

export serverBIP
export serverBSSHPort
export serverBPass
export checker
export data

#File transfer to remote server
    /usr/bin/expect <(cat <<-'EOF'

    spawn scp -r -P $::env(serverBSSHPort) $::env(checker) $::env(serverBIP):/home
    expect "assword:"
    send "$::env(serverBPass)\r"

    spawn scp -r -P $::env(serverBSSHPort) $::env(data) $::env(serverBIP):/home
    expect "assword:"
    send "$::env(serverBPass)\r"

    spawn ssh $::env(serverBIP) -p $::env(serverBSSHPort) "bash -s" < $::env(checker)
    expect "assword:"
    send "$::env(serverBPass)\r"

    interact

    EOF
    )

finishTime=$(date +%s)
timeElapsed=$((finishTime - startTime))
minute=$((timeElapsed / 60))
sec=$((timeElapsed % 60))
echo It took $minute min $sec sec to complete the task

This is working for small data sizes like below 20 MB but when the data size is big(even 5o MB) all data are not being transferred and giving a giving error:

gzip: stdin: unexpected end of file
tar: Unexpected EOF in archive
tar: Unexpected EOF in archive
tar: Error is not recoverable: exiting now

This error is showing probably because the archived/compressed data from ServerA was not transferred to ServerB.

Could anyone please tell me what is the exact cause and how can I solve this? I've also tried using rsync but getting syntax errors. I need to automate this task(don't want to give IP, passwords everytime I do something on remote server) that's why I'm using expect and I can't use public key authentication for some reasons. I have checked transferring bigger data(over 2 GB) using scp and rsync between ServerA and ServerB and no issue occurred except when I try to use expect.

Upvotes: 0

Views: 1449

Answers (1)

pynexj
pynexj

Reputation: 20798

You need to wait for every spawn'ed process to complete or the process may be killed before it completes. So you can add an interact for each spawn:

spawn ...
...
interact

spawn ...
...
interact

or

spawn ...
...
expect -timeout -1 eof

spawn ...
...
expect -timeout -1 eof

Upvotes: 1

Related Questions