CODEWITHSUNDEEP
shellscp
jimko
jimko

Reputation:

Automate scp file transfer using a shell script

I have some n number of files in a directory on my unix system. Is there a way to write a shellscript that will transfer all those files via scp to a specified remote system. I'll specify the password within the script, so that I don't have to enter it for each file.

Upvotes: 97

Views: 427762

Answers (14)

Software Developer
Software Developer

Reputation: 868

There are 2 quick ways of achieving this:

  1. Using scp

    #!/usr/bin/env bash

password="YOURPASSWORD"
username="YOURUSERNAME"
dir_origin="YOURSOURCEDIRECTORY"
dir_destination="REMOTEDESTINATION"
Ip="SERVERIP"

echo "Uploading files to remote server...."
sshpass -p "$password" scp -rC $dir_origin $username@$Ip:$dir_destination
echo "File upload to remote server completed! ;)"

  2. Using rsync

    #!/usr/bin/env bash

password="YOURPASSWORD"
username="YOURUSERNAME"
dir_origin="YOURSOURCEDIRECTORY"
dir_destination="REMOTEDESTINATION"
Ip="SERVERIP"

echo "Uploading files to remote server...."
sshpass -p "$password" rsync -avzh $dir_origin $username@$Ip:$dir_destination
echo "File upload to remote server completed! ;)"

**NOTE :**You need to install sshpass (eg by running apt install sshpass for deb like os eg Ubuntu) that will enable you to auto upload files without password prompts

Upvotes: 4

Pradeep Pati
Pradeep Pati

Reputation: 5919

Instead of hardcoding password in a shell script, use SSH keys, its easier and secure.

$ scp -i ~/.ssh/id_rsa *.derp [email protected]:/path/to/target/directory/

assuming your private key is at ~/.ssh/id_rsa and the files you want to send can be filtered with *.derp

To generate a public / private key pair :

$ ssh-keygen -t rsa

The above will generate 2 files, ~/.ssh/id_rsa (private key) and ~/.ssh/id_rsa.pub (public key)

To setup the SSH keys for usage (one time task) : Copy the contents of ~/.ssh/id_rsa.pub and paste in a new line of ~devops/.ssh/authorized_keys in myserver.org server. If ~devops/.ssh/authorized_keys doesn't exist, feel free to create it.

A lucid how-to guide is available here.

Upvotes: 105

Chief_Arbiter
Chief_Arbiter

Reputation: 19

here's bash code for SCP with a .pem key file. Just save it to a script.sh file then run with 'sh script.sh'

Enjoy

#!/bin/bash
#Error function
function die(){
echo "$1"
exit 1
}

Host=ec2-53-298-45-63.us-west-1.compute.amazonaws.com
User=ubuntu
#Directory at sent destination
SendDirectory=scp
#File to send at host
FileName=filetosend.txt
#Key file
Key=MyKeyFile.pem

echo "Aperture in Process...";

#The code that will send your file scp
scp -i $Key $FileName $User@$Host:$SendDirectory || \
die "@@@@@@@Houston we have problem"

echo "########Aperture Complete#########";

Upvotes: 1

azazil
azazil

Reputation: 51

Try lftp 

lftp -u $user,$pass sftp://$host << --EOF--

cd $directory

put $srcfile

quit

--EOF--

Upvotes: 1

veerendra2
veerendra2

Reputation: 2273

why don't you try this?

password="your password"
username="username"
Ip="<IP>"
sshpass -p "$password" scp /<PATH>/final.txt $username@$Ip:/root/<PATH>

Upvotes: 25

Rishi Sinha
Rishi Sinha

Reputation: 21

This will work:

#!/usr/bin/expect -f

spawn scp -o UserKnownHostsFile=/dev/null -o StrictHostKeyChecking=no file1 file2 file3 user@host:/path/
expect "password:"
send "xyz123\r"
expect "*\r"
expect "\r"
interact

Upvotes: 2

Felix Rabe
Felix Rabe

Reputation: 4286

If you are ok with entering your password once for every run of the script, you can do so easily using an SSH master connection.

#!/usr/bin/env bash

USER_AT_HOST="user@host"  # use "$1@$2" here if you like
SSHSOCKET=~/".ssh/$USER_AT_HOST"

# This is the only time you have to enter the password:
# Open master connection:
ssh -M -f -N -o ControlPath="$SSHSOCKET" "$USER_AT_HOST"

# These do not prompt for your password:
scp -o ControlPath="$SSHSOCKET" file1.xy "$USER_AT_HOST":remotefile1.xy
scp -o ControlPath="$SSHSOCKET" file2.xy "$USER_AT_HOST":remotefile2.xy

# You can also use the flag for normal ssh:
ssh -o ControlPath="$SSHSOCKET" "$USER_AT_HOST" "echo hello"
ssh -o ControlPath="$SSHSOCKET" "$USER_AT_HOST" "echo world"

# Close master connection:
ssh -S "$SSHSOCKET" -O exit "$USER_AT_HOST"

Upvotes: 6

JohnA
JohnA

Reputation: 1913

#!/usr/bin/expect -f

# connect via scp
spawn scp "[email protected]:/home/santhosh/file.dmp" /u01/dumps/file.dmp
#######################
expect {
  -re ".*es.*o.*" {
    exp_send "yes\r"
    exp_continue
  }
  -re ".*sword.*" {
    exp_send "PASSWORD\r"
  }
}
interact

http://blogs.oracle.com/SanthoshK/entry/automate_linux_scp_command

Upvotes: 44

K.Govind Anupam
K.Govind Anupam

Reputation: 61

rsync is a program that behaves in much the same way that rcp does, but has many more options and uses the rsync remote-update protocol to greatly speed up file transfers when the destination file is being updated.

The rsync remote-update protocol allows rsync to transfer just the differences between two sets of files across the network connection, using an efficient checksum-search algorithm described in the technical report that accompanies this package.

Copying folder from one location to another

   #!/usr/bin/expect -f   
   spawn rsync -a -e ssh [email protected]:/cool/cool1/* /tmp/cool/   
   expect "password:"   
   send "cool\r"   
   expect "*\r"   
   expect "\r"

Upvotes: 6

Dimitri
Dimitri

Reputation: 8290

The command scp can be used like a traditional UNIX cp. SO if you do :

scp -r myDirectory/ mylogin@host:TargetDirectory

will work

Upvotes: -1

Abhinav Gupta
Abhinav Gupta

Reputation: 101

#!/usr/bin/expect -f
spawn scp -r BASE.zip [email protected]:/tmp
expect "password:"
send "wifinetworks\r"
expect "*\r"
expect "\r"

Upvotes: 10

Vereb
Vereb

Reputation: 14736

You can do it with ssh public/private keys only. Or use putty in which you can set the password. scp doesn't support giving password in command line.

You can find the instructions for public/private keys here: http://www.softpanorama.org/Net/Application_layer/SSH/scp.shtml

Upvotes: 2

bsisco
bsisco

Reputation: 809

you could also use rsync. It seems to work better for multiple files than scp IMHO.

rsync -avzh /path/to/dir/ user@remote:/path/to/remote/dir/

Update

You can use rsync via ssh by adding the '-e' switch:

rsync -avzh -e ssh /path/do/dir/ user@remote:/path/to/remote/dir/

Upvotes: 20

lutz
lutz

Reputation:

What about wildcards or multiple files?

scp file1 file2 more-files* user@remote:/some/dir/

Upvotes: 4

Related Questions