CODEWITHSUNDEEP
bashwhile-loop
Carl Wainwright
Carl Wainwright

Reputation: 328

Bash script fails to read entire file

I have a bash script for backing up a set of folders/files. The script loops through a pre-configured file to determine how to copy the folder/file.

The problem I am facing is the script exits the while loop after processing the first "file" entry (2nd column value).

Here is the script.

#!/bin/bash

sudo crontab -l > /home/ipaccess/crontab.txt

DATE=`date +%m-%d-%Y`

TARGETDIR="/exports/backup/Web-Server/$DATE/"
BACKUPLIST="/home/ipaccess/backup/backuplist"
ssh 172.29.32.246 mkdir -p $TARGETDIR
scp -r /home/ipaccess/crontab.txt 172.29.32.246:$TARGETDIR

cat $BACKUPLIST

while read dir type args
do
   echo "Processing [$dir $type $args]"
   echo "Directory: $dir"
   echo "Element type: $type"

   if [ $type = "folder" ] ; then
        echo "Source: $args"
        echo "Transfering $args to 172.29.32.246:$TARGETDIR$dir"
        scp -r $args 172.29.32.246:$TARGETDIR
   fi
   if [ $type = "file" ] ; then
        # Create folders first
        echo "Creating $TARGETDIR$dir"
        ssh 172.29.32.246 mkdir -p $TARGETDIR$dir
        scp $args 172.29.32.246:$TARGETDIR$dir
   fi
   echo "Finished processing $dir $type $args"
   echo "Next entry.."
done<$BACKUPLIST
echo "Backup complete."

Here is the pre-configured file (backuplist)

html/GS folder /var/www/html/GS/css
html/GS folder /var/www/html/GS/images
html/GS file /var/www/html/GS/*.html
html/GS file /var/www/html/GS/*.inc
html/GS file /var/www/html/GS/*.png
html/GS file /var/www/html/GS/*.mysql
html/GS file /var/www/html/GS/ac-db
html/HC file /var/www/html/HC/*.php
html/HC file /var/www/html/HC/*.html
html/HC file /var/www/html/HC/images
html/HC file /var/www/html/HC/PM/*.php
html/HC/PM file /var/www/html/HC/PM/jsapi
html/HourlyKpi file /var/www/html/HourlyKpi/*.php
html/KPI folder /var/www/html/KPI
html/KpiQuery folder /var/www/html/KpiQuery
html/wordpress/ folder /var/www/html/wordpress
usr/local/apache2/conf file /usr/local/apache2/conf/httpd.conf
cgi-bin folder /var/www/cgi-bin
html/GS file /var/www/html/GS/*.php

Here is the output of the script

Processing [html/GS file /var/www/html/GS/*.html]
+ echo 'Directory: html/GS'
Directory: html/GS
+ echo 'Element type: file'
Element type: file
+ '[' file = folder ']'
+ '[' file = file ']'
+ echo 'Creating /exports/backup/Web-Server/04-18-2013/html/GS'
Creating /exports/backup/Web-Server/04-18-2013/html/GS
+ ssh 172.29.32.246 mkdir -p /exports/backup/Web-Server/04-18-2013/html/GS
+ scp /var/www/html/GS/gs.html /var/www/html/GS/login_header.html /var/www/html/GS/nologin_header.html /var/www/html/GS/overlaynal.html /var/www/html/GS/tab-test.html 172.29.32.246:/exports/backup/Web-Server/04-18-2013/html/GS
gs.html                                                                                            100% 3204     3.1KB/s   00:0
login_header.html                                                                                  100%  418     0.4KB/s   00:0
nologin_header.html                                                                                100%  464     0.5KB/s   00:0
overlay-external.html                                                                              100% 2308     2.3KB/s   00:0
tab-test.html                                                                                      100% 1573     1.5KB/s   00:0
+ echo 'Finished processing html/GS file /var/www/html/GS/*.html'
Finished processing html/GS file /var/www/html/GS/*.html
+ echo 'Next entry..'
Next entry..
+ read dir type args
+ echo 'Backup complete.'
Backup complete.

Upvotes: 0

Views: 1973

Answers (2)

Gordon Davisson
Gordon Davisson

Reputation: 125708

Something inside the loop (almost certainly ssh) is reading the remainder of the file during the first loop. You can redirect ssh's input (either with < /dev/null or its -n option), but it's slightly possible that something else in the loop will read from stdin (I don't think scp ever tries to read a password from stdin, but I could be wrong).

What I like to do in cases like this is to pass the file's contents to the read command via something other than stdin. I tend to use file descriptor #3 for this, since it's hardly ever used for anything else, and hence very unlikely that anything inside the loop will mess with it. To do this, just use 3<filename to feed the file to fd3, and read -u3 to read from it:

while read -u3 dir type args
do
    echo "Processing [$dir $type $args]"
    ...
done 3<$BACKUPLIST

Upvotes: 1

chepner
chepner

Reputation: 530882

The call to ssh in your loop reads from standard input, even though the command you give it doesn't use the standard input. Use the -n option:

ssh -n 172.29.32.246 mkdir -p $TARGETDIR$dir

to redirect standard input from /dev/null.

Upvotes: 3

Related Questions