Reputation: 5
i have this master script
>loanIds
>log
>count
mkdir -p duplicates
rm -rf duplicates/*
for file in x*
do
./getXmlData.sh $file &
done
Source for getXmlData.sh
cat $file | while read line
do
echo $file" | "$line
done
i have 40 files that start with letter x each x* file has 1000 lines.
when i run the script, as you can see it will create 40 instances in the background and process each x* file
The problem i have is, ./getXmlData.sh does not echo all 1000 lines for each of those 40 files... for some files it echos all 1000 lines.. for some of them it echos only 10-15 lines
What could be the issue?
Upvotes: 0
Views: 1563
Reputation: 2230
try this way ->
file=$1
chmod 755 $file
exec <$file
while IFS= read -r
do
echo "$file | $REPLY"
done
Or to avoid issues with "read" command, I came up with below logic which basically calculates the number of lines and prints it one by one using head/tail command ->
file=$1
lines=`wc -l $file | cut -d" " -f1`
for ((i=1;i<=$lines;i++));
do
line=`head -$i $file | tail -1`
echo "$file | $line"
done
Upvotes: 0
Reputation: 755114
Unless you've not shown an assignment in your getXmlData.sh
script, the problem is likely that you are passing $file
in the calling script but you need to reference it as $1
in the called script. That is, you should be writing:
cat "$1" |
while read -r line
do
echo "$1 | $line"
done
or (better — avoiding UUOC or Useless Use of cat
):
while read -r line
do
echo "$1 | $line"
done < "$1"
or something similar. Indeed, we could even suggest:
sed "s%^%$1 | %" "$1"
This assumes that the file name doesn't contain a newline or a percent (but allows the name to include slashes, hence the use of %
instead of /
in the s///
command). Note that the file name is enclosed in double quotes each time it is used; this protects your script from file names containing spaces, etc. Also note that the echo
encloses $line
(as well as $1
) in double quotes; this preserves the spacing in the data, and avoids other metacharacter expansions too. If you really wanted all sets of multiples spaces or tabs converted to a single space, then the original formulation was OK, but that is seldom the desired result.
One other possibility: maybe after the loop, you should wait for all the children to complete. You do that by telling the shell to:
wait
It won't progress further until its background processes terminate.
Upvotes: 3