CODEWITHSUNDEEP
bashloopsfor-loop
SatelliteEyes
SatelliteEyes

Reputation: 75

Loop over directories and act on files in bash script

I have a script, /home/user/me/my_script.sh that is supposed to loop over multiple directories and process files. My current working directory is /home/user/me. A call to ls -R yields:

./projects:

dir1 dir2 dir3

./projects/dir1:

image1.ntf points2.csv image1.img image1.hdr

./projects/dir2:

image2.ntf points2.csv image2.img image2.hdr

./projects/dir3:

image3.ntf points3.csv image3.img image3.hdr

I have this script:

#! /bin/bash -f
for $dir in $1*
do
  echo $dir
  set cmd = `/home/tools/tool.sh -i $dir/*.ntf -flag1 -flag2 -flag3 opt3`
  $cmd
done

This is how it is run (from cwd /home/user/me) and the result:

bash-4.1$ ./myscript.sh projects/
projects/*
bash-4.1$

This is not the expected output.The expected output is:

bash-4.1$ ./myscript.sh projects/
projects/dir1
[output from tool.sh]
projects/dir2
[output from tool.sh]
projects/dir3
[output from tool.sh]
bash-4.1$

What should happen is the script should go into the first directory, find the *.ntf file and pass it to tool.sh. At that point I would start seeing output from that tool. I have run the tool on a single file:

bash-4.1$ /home/tools/tool.sh -i /home/user/me/projects/dir1/image1.ntf -flag1 -flag2 -flag3 opt3
[expected output from tool. lengthy.]
bash-4.1$

I have tried syntax found here: How to loop over directories in Linux? and here: Looping over directories in Bash

for $dir in /$1*/
do ...

Result:

bash-4.1$ ./myscript.sh projects/
/projects/*/

And:

for $dir in $1/*
do ...

Result:

bash-4.1$ ./myscript.sh projects
projects/*

I'm not sure how many other iterations of wildcard and slash I can come up with. What is the correct syntax?

Upvotes: 0

Views: 630

Answers (1)

ErikMD
ErikMD

Reputation: 14723

First, you should remove flag -f in your shebang, because it utterly means:

$ man bash
[…]
        -f      Disable pathname expansion.

Second, there are some typical bug patterns: spaces missing around variables (write "$dir" to cope with directory names containing spaces), there is a spurious $ in your for line (write for dir in "$1"*) instead, the set line is incorrect (set is a shell builtin only used to change the configuration of the shell, e.g., set -x), according to your answer to @ghoti's question it seems that the $cmd line is unnecessary. Also, the backquotes syntax is deprecated and could have been replaced with cmd=$(/home/tools/tool.sh -i "$dir"/*.ntf -flag1 -flag2 -flag3 opt3).

This would lead to the following script:

#!/bin/bash
for dir in "$1"*
do
  [[ -d "$dir" ]] || continue  # only consider existing folders
  printf "%s=%q\n" dir "$dir"
  /home/tools/tool.sh -i "$dir"/*.ntf -flag1 -flag2 -flag3 opt3
done

As an aside, I would recommend to always run the ShellCheck static analyzer on your Bash scripts, in order to detect typical bugs and have feedback w.r.t. good practices. If you have a Linux distribution, it should be installable with your standard package manager.

Upvotes: 2

Related Questions