CODEWITHSUNDEEP
arraysbashfor-loop
Esben86
Esben86

Reputation: 493

Problems with sed command in for loop

I am trying to make a bash script that does the following:

  1. Create an array of filepaths by recursively searching a dir and subdir's
  2. Loop through the array, creating a checksum for each file in the array and store the filepath and the file checksum in its own file.
  3. If the file is hashed previously, the old line should be removed and then replaced with an updated checksum. For each element, the filepath is searched for in the file where alle the filepaths with checksums are stored. If found, a filecount variable stores the amount of matches, and uses this variable to determine if the file should be replaced, or just added.

The problem is that the sed command that deletes the line that contains a filepath/checksum that has been stored previously does not work in a for loop. However, the same algorithm works if I pass a single file as an argument instead of creating a file array and looping throught it.

Code example below:

#! /bin/bash

file_array=$(find sdcard -type f)

for path in ${file_array[@]}
do
    hash=$(shasum $path | cut -f 1 -d " ")

    # Search through the file that stores hashed files (path and chksum)
    file_count=$(grep -c -w $path droidhash.csv)

    echo "filecount: $file_count"

    #If the file is found, preform commands based on this
    if [ $file_count -ge 1 ]
    then
        echo "File found, replacing line"
        sed -i "/"$path"/d" droidhash.csv
        echo $path';'$hash | cat >> droidhash.csv
    else
        echo "File not found, adding line"
        echo $path';'$hash | cat >> droidhash.csv
    fi
done

The error message i get when running the script is:

"sed: -e expression #1, char 9: unknown command: `f'"

Is it possible to use this sed command in a for loop?

Upvotes: 1

Views: 1379

Answers (2)

glenn jackman
glenn jackman

Reputation: 246837

To actually create an array, you need extra parentheses:

file_array=( $(find sdcard -type f) )

Then, it's very important to quote the array expansion

for path in "${file_array[@]}"

And then quote the path variable everywhere you use it

hash=$(shasum "$path" | cut -f 1 -d " ")

See Security implications of forgetting to quote a variable in bash/POSIX shells

Since you're grepping for a fixed string, tell grep that:

file_count=$(grep -F -c -w "$path" droidhash.csv)

And then follow @JH's advice: "$path" contains the slash character, so you need a different delimiter in sed's s/// command.

On second thought, I'd rewrite like this:

find sdcard -type f -print0 | while IFS= read -d "" -r path; do
    hash=$(shasum "$path" | cut -f 1 -d " ")
    if grep -q "^$path;" droidhash.csv; then
        echo "File $path found, replacing line"
        gawk -i inplace  '$1 == p {$2 = h}; 1'  FS=";" OFS=";" p="$path" h="$hash"  droidhash.csv
    else
        echo "File $path not found, adding line"
        echo "$path;$hash" >> droidhash.csv
    fi
done

Upvotes: 2

J_H
J_H

Reputation: 20460

    sed -i "/"$path"/d" droidhash.csv

Your quoting is odd. You may be happier with sed -i ";${path};d" droidhash.csv.

Rather than an else clause, just unconditionally append to the CSV. Without using cat, as echo can append just as well.

Upvotes: 1

Related Questions