CODEWITHSUNDEEP
linuxbashshell
Alex
Alex

Reputation: 7688

Get the number of non existing files

I have wrote this bash script to automate the process of finding the files that are missing:

echo "--|";
echo "  |";
read -p "  ->Enter the text file that contains the video files to look for: " fname
if [ ! -e "$fname" ]; then
        echo "  |";
        echo "--> The file $fname is not valid or doesn't exist";
else
        echo "  |";
        read -p "  ->File ready to be processed, enter the path where to look for the files: [/home/efeikuna/public_html/files/flv/]": path
        if [ ! -e "$path" ]; then
                path="/home/efeikuna/public_html/files/flv/";
        fi
                i=0
                for line in `cat $fname`;
                do
                        file=$path$line;
                        #echo "   |";
                        if [ ! -e $file ]; then
                                echo "   -> $i - $file => DOES NOT EXIST";
                        fi
                        i=$(expr $i + 1);
                        #if [ $i == 3 ]; then
                        #       break;
                        #fi
                done
fi

The script is working, but it needs some more features, so I was hopping you can guide/point me to the right way:

Thanks and sorry for any possible misunderstandings

Upvotes: 0

Views: 229

Answers (2)

hungnv
hungnv

Reputation: 150

.I need to get the total number of non existing files and the number of the existing ones With this, I use python instead of bash, actually it can be done with bash, but it found python is more efficient for this purpose: 

 #!/usr/bin/env python

 import os
 import sys

 def check_file(directory, filelist):
     is_file = []
     not_file = []
     with open(filelist) as f:
         for filename in f:
             fp = directory + '/' + filename.strip()
             print fp
             if os.path.isfile(fp): is_file.append(filename.strip())
             else: not_file.append(filename.strip())
     print "Number of file: %s" %(len(is_file))
     for item in is_file: print "----- %s" %(item)
     print "Number of non file: %s" %(len(not_file))
     for item in not_file: print "----- %s" %(item)

 if __name__ == '__main__':
     if len(sys.argv) < 3:
         print "Usage: %s [dir] [listfile]" %(sys.argv[0])
         sys.exit(2)
     directory = sys.argv[1]
     filelist  = sys.argv[2]
     check_file(directory, filelist)

What do you recommend to use to check the integrity of the file (usually flv files and txt files, but will be other in the future)

md5sum is the right way, but it can be very slow if you have more and large files. Use inotify instead. With inotify, you will be notified when a file was updated.

If the if [ ! -e $file ] enters, how can I match $fname with an locate $fname just to see if the file exists some where else? If locate $fname returns true, than show where is located, if not, continue.

Here is another python way:

for path, dirs, files in os.walk(directory):
     for f in [os.path.abspath(os.path.join(path, filename)) for filename in files]:
         fn = filename.split['/'][-1]
         if fn == file_you_want_to_search_for:
             print filename

Hope this help.

Upvotes: 1

Ignacio Vazquez-Abrams
Ignacio Vazquez-Abrams

Reputation: 798606

md5sum (and sha1sum) produces a file that contains checksums. Simply parse the result of running md5sum -c against the file containing the checksums to see how many files are valid, invalid, or missing.

Upvotes: 1

Related Questions