CODEWITHSUNDEEP
linuxbashshell
5k1zk17
5k1zk17

Reputation: 389

Bash script wrapper for tar command takes a very long time to execute

I'm attempting to extract specific files of interest from a tar file (~50Gb). When I manually extract the file from the command line it takes a very short amount of time (1-2s). When I attempt to process it via a bash script to automate the specific files to extract, the script hangs at the tar extraction. I know it hangs here as I can kill the tar process and the script continues. The file gets extracted as I can "view" the file and contents while the script is still waiting for the tar process to return.

Here's a snippet of the code:

for line in $contents
do
    if [ -f $line ]; then
            /bin/tar xf $tarfile -C $tmp $line
    fi
done

I attempted to create a subshell for just the tar extraction but then I had 100s of hanging tar processes. Any thoughts as to why this is the case?

Thanks!!

Upvotes: 1

Views: 784

Answers (2)

Tripp Kinetics
Tripp Kinetics

Reputation: 5449

Try:

echo $contents | xargs find $start_directory -type f -name | xargs tar xvf $tarfile -C $tmp

Which (at least I think) adds your criterion for only extracting a file if it already exists and is a regular file.

Upvotes: 1

Barmar
Barmar

Reputation: 781751

You can supply multiple filename arguments to tar when extracting, so do:

/bin/tar xf $tarfile -C $tmp $contents

Your way has to search through the tarfile for each file being extracted. If it takes 1-2 seconds to extract a file, and you want to extract 50 files, that will take 50-100 seconds.

By supplying all the filenames at once, it scans through the tarfile just once, and extracts each file as it's encountered.

Upvotes: 3

Related Questions