CODEWITHSUNDEEP
regexlinuxbashsed
170730350
170730350

Reputation: 622

Remove duplicate filename extensions

I have thousands of files named something like filename.gz.gz.gz.gz.gz.gz.gz.gz.gz.gz.gz

I am using the find command like this find . -name "*.gz*" to locate these files and either use -exec or pipe to xargs and have some magic command to clean this mess, so that I end up with filename.gz

Someone please help me come up with this magic command that would remove the unneeded instances of .gz. I had tried experimenting with sed 's/\.gz//' and sed 's/(\.gz)//' but they do not seem to work (or to be more honest, I am not very familiar with sed). I do not have to use sed by the way, any solution that would help solve this problem would be welcome :-)

Upvotes: 2

Views: 1361

Answers (8)

lind
lind

Reputation: 2269

Another way with rename:

find . -iname '*.gz.gz' -exec rename -n 's/(\.\w+)\1+$/$1/' {} +

When happy with the results remove -n (dry-run) option.

Upvotes: 0

Joseph Quinsey
Joseph Quinsey

Reputation: 9972

Using bash string substitution:

for f in *.gz.gz; do
    mv "$f" "${f%%.gz.gz*}.gz"
done

This is a slight modification of jaypal's nice answer (which would fail if any of your files had a period as part of its name, such as foo.c.gz.gz). (Mine is not perfect, either) Note the use of double-quotes, which protects against filenames with "bad" characters, such as spaces or stars.

If you wish to use find to process an entire directory tree, the variant is:

find . -name \*.gz.gz | \
while read f; do
    mv "$f" "${f%%.gz.gz*}.gz"
done

And if you are fussy and need to handle filenames with embedded newlines, change the while read to while IFS= read -r -d $'\0', and add a -print0 to find; see How do I use a for-each loop to iterate over file paths output by the find utility in the shell / Bash?.

But is this renaming a good idea? How was your filename.gz.gz created? gzip has guards against accidentally doing so. If you circumvent these via something like gzip -c $1 > $1.gz, buried in some script, then renaming these files will give you grief.

Upvotes: 0

mklement0
mklement0

Reputation: 439597

find . -name "*.gz.gz" | 
 while read f; do echo mv "$f" "$(sed -r 's/(\.gz)+$/.gz/' <<<"$f")"; done

This only previews the renaming (mv) command; remove the echo to perform actual renaming.

  • Processes matching files in the current directory tree, as in the OP (and not just files located directly in the current directory).
  • Limits matching to files that end in at least 2 .gz extensions (so as not to needlessly process files that end in just one).
  • When determining the new name with sed, makes sure that substring .gz doesn't just match anywhere in the filename, but only as part of a contiguous sequence of .gz extensions at the end of the filename.
  • Handles filenames with special chars. such as embedded spaces correctly (with the exception of filenames with embedded newlines.)

Upvotes: 0

potong
potong

Reputation: 58488

This might work for you (GNU sed):

echo *.gz | sed -r 's/^([^.]*)(\.gz){2,}$/mv -v & \1\2/e'

Upvotes: 0

jaypal singh
jaypal singh

Reputation: 77155

You can do that with bash string substitution: 

for file in *.gz.gz; do
    mv "${file}" "${file%%.*}.gz"
done

Upvotes: 0

Kent
Kent

Reputation: 195209

one way with find and awk:

find $(pwd) -name '*.gz'|awk '{n=$0;sub(/(\.gz)+$/,".gz",n);print "mv",$0,n}'|sh

Note:

  • I assume there is no special chars (like spaces...) in your filename. If there were, you need quote the filename in mv command.
  • I added a $(pwd) to get the absolute path of found name.
  • you can remove the ending |sh to check generated mv ... .... cmd, if it is correct.
  • If everything looks good, add the |sh to execute the mv

see example here:

enter image description here

Upvotes: 4

Jakub M.
Jakub M.

Reputation: 33857

ls *.gz | perl -ne '/((.*?.gz).*)/; print "mv $1 $2\n"'

It will print shell commands to rename your files, it won't execute those commands. It is safe. To execute it, you can save it to file and execute, or simply pipe to shell:

ls *.gz | ... | sh

sed is great for replacing text inside files.

Upvotes: 0

grodzi
grodzi

Reputation: 5703

You may use

ls a.gz.gz.gz |sed -r 's/(\.gz)+/.gz/'

or without the regex flag

ls a.gz.gz.gz |sed 's/\(\.gz\)\+/.gz/'

Upvotes: 0

Related Questions