CODEWITHSUNDEEP
bashshellsedfile-rename
Daniel Underwood
Daniel Underwood

Reputation: 1183

Using sed to mass rename files

Objective

Change these filenames:

to these filenames:

Shell Code

To test:

ls F00001-0708-*|sed 's/\(.\).\(.*\)/mv & \1\2/'

To perform:

ls F00001-0708-*|sed 's/\(.\).\(.*\)/mv & \1\2/' | sh

My Question

I don't understand the sed code. I understand what the substitution command

$ sed 's/something/mv'

means. And I understand regular expressions somewhat. But I don't understand what's happening here:

\(.\).\(.*\)

or here:

& \1\2/

The former, to me, just looks like it means: "a single character, followed by a single character, followed by any length sequence of a single character"--but surely there's more to it than that. As far as the latter part:

& \1\2/

I have no idea.

Upvotes: 118

Views: 215547

Answers (13)

ghostdog74
ghostdog74

Reputation: 342869

You've had your sed explanation. Now you can use just the shell. No need for external commands.

for file in F0000*
do
    echo mv "$file" "${file/#F0000/F000}"
    # ${file/#F0000/F000} means replace the pattern that starts at beginning of string
done

Note that this snippet runs echo as a safety measure that prints what the mv command will do without doing it. To actually perform the mv, you need to remove echo.

Upvotes: 59

Victoria Stuart
Victoria Stuart

Reputation: 5092

Some examples that work for me:

$ tree -L 1 -F .
.
├── A.Show.2020.1400MB.txt
└── Some Show S01E01 the Loreming.txt

0 directories, 2 files

## remove "1400MB" (I: ignore case) ...

$ for f in *; do mv 2>/dev/null -v "$f" "`echo $f | sed -r 's/.[0-9]{1,}mb//I'`"; done;
renamed 'A.Show.2020.1400MB.txt' -> 'A.Show.2020.txt'

## change "S01E01 the" to "S01E01 The"
## \U& : change (here: regex-selected) text to uppercase;
##       note also: no need here for `\1` in that regex expression

$ for f in *; do mv 2>/dev/null "$f" "`echo $f | sed -r "s/([0-9] [a-z])/\U&/"`"; done

$ tree -L 1 -F .
.
├── A.Show.2020.txt
└── Some Show S01E01 The Loreming.txt

0 directories, 2 files
$

Upvotes: 0

Gilles Quénot
Gilles Quénot

Reputation: 185620

Using perl rename (a must have in the toolbox):

rename -n 's/0000/000/' F0000*

Remove -n switch when the output looks good to rename for real.

warning There are other tools with the same name which may or may not be able to do this, so be careful.

The rename command that is part of the util-linux package, won't.

If you run the following command (GNU)

$ rename

and you see perlexpr, then this seems to be the right tool.

If not, to make it the default (usually already the case) on Debian and derivative like Ubuntu :

$ sudo apt install rename
$ sudo update-alternatives --set rename /usr/bin/file-rename

For archlinux:

pacman -S perl-rename

For RedHat-family distros:

yum install prename

The 'prename' package is in the EPEL repository.

For Gentoo:

emerge dev-perl/rename

For *BSD:

pkg install gprename

or p5-File-Rename

For Mac users:

brew install rename

If you don't have this command with another distro, search your package manager to install it or do it manually:

cpan -i File::Rename

Old standalone version can be found here

man rename

This tool was originally written by Larry Wall, the Perl's dad.

Upvotes: 9

user3164360
user3164360

Reputation: 31

for i in *; do mv $i $(echo $i|sed 's/AAA/BBB/'); done

Upvotes: 2

Chris Po
Chris Po

Reputation: 11

Here's what I would do:

for file in *.[Jj][Pp][Gg] ;do 
    echo mv -vi \"$file\" `jhead $file|
                           grep Date|
                           cut -b 16-|
                           sed -e 's/:/-/g' -e 's/ /_/g' -e 's/$/.jpg/g'` ;
done

Then if that looks ok, add | sh to the end. So:

for file in *.[Jj][Pp][Gg] ;do 
    echo mv -vi \"$file\" `jhead $file|
                           grep Date|
                           cut -b 16-|
                           sed -e 's/:/-/g' -e 's/ /_/g' -e 's/$/.jpg/g'` ;
done | sh

Upvotes: 1

Mike
Mike

Reputation: 261

The easiest way would be:

for i in F00001*; do mv "$i" "${i/F00001/F0001}"; done

or, portably,

for i in F00001*; do mv "$i" "F0001${i#F00001}"; done

This replaces the F00001 prefix in the filenames with F0001. credits to mahesh here: http://www.debian-administration.org/articles/150

Upvotes: 26

Edward Anderson
Edward Anderson

Reputation: 13926

First, I should say that the easiest way to do this is to use the prename or rename commands.

On Ubuntu, OSX (Homebrew package rename, MacPorts package p5-file-rename), or other systems with perl rename (prename):

rename s/0000/000/ F0000*

or on systems with rename from util-linux-ng, such as RHEL:

rename 0000 000 F0000*

That's a lot more understandable than the equivalent sed command.

But as for understanding the sed command, the sed manpage is helpful. If you run man sed and search for & (using the / command to search), you'll find it's a special character in s/foo/bar/ replacements.

  s/regexp/replacement/
         Attempt  to match regexp against the pattern space.  If success‐
         ful,  replace  that  portion  matched  with  replacement.    The
         replacement may contain the special character & to refer to that
         portion of the pattern space  which  matched,  and  the  special
         escapes  \1  through  \9  to refer to the corresponding matching
         sub-expressions in the regexp.

Therefore, \(.\) matches the first character, which can be referenced by \1. Then . matches the next character, which is always 0. Then \(.*\) matches the rest of the filename, which can be referenced by \2.

The replacement string puts it all together using & (the original filename) and \1\2 which is every part of the filename except the 2nd character, which was a 0.

This is a pretty cryptic way to do this, IMHO. If for some reason the rename command was not available and you wanted to use sed to do the rename (or perhaps you were doing something too complex for rename?), being more explicit in your regex would make it much more readable. Perhaps something like:

ls F00001-0708-*|sed 's/F0000\(.*\)/mv & F000\1/' | sh

Being able to see what's actually changing in the s/search/replacement/ makes it much more readable. Also it won't keep sucking characters out of your filename if you accidentally run it twice or something.

Upvotes: 179

Guy
Guy

Reputation: 2042

I wrote a small post with examples on batch renaming using sed couple of years ago:

http://www.guyrutenberg.com/2009/01/12/batch-renaming-using-sed/

For example:

for i in *; do
  mv "$i" "`echo $i | sed "s/regex/replace_text/"`";
done

If the regex contains groups (e.g. \(subregex\) then you can use them in the replacement text as \1\,\2 etc.

Upvotes: 50

ghostdog74
ghostdog74

Reputation: 342869

 ls F00001-0708-*|sed 's|^F0000\(.*\)|mv & F000\1|' | bash

Upvotes: 0

Dennis Williamson
Dennis Williamson

Reputation: 360545

If all you're really doing is removing the second character, regardless of what it is, you can do this:

s/.//2

but your command is building a mv command and piping it to the shell for execution.

This is no more readable than your version:

find -type f | sed -n 'h;s/.//4;x;s/^/mv /;G;s/\n/ /g;p' | sh

The fourth character is removed because find is prepending each filename with "./".

Upvotes: 1

paxdiablo
paxdiablo

Reputation: 882326

The sed command

s/\(.\).\(.*\)/mv & \1\2/

means to replace:

\(.\).\(.*\)

with:

mv & \1\2

just like a regular sed command. However, the parentheses, & and \n markers change it a little.

The search string matches (and remembers as pattern 1) the single character at the start, followed by a single character, follwed by the rest of the string (remembered as pattern 2).

In the replacement string, you can refer to these matched patterns to use them as part of the replacement. You can also refer to the whole matched portion as &.

So what that sed command is doing is creating a mv command based on the original file (for the source) and character 1 and 3 onwards, effectively removing character 2 (for the destination). It will give you a series of lines along the following format:

mv F00001-0708-RG-biasliuyda F0001-0708-RG-biasliuyda
mv abcdef acdef

and so on.

Upvotes: 8

Ewan Todd
Ewan Todd

Reputation: 7312

The parentheses capture particular strings for use by the backslashed numbers.

Upvotes: 0

Pointy
Pointy

Reputation: 413966

The backslash-paren stuff means, "while matching the pattern, hold on to the stuff that matches in here." Later, on the replacement text side, you can get those remembered fragments back with "\1" (first parenthesized block), "\2" (second block), and so on.

Upvotes: 3

Related Questions