CODEWITHSUNDEEP
linuxperlcommand-linesedawk
rockstardev
rockstardev

Reputation: 13527

Linux command to replace string in LARGE file with another string

I have a huge SQL file that gets executed on the server. The dump is from my machine and in it there are a few settings relating to my machine. So basically, I want every occurance of "c://temp" to be replace by "//home//some//blah"

How can this be done from the command line?

Upvotes: 8

Views: 17171

Answers (7)

ghostdog74
ghostdog74

Reputation: 342323

gawk

awk '{gsub("c://temp","//home//some//blah")}1' file

Upvotes: 1

Logan
Logan

Reputation: 267

perl -pi -e 's#c://temp#//home//some//blah#g' yourfilename

The -p will treat this script as a loop, it will read the specified file line by line running the regex search and replace.

-i This flag should be used in conjunction with the -p flag. This commands Perl to edit the file in place.

-e Just means execute this perl code.

Good luck

Upvotes: 1

Vinko Vrsalovic
Vinko Vrsalovic

Reputation: 340191

sed is a good choice for large files. 

sed -i.bak -e 's%C://temp%//home//some//blah%' large_file.sql

It is a good choice because doesn't read the whole file at once to change it. Quoting the manual:

A stream editor is used to perform basic text transformations on an input stream (a file or input from a pipeline). While in some ways similar to an editor which permits scripted edits (such as ed), sed works by making only one pass over the input(s), and is consequently more efficient. But it is sed's ability to filter text in a pipeline which particularly distinguishes it from other types of editors.

The relevant manual section is here. A small explanation follows

-i.bak enables in place editing leaving a backup copy with .bak extension

s%foo%bar% uses s, the substitution command, which substitutes matches of first string in between the % sign, 'foo', for the second string, 'bar'. It's usually written as s// but because your strings have plenty of slashes, it's more convenient to change them for something else so you avoid having to escape them.

Example

vinko@mithril:~$ sed -i.bak -e 's%C://temp%//home//some//blah%' a.txt
vinko@mithril:~$ more a.txt
//home//some//blah
D://temp
//home//some//blah
D://temp
vinko@mithril:~$ more a.txt.bak
C://temp
D://temp
C://temp
D://temp

Upvotes: 31

Meredith L. Patterson
Meredith L. Patterson

Reputation: 4911

There's also a non-standard UNIX utility, rpl, which does the exact same thing that the sed examples do; however, I'm not sure whether rpl operates streamwise, so sed may be the better option here.

Upvotes: 3

Paul Dixon
Paul Dixon

Reputation: 300825

Try sed? Something like:

sed 's/c:\/\/temp/\/\/home\/\/some\/\/blah/' mydump.sql > fixeddump.sql

Escaping all those slashes makes this look horrible though, here's a simpler example which changes foo to bar.

sed 's/foo/bar/' mydump.sql > fixeddump.sql

As others have noted, you can choose your own delimiter, which would prevent the leaning toothpick syndrome in this case:

sed 's|c://temp\\|home//some//blah|' mydump.sql > fixeddump.sql

The clever thing about sed is that it operating on a stream rather than a file all at once, so you can process huge files using only a modest amount of memory.

Upvotes: 4

ire_and_curses
ire_and_curses

Reputation: 70142

Just for completeness. In place replacement using perl.

perl -i -p -e 's{c://temp}{//home//some//blah}g' mysql.dmp

No backslash escapes required either. ;)

Upvotes: 12

stefanw
stefanw

Reputation: 10570

The sed command can do that. Rather than escaping the slashes, you can choose a different delimiter (_ in this case):

sed -e 's_c://temp/_/home//some//blah/_' file1.txt > file2.txt

Upvotes: 1

Related Questions