CODEWITHSUNDEEP
rubyfile
B Seven
B Seven

Reputation: 45943

How to efficiently iterate and update a large string in Ruby?

I have a text file that is 6GB. I want to do something like:

str.gsub!('xxx', 'x')

The idea was to read 1MB chunks using seek. Is there a way to do the replacement above more efficiently? Perhaps iterating over the string using a C-like array access?

Upvotes: 3

Views: 122

Answers (2)

Anthony
Anthony

Reputation: 15967

You can do it with a child process using sed which will be very fast:

`sed -i -E 's:xxx:x:g' file_name`

Upvotes: 3

Eric Duminil
Eric Duminil

Reputation: 54233

If the huge file has more than say, 20 lines, you could use :

File.open('new_file', 'w') do |out|
  File.foreach('huge_file.txt') do |line|
    out.puts line.gsub('xxx', 'x')
  end
end

This will have a very low memory footprint and should be reasonably fast.

Upvotes: 2

Related Questions