CODEWITHSUNDEEP
rubygzip
Max Williams
Max Williams

Reputation: 32933

Call File.readlines (or equivalent) on zipped files WITHOUT unzipping them first

I'm trying to write a rake task to scan through some zipped up log files and extract some data that i can then use to update some database records. Because of space limitations on our server i can't unzip all of the files and just call File.readlines on them which is what i would normally do.

Can anyone suggest an alternative approach that doesn't involve unzipping the files? What i need to do is basically scan for lines matching a particular regex, then extract some text from the line AFTER the matched line. They are all gzipped, and are therefore .gz files.

Upvotes: 0

Views: 137

Answers (1)

rdvdijk
rdvdijk

Reputation: 4398

You can open the gzipped file using Ruby, and perform your searching as if you had opened the file as normal, like this:

require 'zlib'

Zlib::GzipReader.open("myfile.gz") { |gz|
  # place your code here, something like:
  p gz.readlines.grep /my-magic-marker/
}

This will load the entire file in memory, beware of that.

Upvotes: 1

Related Questions