CODEWITHSUNDEEP
rubyubuntu
moon.musick
moon.musick

Reputation: 5664

Specify a non-standard temporary directory for open() method in Ubuntu

Is there a way to specify a path to use as a temporary directory for the open() method (while using open-uri)?

I am using Ubuntu 12.04 and Ruby 2.0.0 with RVM - it uses the standard system temp dir (/tmp). As I am downloading large files, it takes a while before the file gets pulled off the web and moved into the target directory; also, I do not want to fill the /tmp filesystem up.

I know there are other methods of downloading files, some allowing to write data in chunks, I am just asking about the standard open method with open-uri.

The code I use is:

['ftp://example1.com/a.gz', 'ftp://example2.com/b', 'example3.com/somefile'].
  each do |uri|
  thread = Thread.new do
    3.times do 
      File.open(uri.split(/\//)[-1], "wb") do |file|
          file.write open(uri).read
      end
    end
  end

  threads << thread
end

(the point of all this is I am to make a script pulling large files out of arbitrary sites as a way of saturating the network link in order to check the throughput).

Upvotes: 2

Views: 1168

Answers (2)

mdesantis
mdesantis

Reputation: 8517

Looking at the OpenURI source code we can see that it uses Tempfile:

[...]
io = Tempfile.new('open-uri')
[...]

Tempfile in order to choose the temporary directory uses Dir.tmpdir, which in turn uses the system temporary directory or a directory specified by the environment variable TMPDIR (between others). So we can write something like this:

require 'open-uri'
require 'fileutils'

d = "#{Dir.home}/.tmp"
Dir.mkdir d
ENV["TMPDIR"] = d
p open("http://www.google.com")
ENV.delete("TMPDIR")
FileUtils.rm_rf d

In a single command (please ensure that $HOME/.tmpdoes not exist and is not used):

ruby -ropen-uri -rfileutils -e 'd = "#{Dir.home}/.tmp"; Dir.mkdir d; ENV["TMPDIR"] = d; p open("http://www.google.com"); ENV.delete("TMPDIR"); FileUtils.rm_rf d'

It should print something like

#<Tempfile:$HOME/.tmp/open-uri20131115-16887-nag9pr>

P.S. I'm using Ruby 2.1.0 preview, so maybe you have to look at #{ruby directory}/lib/ruby/2.0.0/open-uri.rb source in order to understand how OpenURI manages the temporary file (but it should be very similar)

Upvotes: 1

engineersmnky
engineersmnky

Reputation: 29328

the question is unclear are you asking about the open method of File class? If so just specify the directory 

File.open("/your/path/to/new-tmp/#{uri.split(/\//)[-1]}","wb")

The open method of open-uri just reads out into a StringIO using open(uri,&:read) returns a String neither one of these are stored in temp.

require 'open-uri'
open("http://www.google.com/").class
#=> StringIO
open("http://www.google.com/", &:read).class
#=> String

Upvotes: 0

Related Questions