CODEWITHSUNDEEP
pythonurlargumentsfilenames
Catherine Devlin
Catherine Devlin

Reputation: 7743

How to supply URL to function requiring a filename in Python

I'm looking for a general recipe to use a URL for a function that expects a filename. I've puzzled one out, but it's a little complex and easy to get wrong.

In this case, my function is read_file from geopandas, but it would be the same problem in any case.

import tempfile, requests
import geopandas as gpd

def as_file(url):
    tfile = tempfile.NamedTemporaryFile()
    tfile.write(requests.get(url).content)
    return tfile

URL = 'https://raw.githubusercontent.com/bowmanmc/ohiorepresents/master/data/congressional.min.json'
tf = as_file(URL)
gpd.read_file(tf.name)

That works, and doesn't look too awful, but I had to experiment a bunch to find it, because slight variants raise OSError: no such file or directory, due to the lifespan of tempfiles; yet I also don't want to clutter the filesystem with permanent files.

This fails:

def as_file(url):
    tfile = tempfile.NamedTemporaryFile()
    tfile.write(requests.get(url).content)
    return tfile.name

gpd.read_file(as_file(URL))

and even this:

def as_file(url):
    tfile = tempfile.NamedTemporaryFile()
    tfile.write(requests.get(url).content)
    return tfile

gpd.read_file(as_file(URL).name)

Is there a more obvious, memorable, or bulletproof way?

Upvotes: 5

Views: 1314

Answers (1)

Open AI - Opting Out
Open AI - Opting Out

Reputation: 24133

You can use a context manager to manage the lifetime of the temporary file:

from contextlib import contextmanager

@contextmanager
def as_file(url):
    with tempfile.NamedTemporaryFile() as tfile:
        tfile.write(requests.get(url).content)
        tfile.flush()
        yield tfile.name

Note: The with NamedTemporaryFile() as tfile will only work with Python 3. Otherwise you'll have to make sure it cleans up properly yourself for Python 2.

Usage:

with as_file(URL) as filename:
    gpd.read_file(filename)

Upvotes: 6

Related Questions