CODEWITHSUNDEEP
pythonclassconcurrent.futures
Lukas
Lukas

Reputation: 53

How do I run a class concurrently in my main function?

I want to call and run a class in my main() function concurrently. I have different methods in my code that I want to run at the same time using concurrent.futures and I figured out I could put them in a class instead.

This is what I have tried so far:

import requests
import time
import concurrent.futures

img_urls = [
    'https://images.unsplash.com/photo-1516117172878-fd2c41f4a759',
    'https://images.unsplash.com/photo-1532009324734-20a7a5813719',
    'https://images.unsplash.com/photo-1524429656589-6633a470097c',
    'https://images.unsplash.com/photo-1530224264768-7ff8c1789d79'
]

t1 = time.perf_counter()


class Download:
    def __init__(self, img_url):
        self.img_url = img_url

    def download_image(self, img_url):
        img_bytes = requests.get(self.img_url).content
        return img_bytes

    def image_name(self, img_bytes):
        img_bytes = download_image(self, img_url)
        img_name = self.img_url.split('/')[3]
        img_name = f'{img_name}.jpg'
        with open(img_name, 'wb') as img_file:
            img_file.write(img_bytes)
            print(f'{img_name} was downloaded...')

    def run(self):
        download_image(self, img_url)
        image_name(self, img_bytes)


def main():
    with concurrent.futures.ThreadPoolExecutor() as executor:
        executor.map(Download, img_urls)

if __name__ == "__main__":
    main()

t2 = time.perf_counter()

print(f'Finished in {t2-t1} seconds')

Upvotes: 0

Views: 79

Answers (2)

Pralay Ramteke
Pralay Ramteke

Reputation: 21

One way to solve this problem is as Marco mentioned.

As you want to call and run using class, you can call the multithreading code from the __init__ function. 

import requests
import time
import concurrent.futures

img_urls = [
    "https://images.unsplash.com/photo-1516117172878-fd2c41f4a759",
    "https://images.unsplash.com/photo-1532009324734-20a7a5813719",
    "https://images.unsplash.com/photo-1524429656589-6633a470097c",
    "https://images.unsplash.com/photo-1530224264768-7ff8c1789d79",
]

t1 = time.perf_counter()


class Download:
    def __init__(self, img_url):
        self.img_url = img_url
        self.download_all_images()

    def download_image(self, img_url):
        img_bytes = requests.get(img_url, stream=True).content
        return img_bytes

    def image_name(self, img_url):
        try:
            img_bytes = self.download_image(img_url)
            img_name = img_url.split("/")[3]
            img_name = f"{img_name}.jpg"
            print("here", img_name)
            with open(img_name, "wb") as img_file:
                img_file.write(img_bytes)
                print(f"{img_name} was downloaded...")
        except Exception as e:
            print(e)

    def download_all_images(self):
        try:
            with concurrent.futures.ThreadPoolExecutor() as executor:
                executor.map(self.image_name, self.img_url)
            return "Success"
        except:
            return "Failed"


def main():
    response = Download(img_urls)
    print(response)


if __name__ == "__main__":
    main()

t2 = time.perf_counter()

print(f"Finished in {t2 - t1} seconds")

Upvotes: 0

Marco Zamboni
Marco Zamboni

Reputation: 224

As I understand you want to execute the run function of different Download objects concurrently.

The first thing is that there is a syntax error in the run function, it should be:

def run(self):
    img_bytes = download_image(self, img_url)
    image_name(self, img_bytes)

otherwise img_bytes isn't defined.

Then you need to pass the correct callable to the executor. If you pass the class Download, it will only create an instance of it, not actually call the run method; to do so with every time a new instance of Download something like this should work:

executor.map(lambda url: Download(url).run, img_urls)

Upvotes: 1

Related Questions