CODEWITHSUNDEEP
javamultithreading
user373201
user373201

Reputation: 11425

java download multiple files using threads

I am trying to download multiple files that matches a pattern using threads. The pattern could match 1 or 5 or 10 files of diff sizes.

lets say for simplicity sake the actual code that would download the file is in downloadFile() method and fileNames is the list of filenames that match the pattern. How do I do this using threads. Each thread will download only one file. Is it advisable to create a new thread inside the for loop. 

for (String name : fileNames){
    downloadFile(name, toPath);
}

Upvotes: 15

Views: 21975

Answers (5)

ihappyk
ihappyk

Reputation: 555

All the Above mentioned approach creates Threads but the actual Concurreny is not achieved.

ExecutorService pool = Executors.newFixedThreadPool(5);
final File folder = new File("YOUR_FILES_PATH");
int l = folder.listFiles().length;
System.out.println("Total Files----"+folder.listFiles().length);
long timeStartFuture = Calendar.getInstance().getTimeInMillis();
    pool.execute(new DownloadFile(folder,0,l/2));
    pool.execute(new DownloadFile(folder,(l/2),l));
    pool.shutdown();
    try {
        pool.awaitTermination(Long.MAX_VALUE, TimeUnit.MILLISECONDS);
    } catch (InterruptedException e) {
        // TODO Auto-generated catch block
        e.printStackTrace();
    }

    long timeEndFuture = Calendar.getInstance().getTimeInMillis();
    long timeNeededFuture = timeEndFuture - timeStartFuture;
    System.out.println("Parallel calculated in " + timeNeededFuture + " ms");

The above program is used to achieve concurreny and please modify as per your requirement.

Upvotes: 0

Bitmap
Bitmap

Reputation: 12538

Use Executor, Try this.

ExecutorService  exec= Executors.newCachedThreadPool()
for (String name : fileNames){
 exec.submit(new Runnable()
 {
  public void run()
  {
    downloadFile(name, toPath);
  }
 });
}

If you want say three download running concurrently, you can use:

Executors.newFixedThreadPool(3)

Upvotes: 1

Philipp Reichart
Philipp Reichart

Reputation: 20961

You really want to use an ExecutorService instead of individual threads, it's much cleaner, likely more performant and will enable you to change things more easily later on (thread counts, thread names, etc.):

ExecutorService pool = Executors.newFixedThreadPool(10);
for (String name : fileNames) {
    pool.submit(new DownloadTask(name, toPath));
}
pool.shutdown();
pool.awaitTermination(Long.MAX_VALUE, TimeUnit.MILLISECONDS);
// all tasks have now finished (unless an exception is thrown above)

And somewhere else in your class define the actual work horse DownloadTask:

private static class DownloadTask implements Runnable {

    private String name;
    private final String toPath;

    public DownloadTask(String name, String toPath) {
        this.name = name;
        this.toPath = toPath;
    }

    @Override
    public void run() {
        // surround with try-catch if downloadFile() throws something
        downloadFile(name, toPath);
    }
}

The shutdown() method has a very confusing name, because it "will allow previously submitted tasks to execute before terminating". awaitTermination() declares an InterruptedException you need to handle.

Upvotes: 39

maerics
maerics

Reputation: 156404

Yes, you certainly could create a new thread inside the for-loop. Something like this:

List<Thread> threads = new ArrayList<Thread>();
for (String name : fileNames) {
  Thread t = new Thread() {
    @Override public void run() { downloadFile(name, toPath); }
  };
  t.start();
  threads.add(t);
}
for (Thread t : threads) {
  t.join();
}
// Now all files are downloaded.

You should also consider using an Executor, for example, in a thread pool created by Executors.newFixedThreadPool(int).

Upvotes: 5

Garrett Hall
Garrett Hall

Reputation: 30022

Yes you can create the Threads inline.

for (final String name : fileNames){
    new Thread() {
       public void run() {
           downloadFile(name, toPath);
       }
    }.start();
}

Upvotes: 1

Related Questions