CODEWITHSUNDEEP
javamultithreadingparsing
hetmanJIIIS
hetmanJIIIS

Reputation: 104

Java parsing txt file multithreading

I have a method to parse txt file and saving datas to database. Method named: parseFileAndSaveToDB(String fileAddress); This file are big (100 000 rows) so inserting datas into DB are very long.

Can anyone help me with do this in multithreading?? I think that each thread can use this method with each files forexample: 

parseFileAndSaveToDB("c:/file1");//thread 1
parseFileAndSaveToDB("c:/file2");//thread 2
parseFileAndSaveToDB("c:/file3");//thread 3
parseFileAndSaveToDB("c:/file4");//thread 4

But how to do it with multithreads?? And is it good idea?

Upvotes: 1

Views: 1982

Answers (4)

RMachnik
RMachnik

Reputation: 3684

This is quick draft. As the argument of program pass list of file names that should be read and store.

public class ParsingThread implements Runnable{
private String fileName;

public ParsingThread(String fName){
    fileName = fName;
}

@Override
public void run(){
    parseFileAndSaveToDB(fileName);

}
public void parseFileAndSaveToDB(String fileName){
    //your implementation
}



public static void main(String args[]){
    for(String fileName : args){
        Thread runAndStore = new Thread(new ParsingThread(fileName));
        runAndStore.run();
    }

  }
}

Upvotes: 1

A.B
A.B

Reputation: 114

Since I can't add comments yet I'll add an answer instead. I agree with Rafik991's answer and in the implementation you can start by counting the number of lines you have in order to distribute your data equally between your threads. check this link for fast implementations. i'm not sure if that's a good idea since you will have go through the file to get the right line but I think that it's what you are trying to do.

Upvotes: -1

Ameer Tamboli
Ameer Tamboli

Reputation: 1298

Please take a look at this link here which explains how to use ExecutorService in java. It can be something like:

ExecutorService executorService = Executors.newFixedThreadPool(10);

MyDataLoader myDataLoader1 = new MyDataLoader("c:/file1");
MyDataLoader myDataLoader2 = new MyDataLoader("c:/file2");
MyDataLoader myDataLoader3 = new MyDataLoader("c:/file3");

executorService.execute(myDataLoader1);
executorService.execute(myDataLoader2);
executorService.execute(myDataLoader3);

executorService.shutdown();

You can write MyDataLoader class like:

public class MyDataLoader implements Runnable {

    String fileName = null;

    public MyDataLoader(String fileName) {
        this.fileName = fileName;
    }

    public void run() {
        //Your logic to parse file and insert the data in DB.
    }
}

Hope this helps.

Upvotes: 2

Saro Taşciyan
Saro Taşciyan

Reputation: 5236

Very basic template:

public static void parseFileAndSaveToDB(String filePath)
{
    new Thread(new Runnable() {

        @Override
        public void run() {
            // # TODO : Open file with path "filePath"
            // # Read file
            // # Write to database
        }
    }).start();
}

public static void main(String args[]) {
    List<String> paths; // # TODO : fill this list with the file paths you have

    for(String filePath : paths) 
    {
        parseFileAndSaveToDB(filePath);
    }
}

But before that, i strongly recommend you read some articals on the consecpt of multithreading in java. You might find the following sources helpful:

Upvotes: 0

Related Questions