CODEWITHSUNDEEP
multithreadingio
ppk
ppk

Reputation: 568

How read and process a large file efficiently?

I would to know how I can read a large file on disk and do some work with it efficiently.

I am dealing with this question some time.

I am thinking about it, and I believe a general solution ( not platform tied ) could be using threads and a buffer.

But I am not sure to be the best method.

I have read these entries and both are right.

What is the best way to do a search in a large file?

What's the best way to read and parse a large text file over the network?

So, what's your advice ?

Thanks

Upvotes: 0

Views: 839

Answers (1)

Tudor
Tudor

Reputation: 62439

I would do it using a producer-consumer model, with a single producer thread reading portions of the file sequentially and putting them in a buffer for multiple producer threads to process them in parallel.

Upvotes: 1

Related Questions