CODEWITHSUNDEEP
stringfileparsingerlang
Rahib Rasheed
Rahib Rasheed

Reputation: 367

How can I read a text file with size 6GB in Erlang?

I have 6 GB of text file. I want to read this file and parse the data from each line and save it in my database. But it is not possible to the entire 6 GB file at once due to memory issues. So How can I read the data chunk by chunk and then process it?

Upvotes: 1

Views: 216

Answers (2)

Nalin Ranjan
Nalin Ranjan

Reputation: 1782

Maybe you can try the read function. It allows for an argument to limit the number of bytes or characters from the file.

Upvotes: 2

legoscia
legoscia

Reputation: 41618

Something like this:

process_file(Filename) ->
    {ok, F} = file:open(Filename, [read]),
    process_lines(F).

process_lines(F) ->
    case file:read_line(F) of
        {ok, Line} ->
            %% do something with Line
            process_lines(F);
        eof ->
            file:close(F)
    end.

Upvotes: 0

Related Questions