CODEWITHSUNDEEP
filememorylua
128Gigabytes
128Gigabytes

Reputation: 193

How can I read a large file (A gig or more) in lua?

I have been searching for the answer to this and everyone's answer is always just do it line by line, but the thing is my file is all just one line of characters, and trying to io.open("file.txt", "rb"):read("*a") results in a memory error. I can not think of how to load it in part at a time because like I said, its all one giant line.

Upvotes: 3

Views: 1620

Answers (2)

王奕然
王奕然

Reputation: 4059

you could use table as buffer:

function readFile(file)
local t = {}
 for line in io.lines(file) do
    t[#t + 1] = line .. "\n"
end
  local s = table.concat(t)
  return s
end

Upvotes: 0

Paul Kulchenko
Paul Kulchenko

Reputation: 26794

You can use io.read(size) to read a buffer of a specified size (as is already discussed in comments). See the example at the end of the I/O section in Programming in Lua.

Since you are doing a search in the chunks you read, the string you are searching for may be split between different chunks, so you need to take that into account. Another example from PiL that talks about reading large files may be of interest.

Upvotes: 3

Related Questions