CODEWITHSUNDEEP
javascriptfull-text-searchfile-search
svv
svv

Reputation: 43

How to resolve Buffer overflow while reading large text files over 2GB?

I am trying to read a large text file in JScript to search for a string, but encountering overflow exception. I wrote the following code.

var ForReading = 1;
var TriStateFalse = 0;
var strFileData;

var fso, objFile, objTS;
fso = new ActiveXObject("Scripting.FileSystemObject");
objFile = fso.GetFile("Sample2GBFile");

strFileData = objTS.Read(objFile.Size);
if(strFileData .indexOf("String to search") > 0 )
{
wShShell.Echo("Found...");
}

In the above code, I am getting the buffer overflow as the 2GB file is unable to fit into the buffer. pls help how to resolve this.

Upvotes: 1

Views: 760

Answers (2)

mplungjan
mplungjan

Reputation: 178350

Loop over a line instead of reading the whole file

http://techsupt.winbatch.com/ts/T000001033F64.html

:test4
; read single chars from a line, count chars.
testfile = "d:\temp\test.ascii.txt"
MyFile = fso.OpenTextFile(testfile, ForReading)       ; Open a file as a TextStream
ThisLine = ""
While !MyFile.AtEndOfLine                             ; Is the current position at the end of a line? 
   ThisColumnCount = MyFile.Column                    ; Current column number.
   ThisLine        = StrCat(ThisLine, MyFile.Read(1)) ; Read a specific number of characters into a string.
   NextColumnCount = MyFile.Column                    ; Current column number.
EndWhile
MyFile.Close                                          ; Close a text stream.
ObjectClose(MyFile)

:test5
; read lines, count lines.
testfile = "d:\temp\test.ascii.txt"
MyFile = fso.OpenTextFile(testfile, ForReading)       ; Open a file as a TextStream
While !MyFile.AtEndOfStream                           ; Is the current position at the end of the stream?
   ThisLineCount = MyFile.Line                        ; Current line number.
   ThisLine      = MyFile.ReadLine                    ; Read an entire line into a string.
   NextLineCount = MyFile.Line                        ; Current line number.
EndWhile
MyFile.Close                                          ; Close a text stream.
ObjectClose(MyFile)

Upvotes: 0

code_burgar
code_burgar

Reputation: 12323

Read smaller portions of the file, check for string, empty buffer, read more. Just make sure you account for the length of the string when reading a chunk so as to prevent cutting it and thus missing a hit.

Upvotes: 1

Related Questions