CODEWITHSUNDEEP
while-loopeoffreepascallazarus
Gizmo_the_Great
Gizmo_the_Great

Reputation: 979

Using Free Pascal\Lazarus to parse a large binary file for specific values

I need to parse a RAM dump for MFT records (from the NTFS filesystem).

I've done some programming in the past with regard to reading headers of multiple files (using FileSearcher class, etc.), but I'm not entirely sure how to start reading from the start of a large file, read through it, and when a certain value is found I need to read in 1024 bytes from the point that the magic value is found (FILE0, in the case of MFT entries) and 'do stuff' with the values in between that and the end of the 1024 byte range. It then needs to carry on searching for the next FILE0 record.

So far I have the following - my intention is that it reads through the source file (which is a TFileStream) looking for 'FILE0'. When it finds it, at this stage I just want it to report that it has found a record and output the position, but in due course I need it to then read a series of bytes from the point at which FILE0 was found:

type
    MFTRecordsStore = packed record
    FILE0MagicMarker: array[0..4] of byte;
    // Lots more follow....
end;

var
    MFTHeaderArray : MFTRecordsStore;
    FILE0Present : string;
    i : integer;

begin
    SourceFile.Position := 0;
    while (SourceFile.Position < SourceFile.Size) do
        begin
            SourceFile.ReadBuffer(MFTHeaderArray, SizeOf(MFTHeaderArray));
            for i := 0 to 4 do
                FILE0Present := FILE0Present + IntToHex(MFTHeaderArray.FILE0MagicMarker[i], 2);
                if FILE0Present = 'FILE0' then
                    begin
                        Memo1.Lines.Add('FILE0 Entry found at '+ IntToStr(SourceFile.Position));
                    end;
        end;
  end;

This code compiles and runs (it starts to parse the file), but after several minutes of heavy CPU usage the program crashes and reports that it cannot read the stream. I have a feeling this has something to do with getting to the end of the file and there not been a full 'chunk' left to read so it crashes?

What is the solution?

Upvotes: 3

Views: 4308

Answers (2)

TLama
TLama

Reputation: 76693

I'm posting an example of how I would write and read file of records using streams and seek for a certain ANSI text in it. You may check also the commented version of this post.

Here is the record definition used in this example:

type
  TFileRecord = packed record
    Marker: array [0..4] of Byte;
    Width: Integer;
    Height: Integer;
    Useful: Boolean;
  end;

Here is how to create such file of records (what you already have :)

procedure TForm1.Button1Click(Sender: TObject);
var
  FileStream: TFileStream;
  FileRecord: TFileRecord;
const
  RecordSize = SizeOf(TFileRecord);

  procedure FillFileRecord(const AMarker: string; const AWidth: Integer;
    const AHeight: Integer; const AUseful: Boolean);
  begin
    FillChar(FileRecord, RecordSize, 0);
    Move(AMarker[1], FileRecord.Marker, Length(FileRecord.Marker));
    FileRecord.Width := AWidth;
    FileRecord.Height := AHeight;
    FileRecord.Useful := AUseful;
  end;

begin
  FileStream := TFileStream.Create('File.dat', fmCreate);
  try
    FillFileRecord('FILE1', 111, 112, False);
    FileStream.Write(FileRecord, RecordSize);
    FillFileRecord('FILE2', 211, 212, False);
    FileStream.Write(FileRecord, RecordSize);
    FillFileRecord('FILE3', 311, 312, False);
    FileStream.Write(FileRecord, RecordSize);
    FillFileRecord('FILE4', 411, 412, False);
    FileStream.Write(FileRecord, RecordSize);
    FillFileRecord('FILE0', 666, 777, True);
    FileStream.Write(FileRecord, RecordSize);
    FillFileRecord('FILE5', 511, 512, False);
    FileStream.Write(FileRecord, RecordSize);
    FillFileRecord('FILE0', 11111, 22222, True);
    FileStream.Write(FileRecord, RecordSize);
    FillFileRecord('FILE6', 611, 612, False);
    FileStream.Write(FileRecord, RecordSize);
  finally
    FileStream.Free;
  end;
end;

And here is how to read such file:

procedure TForm1.Button2Click(Sender: TObject);
var
  FileStream: TFileStream;
  FileRecord: TFileRecord;
const
  HeaderSeq = 'FILE0';
  HeaderLen = Length(HeaderSeq);
  RecordSize = SizeOf(TFileRecord);
begin
  FileStream := TFileStream.Create('File.dat', fmOpenRead);
  try
    while FileStream.Read(FileRecord, RecordSize) = RecordSize do
    begin
      if CompareMem(@HeaderSeq[1], @FileRecord.Marker[0], HeaderLen) then
      begin
        Memo1.Lines.Add('FILE0 entry found at '+
          IntToStr(FileStream.Position - RecordSize));
        Memo1.Lines.Add('FileRecord.Width = ' +
          IntToStr(FileRecord.Width));
        Memo1.Lines.Add('FileRecord.Height = ' +
          IntToStr(FileRecord.Height));
        Memo1.Lines.Add('FileRecord.Useful = ' +
          BoolToStr(FileRecord.Useful, True));
      end;
    end;
  finally
    FileStream.Free;
  end;
end;

Upvotes: 6

user323094
user323094

Reputation: 3653

If you really suspect reading past EOF, try:

while (SourceFile.Position + SizeOf(MFTHeaderArray) <= SourceFile.Size) do

Upvotes: 2

Related Questions