Delphi stack overflow and access violation error when setting array (of records) length

Question

I am busy building an application in which I am reading data from more two files of "records". I have a very strange error, which pops up depending on the sequence in which I open the files (see code below).

If I click button1 followed by button 2, thus calling the file of "weather data records" followed by the file of "parameters records", all is fine. If I do this the other way around, I get a "stack overflow" followed by "access violation at 0x7c90e898: write of address" error. This happens when I call SetLength for the array in Button1Click.

The weather data file has about 550 records, and the parameters file has about 45 records.

Can anyone see anything obvious wrong with my code? I am not sure how to attach files, or make them available, if anyone wants to use them to test...

unit Unit1;

interface

uses
  Windows, Messages, SysUtils, Variants, Classes, Graphics, Controls, Forms,
  Dialogs, StdCtrls, Buttons, ExtCtrls, Grids,FileCtrl,Contnrs;

type  
    TWeatherData = record  
    MyDate : TDate;  
    Rainfall : Double;  
    Temperature : Double;  

  end;

  TParameters = record
    Species : string[50];
    ParameterName: string[50];
    ParameterValue : double;
  end;

  TForm1 = class(TForm)
    Button1: TButton;
    Button2: TButton;
    procedure Button1Click(Sender: TObject);
    procedure Button2Click(Sender: TObject);

  private
    { Private declarations }
  public
    { Public declarations }
  end;

var
  Form1: TForm1;

implementation

{$R *.dfm}

procedure TForm1.Button1Click(Sender: TObject);
Var
  WeatherDataFile : file of TWeatherData;
  j : integer;
  WeatherDataArray : array of TWeatherData;
  MyFileSize : Integer;

begin


  AssignFile(WeatherDataFile,'C:\Test5.cmbwthr') ;
  Reset(WeatherDataFile);
  MyFileSize := FileSize(WeatherDataFile);

  SetLength(WeatherDataArray,MyFileSize);

  j := 0;

  try
   while not Eof(WeatherDataFile) do begin
    j := j + 1;
    Read (WeatherDataFile, WeatherDataArray[j]) ;
   end;
  finally
   CloseFile(WeatherDataFile) ;
  end;
end;

procedure TForm1.Button2Click(Sender: TObject);
var
  ParametersFile : file of TParameters;
  j : integer;
  CurrentParameters : array of TParameters;
  MyFileSize : Integer;

begin
  AssignFile(ParametersFile,'C:\Test5.cmbpara') ;
  Reset(ParametersFile);

  Reset(ParametersFile);
  MyFileSize := FileSize(ParametersFile);

  SetLength(CurrentParameters,MyFileSize);

  j := 0;

  try
   while not Eof(ParametersFile) do begin
    j := j + 1;
    Read (ParametersFile, CurrentParameters[j]) ;
   end;
  finally
   CloseFile(ParametersFile) ;
  end;
end;

end. 

Answer 1

Take a look at our TDynArray wrapper available in our SynCommons.pas unit. There is serialization feature included.

And you could put regular string inside the records, instead of shortstring: it will use less space on disk, and will be Unicode Ready since Delphi 2009.

type  
  TWeatherData = record  
    MyDate : TDate;  
    Rainfall : Double;  
    Temperature : Double;  
  end;
  TWeatherDatas = array of TWeatherData;

  TParameter = record
    Species : string;
    ParameterName: string;
    ParameterValue : double;
  end;
  TParameters = array of TParameter;

var
  Stream: TMemoryStream;
  Params: TParameters;
  Weather: TWeatherDatas;
begin
  Stream := TMemoryStream.Create;
  try
    Stream.LoadFromFile('C:\Test5.cmbpara');
    DynArray(TypeInfo(TParameters),Params).LoadFromStream(Stream));
    Stream.LoadFromFile('C:\Test5.cmbwthr');
    DynArray(TypeInfo(TWeatherDatas),Weather).LoadFromStream(Stream));
  finally
    Stream.Free;
  end;
end;

With TDynArray, you can access any dynamic array using TList-like properties and methods, e.g. Count, Add, Insert, Delete, Clear, IndexOf, Find, Sort and some new methods like LoadFromStream, SaveToStream, LoadFrom and SaveTo which allow fast binary serialization of any dynamic array, even containing strings or records - a CreateOrderedIndex method is also available to create individual index according to the dynamic array content. You can also serialize the array content into JSON, if you wish.

For Delphi 6 up to XE.

Answer 2

You need packed records to save to a file.

type  
  TWeatherData = packed record  
    MyDate : TDate;  
    Rainfall : Double;  
    Temperature : Double;  
  end;

  TParameters = packed record
    Species : string[50];
    ParameterName: string[50];
    ParameterValue : double;
  end;

Answer 3

You're writing past the ends of the arrays by incrementing the index before writing to the array instead of afterward. Since you're writing into memory that doesn't belong to the array, any number of problems may occur.

AssignFile(ParametersFile, 'C:\Test5.cmbpara');
Reset(ParametersFile);
try // Enter "try" block as soon as the file is opened.
  MyFileSize := FileSize(ParametersFile);
  SetLength(CurrentParameters, MyFileSize);

  j := 0;
  while not Eof(ParametersFile) do begin
    Read(ParametersFile, CurrentParameters[j]);
    Inc(j);
  end;
finally
  CloseFile(ParametersFile);
end;

if j <> MyFileSize then
  raise Exception.CreateFmt('Parameter count mismatch: expected %d but got %d instead.',
    [MyFileSize, j]);