How to debug a (possible) RTL problem?

Question

I'm asking this because I'm out of good ideas...hoping for someone else's fresh perspective.

I have a user running our 32-bit Delphi application (compiled with BDS 2006) on a Windows 7 64-bit system. Our software was "working fine" until a couple weeks ago. Now suddenly it isn't: it throws an Access Violation while initializing (instancing objects).

We've had him reinstall all our software--starting all over from scratch. Same AV error. We disabled his anti-virus software; same error.

Our stack tracing code (madExcept) for some reason wasn't able to provide a stack trace to the line of the error, so we've sent a couple error logging versions for the user to install and run, to isolate the line which generates the error...

Turns out, it's a line which instances a simple TStringList descendant (there's no overridden Create constructor, etc.--basically the Create is just instancing a TStringList which has a few custom methods associated with the descendant class.)

I'm tempted to send the user yet another test .EXE; one which just instances a plain-vanilla TStringList, to see what happens. But at this point I feel like I'm flailing at windmills, and risk wearing out the user's patience if I send too many more "things to try".

Any fresh ideas on a better approach to debugging this user's problem? (I don't like bailing out on a user's problems...those tend to be the ones which, if ignored, suddenly become an epidemic that 5 other users suddenly "find".)

EDIT, as Lasse requested:

procedure T_fmMain.AfterConstruction;
begin
  inherited;
      //Logging shows that we return from the Inherited call above,
      //then AV in the following line...
  FActionList := TAActionList.Create;
  ...other code here...
end;

And here's the definition of the object being created...

type
  TAActionList = class(TStringList)
  private
    FShadowList: TStringList;              //UPPERCASE shadow list
    FIsDataLoaded : boolean;
  public
    procedure AfterConstruction; override;
    procedure BeforeDestruction; override;
    procedure DataLoaded;
    function Add(const S: string): Integer; override;
    procedure Delete(Index : integer); override;
    function IndexOf(const S : string) : Integer; override;
  end;

implementation

procedure TAActionList.AfterConstruction;
begin
  Sorted := False;              //until we're done loading
  FShadowList := TStringList.Create;
end;

Answer 1

Our software was "working fine" until a couple weeks ago... suddenly become an epidemic that 5 other users suddenly "find".) :

Sounds like you need to do some forensic analysis, not debugging: You need to discover what changed in that user's environment to trigger the error. All the more so if you have other users with the same deployment that don't have the problem (sounds like that's your situation). Sending a user 'things to try' is one of the best ways to erode user confidence very quickly! (If there is IT support at the user site, get them involved, not the user).

For starters, explore these options:

*) If possible, I'd check the Windows Event Log for events that may have occurred on that machine around the time the problem arose.

*) Is there some kind of IT support person on the user's side that you can talk to about possible changes/problems in that user's environment?

*) Was there some kind of support issue/incident with that user around the time the error surfaced that may be connected to it, and/or caused some kind of data or file corruption particular to them?

(As for the code itself, I agree with @Warran P about decoupling etc)

Answer 2

I hate these kind of problems, but I reckon you should focus on what's happening recently BEFORE the object tries to get constructed.

The symptoms you describe sound like typical heap corruption, so maybe you have something like...

• An array being written to outside bounds? (turn bounds checking on, if you have it off)
• Code trying to access an object which has been deleted?

Since my answer above, you've posted code snippets. This does raise a couple of possible issues that I can see.

a: AfterConstruction vs. modified constructor: As others have mentioned, using AfterConstruction in this way is at best not idiomatic. I don't think it's truly "wrong", but it's a possible smell. There's a good intro to these methods on Dr. Bob's site here.

b: overridden methods Add, Delete, IndexOf I'm guessing these methods use the FshadowList item in some way. Is it remotely possible that these methods are being invoked (and thus using FShadowList) before the FShadowList is created? This seems possible because you're using the AfterConstruction methods above, by which time virtual methods should 'work'. Hopefully this is easy to check with a debugger by setting some breakpoints and seeing the order they get hit in.

Answer 3

Clearly there is nothing suspicious about what TAActionList is doing at time of construction. Even considering ancestor constructors and possible side-effects of setting Sorted := False indicate there shouldn't be a problem. I'm more interested in what's happening inside T_fmMain.

Basically something is happening that causes FActionList := TAActionList.Create; to fail, even though there is nothing wrong in the implementation of TAActionList.Create (a possibility is that the form may have been unexpectedly destroyed).

I suggest you try changing T_fmMain.AfterConstruction as follows:

procedure T_fmMain.AfterConstruction;
begin
  //This is safe because the object created has no form dependencies 
  //that might otherwise need to be initialised first.
  FActionList := TAActionList.Create;

  //Now, if the ancestor's AfterConstruction is causing the problem, 
  //the above line will work fine, and...
  inherited AfterConstruction;

  //... your error will have shifted to one of these lines here.
  //other code here
end;

If an environment issue with a component used by your form is causing it destroy the form during AfterConstruction, then it's the assignment of the new TAActionList.Create instance to FActionList that's actually causing the AV. Another way to test would be to first create the object to a local variable, then assign it to the class field: FActionList := LActionList.

Environment problems can be subtle. E.g. We use a reporting component which we discovered requires that a printer driver is installed, otherwise it prevents our application from starting up.

You can confirm the destruction theory by setting a global variable in the form's destructor. Also you may be able to output a stack trace from the destructor to confirm the exact sequence leading to the destruction of the form.

Answer 4

You should never override AfterConstruction and BeforeDestruction methods in your programs. They are not meant for what you're doing with them, but for low-level VCL hacking (like reference adding, custom memory handling or such).

You should override the Create constructor and Destroy destructor instead and put your initialization code here, like such:

constructor TAActionList.Create;
begin
  inherited; 
  // Sorted := False; // not necessary IMHO
  FShadowList := TStringList.Create;
end;

Take a look at the VCL code, and all serious published Delphi code, and you'll see that AfterConstruction and BeforeDestruction methods are never used. I guess this is the root cause of your problem, and your code must be modified in consequence. It could be even worse in future version of Delphi.

Answer 5

Things to do when MadExcept is NOT Enough (which is rare, I must say):

1. Try Jedi JCL's JCLDEBUG instead. You might get a stack traceback with it, if you change out MadExcept for JCLDEBUG, and write directly the stack trace to the disk without ANY UI interaction.

2. Run a debug-viewer like MS/SysInternals debugview, and trace output things like the Self pointers of the objects where the problems are happening. I suspect that somehow an INVALID instance pointer is ending up in there.

3. Decouple things and refactor things, and write unit tests, until you find the really ugly thing that's trashing you. (Someone suggested heap corruption. I often find heap corruption goes hand in hand with unsafe ugly untested code, and deeply bound UI+model cascading failures.)