The Wiert Corner – irregular stream of stuff

Jeroen W. Pluimers on .NET, C#, Delphi, databases, and personal interests

  • My badges

  • Twitter Updates

  • My Flickr Stream

  • Pages

  • All categories

  • Enter your email address to subscribe to this blog and receive notifications of new posts by email.

    Join 2,466 other followers

Archive for the ‘Undocumented Delphi’ Category

Parser generator toolset for Delphi (Yacc & Lex)

Posted by jpluimers on 2021/08/03

The most current Lex and Yacc for Delphi is [WayBack] GitHub – RomanYankovsky/ndyacclex: Parser generator toolset for Delphi (Yacc & Lex).

It came up during a funny Twitter thread, where one of the Delphi team members asked for it, despite – after decades of asking – there still being no official Delphi grammar available from the Delphi team, nor Google Search skills [WayBack].

Basically a parser generator is only as useful as the grammar you have for it.

There is no open source grammar for Delphi yet, so the best you can start with is from the same author: [WayBack] GitHub – RomanYankovsky/DelphiAST: Abstract syntax tree builder for Delphi

With DelphiAST you can take real Delphi code and get an abstract syntax tree. One unit at time and without a symbol table though.

Relevant tweets:

–jeroen

Posted in Delphi, Development, Software Development, Undocumented Delphi | 1 Comment »

Delphi: got “EOleException with message ‘Microsoft MSXML is not installed'” in a console or test project?

Posted by jpluimers on 2021/03/31

Did you ever get this run-time error in a console or test project?

EOleException with message 'Microsoft MSXML is not installed'

It means that CoInitialize or CoInitializeEx needs to be called in the thread that uses MSXML.

Then an easy workaround is to:

  1. use the unit System.Win.ComObj in any unit that (indirectly) uses Xml.XMLDoc (for instance any unit using an XML Data Binding generated unit),
  2. use the unit System.SysUtils as well (because it defines TProcedure used below)
  3. add this code in in your initialization section (which is what VCL TApplication.Initialize does):

if InitProc <> nil then TProcedure(InitProc); // Calls CoInitialize for the main thread and prevents "EOleException with message 'Microsoft MSXML is not installed'"

The initialization section of System.Win.ComObj sets up InitProc to cals CoInitialize for the main thread, which usually suffices for these simple VCL projects, but not for most console or test projects.

Based on ideas I got after reading [WayBack] 为什么LoadXMLDocument在线程类中使用会报错?-CSDN论坛 (for which Google Translate actually does a goot job [Archive.is])

–jeroen

Posted in Conference Topics, Conferences, Delphi, Development, Event, Software Development, Undocumented Delphi | Leave a Comment »

When your Delphi IDE suddenly skips unsaved changes during compilation(TL;DR: watch early signs your IDE is hosed, then restart without saving)

Posted by jpluimers on 2021/03/24

A while ago, I observed that when compiling, my Delphi IDE would not take into account unsaved changes any more.

This ws in a time when I was tracking down some hard to reproduce problems of code that sometimes would and sometimes would not compile at all.

The solution was this:

[HKEY_CURRENT_USER\Software\Embarcadero\BDS\18.0\Compiling]
"BackgroundCompilation"="False"

Somehow, the Delphi IDE had turned this flag to True without me telling it did, nor me changing an option (heck if you do a “Delphi” “BackgroundCompilation” – Google Search you hardly get any meaningful results).

Luckily, I did remember what happened around the bahaviour change: the compiler had encountered a strange error, and the IDE had become unstable.

With an unstable IDE, I did have seen damage in saved source files in the past, so I always use version control with Delphi as that allows easier to spot file differences.

What I did not anticipate was that it could corrupting my persisted IDE settings, though every now and then.

Detecting early signs of the IDE becoming unstable

  • any internal compiler error (AV or not)
  • refactoring not succeeding while it should
  • insert mode suddenly becomes override or vice versa
  • editor block selection is suddenly turned on
  • any access violation or pointer error exception

Sometimes (but not always) these can be early signs too

  • debugger blue dots not matching compiled code lines
  • the debugger not being able to debug code despite blue dots being there
  • properties in the object inspector having changed without manual action

Be prepared for an unstable IDE

  • Save your work often
  • At the earliest sign of an unstable IDE: kill (do not save work!) the affected bds.32 process using Process Explorer

BackgroundCompilation

A “Delphi” “BackgroundCompilation” – Google Search did not get much relevant results. Below are the most relevant ones I could find from it:

Too bad Google does not index the WayBack machine, as I think it contains relevant material that is now hard to find.

So it looks like the feature was introduced somewhere close to Delphi 5:

[HKEY_CURRENT_USER\Software\Borland\Delphi\5.0\Compiling]
"Show Compiler Progress"="True"
"Warn on Package Rebuild"="-1"
"Compile Beep"="0"
"Cache Headers"="0"
"BackgroundCompilation"="0"

“Delphi” “Background Compilation” – Google Search shows much more information, based on what it returned I found that the first actual documentation was for Background Compilation in Delphi 2010, some 10 years after it became available:

The image in the blog post of former product manager Andreano Lanusse shows why I did not see the behaviour: when background compiling is active, the progress dialog is transparent (and non-modal). I did not have the compiler progress enabled, so never saw that dialog change behaviour.

–jeroen

 

Posted in Conference Topics, Conferences, Delphi, Development, Event, Software Development, Undocumented Delphi | Leave a Comment »

Delphi object instance lifetime demo; do not use AfterConstruction as a poor-mans way to work around non-virtual constructor or undetermined Create hierarchy calls

Posted by jpluimers on 2021/03/23

I think using AfterConstruction is a poor man’s solution that you should only use in exceptional cases, as it is:

  • called only after the last constructor in the chain is called.
  • called outside of the constructor chain (i.e. exceptions in it will not automatically call the destructor chain, nor BeforeDestruction)
  • meant to add any initialization code that requires a fully created object instance.

There were quite a few sites I  visited that were using AfterConstruction. Usage roughly falls into two cases:

  • Virtually all of their usage was distrusting all Create call chains to be called properly.

    Introducing a new AfterConstruction call chain does not work around this problem.

    In fact it adds just another problem of equal magnitude: a new call chain with the same trust on calling the inheritance chain.

  • Some teams use it as a replacement for a virtual constructor, or as a safeguard for developers calling the non-virtual constructor.

    That kind of AfterConstruction usage now introduces a second call hierarchy, making the code more complex, to circumvent a problem you can easily solve:

    1. introduce a virtual Create constructor in a base class
    2. have that constructor set a field to ensure it is called
    3. check that field at appropriate places, for instance in the Destroy destructor (see example below)

Destruction is important too

A surprising thing in many code bases, is that far less attention is paid to destruction.

Of the above sites, many classes had AfterConstruction. Very few of those had a BeforeDestruction counterpart, though a lot of them should have had one.

In the same light, I see many places where a destructor Destroy never got implemented, leading to all sorts of de-initialisation problems. The easiest problems they can cause are memory leaks, which are usually highly visible. Harder problems are for instance handles that are kept open.

Note that raised Exception instances are normally destroyed in the exception handling.

Finally, a lot of destructor and BeforeDestruction code is very sloppy at handling exceptional cases: not just handling exceptions themselves, but also handling partially initialised instances.

It pays to write unit tests for these cases, as it will save you from surprises.

Execution order

Two small examples in one lifetime demo.

Output

Without raising exceptions:
Create(AConstructorBehaviour = NoExceptionInConstructor)
AfterConstruction
BeforeDestruction
Destroy; FAfterConstructionWasCalled = True;
FreeInstance

With raising exception in constructor:
Create(AConstructorBehaviour = RaiseExceptionInConstructor)
Destroy; FAfterConstructionWasCalled = False;
FreeInstance

With raising exception in AfterConstruction:
Create(AConstructorBehaviour = NoExceptionInConstructor)
AfterConstruction
BeforeDestruction
Destroy; FAfterConstructionWasCalled = True;
FreeInstance
Exception: Error Message

Note the second part of the output does not call AfterConstruction nor BeforeDestruction. This is because when construction fails, the destructor is being called: a Delphi language feature that few people know about as on-line documentation is very hard in html documentation [WayBack] Delphi Language Guide, topic [WayBack] Classes and Objects, subtopic [WayBack] Methods: Constructors:

If an exception is raised during execution of a constructor that was invoked on a class reference, the Destroy destructor is automatically called to destroy the unfinished object.

That – very easy to overlook – one little sentence without any example is all to it, that – as of Delphi 10.1 Berlin – moved to [Archive.is] Methods (Delphi) – RAD Studio: Constructors.

A more elaborate part was put in the html documentation of RAD Studio 2010 C++ documentation subtopic “Object Destruction” quoted below.

Raising exceptions in constuctors

Given this class hierarchy:

type
  A = class
  end;
  B = class(A)
  end;
  C = class(B)
  end;

[Archive.is] Object Destruction – RAD Studio, limited to only the Pascal bits, then is like this:

Consider the case where an exception is raised in the constructor of class B when constructing an instance of C. What results in .. Object Pascal …is described here:

  • In Object Pascal, only the instantiated class destructor is called automatically. This is the destructor for C. As with constructors, it is entirely the programmer’s responsibility to call inherited in destructors. In this example, if we assume all of the destructors call inherited, then the destructors for C, B, and A are called in that order. Moreover, whether or not inherited was already called in B’s constructor before the exception occurred, A’s destructor is called because inherited was called in B’s destructor. Calling the destructor for A is independent of whether its constructor was actually called. More importantly, because it is common for constructors to call inherited immediately, the destructor for C is called whether or not the body of its constructor was completely executed.

The oldest (PDF) documentation I could find mentioning this is [WayBack] CB6_DevelopersGuide_EN.pdf (C++ Builder 6: Developer’s Guide – Documentation).

It still strikes me as odd that such an important Delphi language feature is only documented in the C++ product line.

Luckily others stepped into this void, all found through delphi raise in constructor calls destructor – Google Search:

There is also on-line information that puts you on the wrong foot:

[WayBack] Exceptions in Constructor and or AfterConstruction ? – delphi

But why than is BeforeDestruction not being call automatically when the the
exception is raised in AfterConstruction?

The above output shows this last observation is clearly wrong.

Detecting if methods got called

The second part of the output also shows you that you can detect the condition that AfterConstruction was indeed called (or in fact any method was called) from the Destroy (or any other method).

The code below shows that this is very easy to do.

Code

program ExceptionInConstructorConsoleProject;

{$APPTYPE CONSOLE}

uses
  System.SysUtils,
  System.Rtti;

type
  TConstructorBehaviour = (RaiseExceptionInConstructor, NoExceptionInConstructor);
  TAfterConstructionBehaviour =  (RaiseExceptionInAfterConstruction, NoExceptionInConstructorAfterConstruction);
  TExceptionInConstructorClass = class(TObject)
  strict private
    FAfterConstructionWasCalled: Boolean;
    FAfterConstructionBehaviour: TAfterConstructionBehaviour;
  public
    constructor Create(const AConstructorBehaviour: TConstructorBehaviour; const AAfterConstructionBehaviour: TAfterConstructionBehaviour);
    destructor Destroy(); override;
    procedure AfterConstruction(); override;
    procedure BeforeDestruction(); override;
  end;

constructor TExceptionInConstructorClass.Create(const AConstructorBehaviour: TConstructorBehaviour);
begin
  Writeln(Format('Create(AConstructorBehaviour = %s)', [TRttiEnumerationType.GetName(AConstructorBehaviour)]));
  inherited Create();
  FAfterConstructionWasCalled := False;
  FAfterConstructionBehaviour := AAfterConstructionBehaviour;
  if AConstructorBehaviour = RaiseExceptionInConstructor then
    raise Exception.Create('Error Message');
end;

destructor TExceptionInConstructorClass.Destroy();
begin
  Writeln(Format('Destroy; FAfterConstructionWasCalled = %s;', [TRttiEnumerationType.GetName<Boolean>(FAfterConstructionWasCalled)]));
  inherited Destroy();
end;

procedure TExceptionInConstructorClass.AfterConstruction();
begin
  Writeln('AfterConstruction');
  inherited AfterConstruction();
  if FAfterConstructionBehaviour = RaiseExceptionInAfterConstruction then
    raise Exception.Create('Error Message');
end;

procedure TExceptionInConstructorClass.BeforeDestruction();
begin
  Writeln('BeforeDestruction');
  inherited BeforeDestruction();
end;

procedure TExceptionInConstructorClass.FreeInstance();
begin
  Writeln('FreeInstance');
  inherited FreeInstance();
end;
begin
  try
    Writeln('Without raising exceptions:');
    with TExceptionInConstructorClass.Create(NoExceptionInConstructor, NoExceptionInConstructorAfterConstruction) do
      Free();
    Writeln;
    Writeln('With raising exception in constructor:');
    try
      TExceptionInConstructorClass.Create(RaiseExceptionInConstructor, NoExceptionInConstructorAfterConstruction);
    finally
      Writeln;
      Writeln('With raising exception in AfterConstruction:');
      TExceptionInConstructorClass.Create(NoExceptionInConstructor, RaiseExceptionInAfterConstruction);
    end;
  except
    on E: Exception do
      Writeln(E.ClassName, ': ', E.Message);
  end;
end.

Statistics

In the Delphi source directory (RTL + FMX + VCL + more), you see this count:

  • constructor Create: ~5000 occurrences
  • destructor Destroy: ~2300 occurrences
  • procedure AfterConstruction: ~400 occurrences of which ~100 relevant, as there are ~300 in unit FMX.DEA.Schema to perform XML schema setup right after the constructor is run.
  • procedure BeforeDestruction: ~25 occurrences.

In % ratios:

  • Create: 100%
  • Destroy: ~45%
  • AfterConstruction: 2%
  • BeforeDestruction: 0.5%

Related

The above were centered around these pieces of Delphi documentation:

If you want to dig deeper into the various methods involved in allocation, initialisation, construction, deconstruction, and de-allocation, then remember that in the Win32 realm (which is the central target for most Delphi developers) not much has changed since Delphi 2007.

Outside that realm, things do have changed. For that, read this great book, that taught me quite a few things: [WayBack] Delphi Memory Management eBook .

Besides the eBook, you can also get it as

  • Paperback: 377 pages
  • Publisher: CreateSpace Independent Publishing Platform (June 24, 2018)
  • Language: English
  • ISBN-10: 1721654909
  • ISBN-13: 978-1721654901

These are the relevant adapted from the Delphi 2007 [WayBack] TObject Members:

WayBack Declaration Name Description
[WayBack]
procedure AfterConstruction; virtual;
AfterConstruction Responds after the last constructor has executed.
[WayBack]
procedure BeforeDestruction; virtual;
BeforeDestruction Responds before the first destructor executes.
[WayBack]
procedure CleanupInstance;
CleanupInstance Performs finalization on long strings, variants, and interface variables within a class.
[WayBack]
constructor Create;
Create Constructs an object and initializes its data before the object is first used.
[WayBack]
destructor Destroy; virtual;
Destroy Disposes of an object instance.
[WayBack]
procedure Free;
Free Destroys an object and frees its associated memory, if necessary.
[WayBack]
procedure FreeInstance; virtual;
FreeInstance Deallocates memory allocated by a previous call to the System::TObject::NewInstance method.
[WayBack]
class function InitInstance(Instance: Pointer): TObject;
InitInstance Initializes a newly allocated object instance to all zeros and initializes the instance’s virtual method table pointer.
[WayBack]
class function InstanceSize: Longint;
InstanceSize Returns the size in bytes of each instance of the object type.
[WayBack]
class function NewInstance: TObject; virtual;
NewInstance Allocates memory for an instance of an object type and returns a pointer to that new instance.

Combinining NewInstance with FeeInstance can be useful in for instance these cases:

I wish the TAbstractWorkerData was more appropriately documented, as some really interesting stuff is going on in it.

–jeroen

Posted in Conference Topics, Conferences, Delphi, Development, Event, Software Development, Undocumented Delphi | 1 Comment »

Delphi use of FS segment: structured exception handling

Posted by jpluimers on 2021/03/16

A while ago, I had to trace through a lot of code in the CPU pane to track down some memory allocation related issues.

I wondered what the use of the FS segment was about, so via [Archive.is] delphi what is fs segment used for – Google Search, I found that it is related to Win32 Structured Exception handling and therefore not limited to Delphi, through these links:

A few disassembly parts to show how the Delphi Win32 compiler uses this for try finally blocks and try except blocks is below. Note that often, there are implicit try finally blocks when having managed method parameters or local variables.

–jeroen

Read the rest of this entry »

Posted in Delphi, Development, Software Development, Undocumented Delphi, Windows Development | Leave a Comment »

 
%d bloggers like this: