The Wiert Corner – irregular stream of stuff

Jeroen W. Pluimers on .NET, C#, Delphi, databases, and personal interests

  • My badges

  • Twitter Updates

  • My Flickr Stream

  • Pages

  • All categories

  • Enter your email address to subscribe to this blog and receive notifications of new posts by email.

    Join 1,764 other followers

Answered @ Stackoverflow – on Parsing a record of unknown structure: use classes with published properties and the Delphi streaming mechanism

Posted by jpluimers on 2009/08/24

At Stackoverflow, user AB asked about Delphi: Parsing a record of unknown structure.

Basically his question came down to iterating over the fields of a record, then writing out the values to some sort of human readable file, and then reading them back in.

His idea was to use INI files, but also needed support for multi-line strings.
I suggested to use classes in stead of records, and published properties in stead of fields, then use the Delphi built-in streaming mechanism to stream to/from Delphi dfm files.

Normally, Delphi uses dfm files (they have been human readable text files since Delphi 6 or so) to store Forms, DataModules and Frames.
But why not use them to store your own components?

I got triggered by user dummzeuch who stated that records do not have RTTI in Delphi.
They do, but the RTTI on records is only available for records that need to be initialized/finalized (i.e. have a string or interface reference). Allen Bauer hints on this in his A “Nullable” post when explaining the FHasValue: IInterface idea.
That record RTTI is very thin: until Delphi 2010, it only exposes fields, as defined by these types in the implementation section of the System unit:

type
  PPTypeInfo = ^PTypeInfo;
  PTypeInfo = ^TTypeInfo;
  TTypeInfo = packed record
    Kind: Byte;
    Name: ShortString;
   {TypeData: TTypeData}
  end;

  TFieldInfo = packed record
    TypeInfo: PPTypeInfo;
    Offset: Cardinal;
  end;

  PFieldTable = ^TFieldTable;
  TFieldTable = packed record
    X: Word;
    Size: Cardinal;
    Count: Cardinal;
    Fields: array [0..0] of TFieldInfo;
  end;

I forgot to notice that dummzeuch (who in fact is Thomas Müller ex-Sysop of the Shuttle to Heaven BBS in Koblenz, Germany, and contributor of GExperts) suggested that classes with published properties using the Delphi DFM streaming mechanism would make this possible.

So I started this answer:

Records can have RTTI, but only if it has at least one reference counted field (like string or interface). And that record RTTI is very limited, so it won’t help you any furter.

At Delphi Live, it was shown that RTTI will be expanded a lot in Delphi 2010, so it might have expanded record RTTI as well.

I suggest you derive a class from TComponent and use the built-in streaming mechanism, or use RTTI to get it to/from an INI file. Much easier!

Lets start the example with a component to stream. It only has one property to be saved: IntegerValue. The actual class to be streamed is TIntegerValueComponent, which published the properties; the core logic is in TCustomIntegerValueComponent.
Splitting logic and publishing into two classes is the normal pattern to follow when writing components. Even in simple cases, it is wise to use it: refactoring that stuff out when it gets more complex in the future usually takes much more time.

unit IntegerValueComponentUnit;

interface

uses
  Classes;

type
  TCustomIntegerValueComponent = class(TComponent)
  strict private
    FIntegerValue: Integer;
  strict protected
    function GetIntegerValue: Integer; virtual;
    procedure SetIntegerValue(const Value: Integer); virtual;
  public
    property IntegerValue: Integer read GetIntegerValue write SetIntegerValue;
  end;

  TIntegerValueComponent = class(TCustomIntegerValueComponent)
  published
    property IntegerValue;
  end;

implementation

function TCustomIntegerValueComponent.GetIntegerValue: Integer;
begin
  Result := FIntegerValue;
end;

procedure TCustomIntegerValueComponent.SetIntegerValue(const Value: Integer);
begin
  FIntegerValue := Value;
end;

end.

A sample of a resulting dfm file is like this:

object TIntegerValueComponent
  IntegerValue = 33
end

That dfm file is reasonably easy to edit, like ini files, you can make errors, so it is not recommended to be edited by casual users.

You need this unit to get going, it does the basic streaming to/from streams, and the conversion of the binary format into the text format (if memory serves me right, the text format has been the default since Delphi 5 as it is much easier to read).

unit ComponentDfmUnit;

interface

uses
  Classes, IntegerValueComponentUnit;

type
  TComponentDfm = class
  public
    class function FromDfm(const Dfm: string): TComponent; static;
    class function GetDfm(const Component: TComponent): string; static;
    class function LoadFromDfm(const FileName: string): TComponent; static;
    class procedure SaveToDfm(const Component: TComponent; const FileName: string); static;
  end;

implementation

class function TComponentDfm.FromDfm(const Dfm: string): TComponent;
var
  MemoryStream: TMemoryStream;
  StringStream: TStringStream;
begin
  MemoryStream := TMemoryStream.Create;
  try
    StringStream := TStringStream.Create(Dfm);
    try
      ObjectTextToBinary(StringStream, MemoryStream);
      MemoryStream.Seek(0, soFromBeginning);
      Result := MemoryStream.ReadComponent(nil);
    finally
      StringStream.Free;
    end;
  finally
    MemoryStream.Free;
  end;
end;

class function TComponentDfm.GetDfm(const Component: TComponent): string;
var
  MemoryStream: TMemoryStream;
  StringStream: TStringStream;
begin
  MemoryStream := TMemoryStream.Create;
  try
    MemoryStream.WriteComponent(Component);
    StringStream := TStringStream.Create('');
    try
      MemoryStream.Seek(0, soFromBeginning);
      ObjectBinaryToText(MemoryStream, StringStream);
      Result := StringStream.DataString;
    finally
      StringStream.Free;
    end;
  finally
    MemoryStream.Free;
  end;
end;

class function TComponentDfm.LoadFromDfm(const FileName: string): TComponent;
var
  DfmStrings: TStrings;
begin
  DfmStrings := TStringList.Create;
  try
    DfmStrings.LoadFromFile(FileName);
    Result := TComponentDfm.FromDfm(DfmStrings.Text);
  finally
    DfmStrings.Free;
  end;
end;

class procedure TComponentDfm.SaveToDfm(const Component: TComponent; const FileName: string);
var
  DfmStrings: TStrings;
begin
  DfmStrings := TStringList.Create;
  try
    DfmStrings.Text := TComponentDfm.GetDfm(Component);
    DfmStrings.SaveToFile(FileName);
  finally
    DfmStrings.Free;
  end;
end;

end.

And then this example should work (first the form code, then the form dfm): The most important line is RegisterClass(TIntegerValueComponent);, as it is easy to forget. The rest of the code is pretty straight forward.

As a bonus, you also see how you can copy a component to the clipboard and paste it back. It streams to/from the clipboard using the binary format.

const
  FileName = 'IntegerValue.dfm';

procedure TStreamingDemoForm.ButtonEnabledTimerTimer(Sender: TObject);
begin
  SaveButton.Enabled := SaveStyleRadioGroup.ItemIndex  -1;
  LoadButton.Enabled := SaveButton.Enabled;
  if SaveStyleRadioGroup.ItemIndex = 0 then
    LoadButton.Enabled := FileExists(FileName);
  if SaveStyleRadioGroup.ItemIndex = 1 then
    LoadButton.Enabled := Clipbrd.Clipboard.HasFormat(CF_COMPONENT);
end;

procedure TStreamingDemoForm.LoadButtonClick(Sender: TObject);
var
  IntegerValueComponent: TIntegerValueComponent;
begin
  IntegerValueComponent := nil;
  if SaveStyleRadioGroup.ItemIndex = 0 then
    IntegerValueComponent := LoadUsingFileStream()
  else
    IntegerValueComponent := LoadUsingClipboard();
  try
    if Assigned(IntegerValueComponent) then
      Log('Loaded: %d', [IntegerValueComponent.IntegerValue])
    else
      Log('nil during Load');
  finally
    IntegerValueComponent.Free;
  end;
end;

function TStreamingDemoForm.LoadUsingClipboard: TIntegerValueComponent;
var
  Component: TComponent;
begin
  Result := nil;
  RegisterClass(TIntegerValueComponent);
  Component := Clipboard.GetComponent(nil, nil);
  if Assigned(Component) then
    if Component is TIntegerValueComponent then
      Result := TIntegerValueComponent(Component);
end;

function TStreamingDemoForm.LoadUsingFileStream: TIntegerValueComponent;
var
  Component: TComponent;
begin
  Result := nil;
  RegisterClass(TIntegerValueComponent);
  Component := TComponentDfm.LoadFromDfm(FileName);
  if Assigned(Component) then
    if Component is TIntegerValueComponent then
      Result := TIntegerValueComponent(Component);
end;

procedure TStreamingDemoForm.Log(const Line: string);
begin
  LogMemo.Lines.Add(Line);
end;

function TStreamingDemoForm.Log(const Mask: string; const Args: array of const): string;
begin
  Log(Format(Mask, Args));
end;

procedure TStreamingDemoForm.SaveButtonClick(Sender: TObject);
var
  IntegerValueComponent: TIntegerValueComponent;
begin
  IntegerValueComponent := TIntegerValueComponent.Create(nil);
  try
    IntegerValueComponent.IntegerValue := ValueToSaveSpinEdit.Value;
    if SaveStyleRadioGroup.ItemIndex = 0 then
      SaveUsingFileStream(IntegerValueComponent)
    else
      SaveUsingClipboard(IntegerValueComponent);
    Log('Saved: %d', [IntegerValueComponent.IntegerValue])
  finally
    IntegerValueComponent.Free;
  end;
end;

procedure TStreamingDemoForm.SaveUsingClipboard(IntegerValueComponent: TIntegerValueComponent);
begin
  Clipboard.SetComponent(IntegerValueComponent);
end;

procedure TStreamingDemoForm.SaveUsingFileStream(IntegerValueComponent: TIntegerValueComponent);
begin
  TComponentDfm.SaveToDfm(IntegerValueComponent, Filename);
end;

Finally the form dfm:

object StreamingDemoForm: TStreamingDemoForm
  Left = 0
  Top = 0
  Caption = 'StreamingDemoForm'
  ClientHeight = 348
  ClientWidth = 643
  Color = clBtnFace
  Font.Charset = DEFAULT_CHARSET
  Font.Color = clWindowText
  Font.Height = -11
  Font.Name = 'Tahoma'
  Font.Style = []
  OldCreateOrder = False
  DesignSize = (
    643
    348)
  PixelsPerInch = 96
  TextHeight = 13
  object ValueToSaveLabel: TLabel
    Left = 14
    Top = 8
    Width = 65
    Height = 13
    Caption = '&Value to save'
    FocusControl = ValueToSaveSpinEdit
  end
  object ValueToSaveSpinEdit: TSpinEdit
    Left = 95
    Top = 5
    Width = 121
    Height = 22
    MaxValue = 0
    MinValue = 0
    TabOrder = 0
    Value = 0
  end
  object SaveButton: TButton
    Left = 14
    Top = 33
    Width = 75
    Height = 25
    Caption = '&Save'
    TabOrder = 1
    OnClick = SaveButtonClick
  end
  object LoadButton: TButton
    Left = 14
    Top = 64
    Width = 75
    Height = 25
    Caption = '&Load'
    TabOrder = 2
    OnClick = LoadButtonClick
  end
  object LogMemo: TMemo
    Left = 14
    Top = 95
    Width = 621
    Height = 245
    Anchors = [akLeft, akTop, akRight, akBottom]
    Lines.Strings = (
      'LogMemo')
    TabOrder = 3
  end
  object SaveStyleRadioGroup: TRadioGroup
    Left = 95
    Top = 30
    Width = 121
    Height = 59
    Caption = 'Save st&yle'
    Items.Strings = (
      'Value.dfm'
      'Clipboard')
    TabOrder = 4
  end
  object ButtonEnabledTimer: TTimer
    Interval = 100
    OnTimer = ButtonEnabledTimerTimer
    Left = 270
    Top = 6
  end
end

Note that normally we do not use dfm files to stream configurations to and from, but it an interesting concept.
It would be even more interesting when Delphi used xml files to store components in. Even without an xsd file, you could check if the xml files were well formed. With an xsd, dtd or other kind of definition, you could actually perform a syntax check.

That’s why we use XML files to store configuration in: easy to check to be well formed and/or valid. I wrote a bit about this in my post on getting the sourcefile name from a source file with an Assert trick using EAssertionFailed, but will elaborate on it in a future blog post.

Finally I want to point you to the answer to Smashers‘ own answer to his question What is the best way to serialize Delphi application configuration?
That shows how to to RTTI stuff yourself: it might even be extended to record RTTI.

Have fun with this example!

–jeroen

5 Responses to “Answered @ Stackoverflow – on Parsing a record of unknown structure: use classes with published properties and the Delphi streaming mechanism”

  1. […] solutions use generics that were introduced in Delphi 2009. Until Delphi 2010, RTTI on records was very minimal), so you got the right Delphi version for both […]

  2. CR said

    It would be even more interesting when Delphi used xml files to store components in. Even without an xsd file, you could check if the xml files were well formed. With an xsd, dtd or other kind of definition, you could actually perform a syntax check.

    I fail to see the benefit. Firstly, XML is more verbose and less human readable, increasing the likelihood of syntax errors being created if edited by hand. Secondly, why do you need an explicit syntax check? If there’s an error in the DFM, an exception will be cleanly raised – what more ‘syntax checking’ do you actually need?

    • jpluimers said

      For dfm errors, some have to do with the form, some of the content.
      Some dfm file corruptions (VSS anyone?) give error messages that do not point you to the real error (just like in some situations the Delphi compiler does not point you in the right direction).

      Since in XML you both have the concept of being well formed (is the basic XML ok?) and the concept of being valid (does it conform to a schema like XSD or DTD) it makes it easier to write parsers and generators.

      In the past, some people in and close to the Delphi team hinted that when XML had been more popular earlier on, that it would have made a lot of sense that the dfm files were in fact in a form of XML.

      So I still think it would be an interesting idea :-)

      –jeroen

      • pkulek said

        I can see no reason why it would be harder to write a parser for the DFM file rather than for XML, also the DFM file is much easier to read than an XML file.

        • jpluimers said

          There are many more tools available for handling XML than for handling DFM. But back when DFM got designed, XML was not as well established as it is now.
          Since XML defines both “well formed” and “valid”, it is also easier to write XML than DFM (there are quite a few subtle things in DFM that are easy to get wrong, and hard to detect).

          What you see is that Delphi does more an more XML (for instance, the .dproj files are now XML, they were not in the past), and I think that is a good thing.
          But of course that is a personal opinion :-)

          –jeroen

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

This site uses Akismet to reduce spam. Learn how your comment data is processed.

 
%d bloggers like this: