Answered @ Stackoverflow – on Parsing a record of unknown structure: use classes with published properties and the Delphi streaming mechanism
Posted by jpluimers on 2009/08/24
At Stackoverflow, user AB asked about Delphi: Parsing a record of unknown structure.
Basically his question came down to iterating over the fields of a record, then writing out the values to some sort of human readable file, and then reading them back in.
His idea was to use INI files, but also needed support for multi-line strings.
I suggested to use classes in stead of records, and published properties in stead of fields, then use the Delphi built-in streaming mechanism to stream to/from Delphi dfm files.
Normally, Delphi uses dfm files (they have been human readable text files since Delphi 6 or so) to store Forms, DataModules and Frames.
But why not use them to store your own components?
I got triggered by user dummzeuch who stated that records do not have RTTI in Delphi.
They do, but the RTTI on records is only available for records that need to be initialized/finalized (i.e. have a string or interface reference). Allen Bauer hints on this in his A “Nullable” post when explaining the FHasValue: IInterface idea.
That record RTTI is very thin: until Delphi 2010, it only exposes fields, as defined by these types in the implementation section of the System unit:
type
PPTypeInfo = ^PTypeInfo;
PTypeInfo = ^TTypeInfo;
TTypeInfo = packed record
Kind: Byte;
Name: ShortString;
{TypeData: TTypeData}
end;
TFieldInfo = packed record
TypeInfo: PPTypeInfo;
Offset: Cardinal;
end;
PFieldTable = ^TFieldTable;
TFieldTable = packed record
X: Word;
Size: Cardinal;
Count: Cardinal;
Fields: array [0..0] of TFieldInfo;
end;
I forgot to notice that dummzeuch (who in fact is Thomas Müller ex-Sysop of the Shuttle to Heaven BBS in Koblenz, Germany, and contributor of GExperts) suggested that classes with published properties using the Delphi DFM streaming mechanism would make this possible.
So I started this answer:
Records can have RTTI, but only if it has at least one reference counted field (like string or interface). And that record RTTI is very limited, so it won’t help you any furter.
At Delphi Live, it was shown that RTTI will be expanded a lot in Delphi 2010, so it might have expanded record RTTI as well.
I suggest you derive a class from TComponent and use the built-in streaming mechanism, or use RTTI to get it to/from an INI file. Much easier!
Lets start the example with a component to stream. It only has one property to be saved: IntegerValue. The actual class to be streamed is TIntegerValueComponent, which published the properties; the core logic is in TCustomIntegerValueComponent.
Splitting logic and publishing into two classes is the normal pattern to follow when writing components. Even in simple cases, it is wise to use it: refactoring that stuff out when it gets more complex in the future usually takes much more time.
unit IntegerValueComponentUnit;
interface
uses
Classes;
type
TCustomIntegerValueComponent = class(TComponent)
strict private
FIntegerValue: Integer;
strict protected
function GetIntegerValue: Integer; virtual;
procedure SetIntegerValue(const Value: Integer); virtual;
public
property IntegerValue: Integer read GetIntegerValue write SetIntegerValue;
end;
TIntegerValueComponent = class(TCustomIntegerValueComponent)
published
property IntegerValue;
end;
implementation
function TCustomIntegerValueComponent.GetIntegerValue: Integer;
begin
Result := FIntegerValue;
end;
procedure TCustomIntegerValueComponent.SetIntegerValue(const Value: Integer);
begin
FIntegerValue := Value;
end;
end.
A sample of a resulting dfm file is like this:
object TIntegerValueComponent IntegerValue = 33 end
That dfm file is reasonably easy to edit, like ini files, you can make errors, so it is not recommended to be edited by casual users.
You need this unit to get going, it does the basic streaming to/from streams, and the conversion of the binary format into the text format (if memory serves me right, the text format has been the default since Delphi 5 as it is much easier to read).
unit ComponentDfmUnit;
interface
uses
Classes, IntegerValueComponentUnit;
type
TComponentDfm = class
public
class function FromDfm(const Dfm: string): TComponent; static;
class function GetDfm(const Component: TComponent): string; static;
class function LoadFromDfm(const FileName: string): TComponent; static;
class procedure SaveToDfm(const Component: TComponent; const FileName: string); static;
end;
implementation
class function TComponentDfm.FromDfm(const Dfm: string): TComponent;
var
MemoryStream: TMemoryStream;
StringStream: TStringStream;
begin
MemoryStream := TMemoryStream.Create;
try
StringStream := TStringStream.Create(Dfm);
try
ObjectTextToBinary(StringStream, MemoryStream);
MemoryStream.Seek(0, soFromBeginning);
Result := MemoryStream.ReadComponent(nil);
finally
StringStream.Free;
end;
finally
MemoryStream.Free;
end;
end;
class function TComponentDfm.GetDfm(const Component: TComponent): string;
var
MemoryStream: TMemoryStream;
StringStream: TStringStream;
begin
MemoryStream := TMemoryStream.Create;
try
MemoryStream.WriteComponent(Component);
StringStream := TStringStream.Create('');
try
MemoryStream.Seek(0, soFromBeginning);
ObjectBinaryToText(MemoryStream, StringStream);
Result := StringStream.DataString;
finally
StringStream.Free;
end;
finally
MemoryStream.Free;
end;
end;
class function TComponentDfm.LoadFromDfm(const FileName: string): TComponent;
var
DfmStrings: TStrings;
begin
DfmStrings := TStringList.Create;
try
DfmStrings.LoadFromFile(FileName);
Result := TComponentDfm.FromDfm(DfmStrings.Text);
finally
DfmStrings.Free;
end;
end;
class procedure TComponentDfm.SaveToDfm(const Component: TComponent; const FileName: string);
var
DfmStrings: TStrings;
begin
DfmStrings := TStringList.Create;
try
DfmStrings.Text := TComponentDfm.GetDfm(Component);
DfmStrings.SaveToFile(FileName);
finally
DfmStrings.Free;
end;
end;
end.
And then this example should work (first the form code, then the form dfm): The most important line is RegisterClass(TIntegerValueComponent);, as it is easy to forget. The rest of the code is pretty straight forward.
As a bonus, you also see how you can copy a component to the clipboard and paste it back. It streams to/from the clipboard using the binary format.
const
FileName = 'IntegerValue.dfm';
procedure TStreamingDemoForm.ButtonEnabledTimerTimer(Sender: TObject);
begin
SaveButton.Enabled := SaveStyleRadioGroup.ItemIndex -1;
LoadButton.Enabled := SaveButton.Enabled;
if SaveStyleRadioGroup.ItemIndex = 0 then
LoadButton.Enabled := FileExists(FileName);
if SaveStyleRadioGroup.ItemIndex = 1 then
LoadButton.Enabled := Clipbrd.Clipboard.HasFormat(CF_COMPONENT);
end;
procedure TStreamingDemoForm.LoadButtonClick(Sender: TObject);
var
IntegerValueComponent: TIntegerValueComponent;
begin
IntegerValueComponent := nil;
if SaveStyleRadioGroup.ItemIndex = 0 then
IntegerValueComponent := LoadUsingFileStream()
else
IntegerValueComponent := LoadUsingClipboard();
try
if Assigned(IntegerValueComponent) then
Log('Loaded: %d', [IntegerValueComponent.IntegerValue])
else
Log('nil during Load');
finally
IntegerValueComponent.Free;
end;
end;
function TStreamingDemoForm.LoadUsingClipboard: TIntegerValueComponent;
var
Component: TComponent;
begin
Result := nil;
RegisterClass(TIntegerValueComponent);
Component := Clipboard.GetComponent(nil, nil);
if Assigned(Component) then
if Component is TIntegerValueComponent then
Result := TIntegerValueComponent(Component);
end;
function TStreamingDemoForm.LoadUsingFileStream: TIntegerValueComponent;
var
Component: TComponent;
begin
Result := nil;
RegisterClass(TIntegerValueComponent);
Component := TComponentDfm.LoadFromDfm(FileName);
if Assigned(Component) then
if Component is TIntegerValueComponent then
Result := TIntegerValueComponent(Component);
end;
procedure TStreamingDemoForm.Log(const Line: string);
begin
LogMemo.Lines.Add(Line);
end;
function TStreamingDemoForm.Log(const Mask: string; const Args: array of const): string;
begin
Log(Format(Mask, Args));
end;
procedure TStreamingDemoForm.SaveButtonClick(Sender: TObject);
var
IntegerValueComponent: TIntegerValueComponent;
begin
IntegerValueComponent := TIntegerValueComponent.Create(nil);
try
IntegerValueComponent.IntegerValue := ValueToSaveSpinEdit.Value;
if SaveStyleRadioGroup.ItemIndex = 0 then
SaveUsingFileStream(IntegerValueComponent)
else
SaveUsingClipboard(IntegerValueComponent);
Log('Saved: %d', [IntegerValueComponent.IntegerValue])
finally
IntegerValueComponent.Free;
end;
end;
procedure TStreamingDemoForm.SaveUsingClipboard(IntegerValueComponent: TIntegerValueComponent);
begin
Clipboard.SetComponent(IntegerValueComponent);
end;
procedure TStreamingDemoForm.SaveUsingFileStream(IntegerValueComponent: TIntegerValueComponent);
begin
TComponentDfm.SaveToDfm(IntegerValueComponent, Filename);
end;
Finally the form dfm:
object StreamingDemoForm: TStreamingDemoForm
Left = 0
Top = 0
Caption = 'StreamingDemoForm'
ClientHeight = 348
ClientWidth = 643
Color = clBtnFace
Font.Charset = DEFAULT_CHARSET
Font.Color = clWindowText
Font.Height = -11
Font.Name = 'Tahoma'
Font.Style = []
OldCreateOrder = False
DesignSize = (
643
348)
PixelsPerInch = 96
TextHeight = 13
object ValueToSaveLabel: TLabel
Left = 14
Top = 8
Width = 65
Height = 13
Caption = '&Value to save'
FocusControl = ValueToSaveSpinEdit
end
object ValueToSaveSpinEdit: TSpinEdit
Left = 95
Top = 5
Width = 121
Height = 22
MaxValue = 0
MinValue = 0
TabOrder = 0
Value = 0
end
object SaveButton: TButton
Left = 14
Top = 33
Width = 75
Height = 25
Caption = '&Save'
TabOrder = 1
OnClick = SaveButtonClick
end
object LoadButton: TButton
Left = 14
Top = 64
Width = 75
Height = 25
Caption = '&Load'
TabOrder = 2
OnClick = LoadButtonClick
end
object LogMemo: TMemo
Left = 14
Top = 95
Width = 621
Height = 245
Anchors = [akLeft, akTop, akRight, akBottom]
Lines.Strings = (
'LogMemo')
TabOrder = 3
end
object SaveStyleRadioGroup: TRadioGroup
Left = 95
Top = 30
Width = 121
Height = 59
Caption = 'Save st&yle'
Items.Strings = (
'Value.dfm'
'Clipboard')
TabOrder = 4
end
object ButtonEnabledTimer: TTimer
Interval = 100
OnTimer = ButtonEnabledTimerTimer
Left = 270
Top = 6
end
end
Note that normally we do not use dfm files to stream configurations to and from, but it an interesting concept.
It would be even more interesting when Delphi used xml files to store components in. Even without an xsd file, you could check if the xml files were well formed. With an xsd, dtd or other kind of definition, you could actually perform a syntax check.
That’s why we use XML files to store configuration in: easy to check to be well formed and/or valid. I wrote a bit about this in my post on getting the sourcefile name from a source file with an Assert trick using EAssertionFailed, but will elaborate on it in a future blog post.
Finally I want to point you to the answer to Smashers‘ own answer to his question What is the best way to serialize Delphi application configuration?
That shows how to to RTTI stuff yourself: it might even be extended to record RTTI.
Have fun with this example!
–jeroen
The Wiert Corner - irregular stream of stuff 
Delphi/Fortran memory allocation difference: row/column major order makes a big difference. StackOverflow answer. « The Wiert Corner – irregular stream of Wiert stuff said
[…] solutions use generics that were introduced in Delphi 2009. Until Delphi 2010, RTTI on records was very minimal), so you got the right Delphi version for both […]
CR said
It would be even more interesting when Delphi used xml files to store components in. Even without an xsd file, you could check if the xml files were well formed. With an xsd, dtd or other kind of definition, you could actually perform a syntax check.
I fail to see the benefit. Firstly, XML is more verbose and less human readable, increasing the likelihood of syntax errors being created if edited by hand. Secondly, why do you need an explicit syntax check? If there’s an error in the DFM, an exception will be cleanly raised – what more ‘syntax checking’ do you actually need?
jpluimers said
For dfm errors, some have to do with the form, some of the content.
Some dfm file corruptions (VSS anyone?) give error messages that do not point you to the real error (just like in some situations the Delphi compiler does not point you in the right direction).
Since in XML you both have the concept of being well formed (is the basic XML ok?) and the concept of being valid (does it conform to a schema like XSD or DTD) it makes it easier to write parsers and generators.
In the past, some people in and close to the Delphi team hinted that when XML had been more popular earlier on, that it would have made a lot of sense that the dfm files were in fact in a form of XML.
So I still think it would be an interesting idea :-)
–jeroen
pkulek said
I can see no reason why it would be harder to write a parser for the DFM file rather than for XML, also the DFM file is much easier to read than an XML file.
jpluimers said
There are many more tools available for handling XML than for handling DFM. But back when DFM got designed, XML was not as well established as it is now.
Since XML defines both “well formed” and “valid”, it is also easier to write XML than DFM (there are quite a few subtle things in DFM that are easy to get wrong, and hard to detect).
What you see is that Delphi does more an more XML (for instance, the .dproj files are now XML, they were not in the past), and I think that is a good thing.
But of course that is a personal opinion :-)
–jeroen