Answered @ Stackoverflow – on Parsing a record of unknown structure: use classes with published properties and the Delphi streaming mechanism
Posted by jpluimers on 2009/08/24
At Stackoverflow, user AB asked about Delphi: Parsing a record of unknown structure.
Basically his question came down to iterating over the fields of a record, then writing out the values to some sort of human readable file, and then reading them back in.
His idea was to use INI files, but also needed support for multi-line strings.
I suggested to use classes in stead of records, and published properties in stead of fields, then use the Delphi built-in streaming mechanism to stream to/from Delphi dfm files.
Normally, Delphi uses dfm files (they have been human readable text files since Delphi 6 or so) to store Forms, DataModules and Frames.
But why not use them to store your own components?
I got triggered by user dummzeuch who stated that records do not have RTTI in Delphi.
They do, but the RTTI on records is only available for records that need to be initialized/finalized (i.e. have a string or interface reference). Allen Bauer hints on this in his A “Nullable” post when explaining the FHasValue: IInterface idea.
That record RTTI is very thin: until Delphi 2010, it only exposes fields, as defined by these types in the implementation section of the System unit:
type PPTypeInfo = ^PTypeInfo; PTypeInfo = ^TTypeInfo; TTypeInfo = packed record Kind: Byte; Name: ShortString; {TypeData: TTypeData} end; TFieldInfo = packed record TypeInfo: PPTypeInfo; Offset: Cardinal; end; PFieldTable = ^TFieldTable; TFieldTable = packed record X: Word; Size: Cardinal; Count: Cardinal; Fields: array [0..0] of TFieldInfo; end;
I forgot to notice that dummzeuch (who in fact is Thomas Müller ex-Sysop of the Shuttle to Heaven BBS in Koblenz, Germany, and contributor of GExperts) suggested that classes with published properties using the Delphi DFM streaming mechanism would make this possible.
So I started this answer:
Records can have RTTI, but only if it has at least one reference counted field (like string or interface). And that record RTTI is very limited, so it won’t help you any furter.
At Delphi Live, it was shown that RTTI will be expanded a lot in Delphi 2010, so it might have expanded record RTTI as well.
I suggest you derive a class from TComponent and use the built-in streaming mechanism, or use RTTI to get it to/from an INI file. Much easier!
Lets start the example with a component to stream. It only has one property to be saved: IntegerValue. The actual class to be streamed is TIntegerValueComponent, which published the properties; the core logic is in TCustomIntegerValueComponent.
Splitting logic and publishing into two classes is the normal pattern to follow when writing components. Even in simple cases, it is wise to use it: refactoring that stuff out when it gets more complex in the future usually takes much more time.
unit IntegerValueComponentUnit; interface uses Classes; type TCustomIntegerValueComponent = class(TComponent) strict private FIntegerValue: Integer; strict protected function GetIntegerValue: Integer; virtual; procedure SetIntegerValue(const Value: Integer); virtual; public property IntegerValue: Integer read GetIntegerValue write SetIntegerValue; end; TIntegerValueComponent = class(TCustomIntegerValueComponent) published property IntegerValue; end; implementation function TCustomIntegerValueComponent.GetIntegerValue: Integer; begin Result := FIntegerValue; end; procedure TCustomIntegerValueComponent.SetIntegerValue(const Value: Integer); begin FIntegerValue := Value; end; end.
A sample of a resulting dfm file is like this:
object TIntegerValueComponent IntegerValue = 33 end
That dfm file is reasonably easy to edit, like ini files, you can make errors, so it is not recommended to be edited by casual users.
You need this unit to get going, it does the basic streaming to/from streams, and the conversion of the binary format into the text format (if memory serves me right, the text format has been the default since Delphi 5 as it is much easier to read).
unit ComponentDfmUnit; interface uses Classes, IntegerValueComponentUnit; type TComponentDfm = class public class function FromDfm(const Dfm: string): TComponent; static; class function GetDfm(const Component: TComponent): string; static; class function LoadFromDfm(const FileName: string): TComponent; static; class procedure SaveToDfm(const Component: TComponent; const FileName: string); static; end; implementation class function TComponentDfm.FromDfm(const Dfm: string): TComponent; var MemoryStream: TMemoryStream; StringStream: TStringStream; begin MemoryStream := TMemoryStream.Create; try StringStream := TStringStream.Create(Dfm); try ObjectTextToBinary(StringStream, MemoryStream); MemoryStream.Seek(0, soFromBeginning); Result := MemoryStream.ReadComponent(nil); finally StringStream.Free; end; finally MemoryStream.Free; end; end; class function TComponentDfm.GetDfm(const Component: TComponent): string; var MemoryStream: TMemoryStream; StringStream: TStringStream; begin MemoryStream := TMemoryStream.Create; try MemoryStream.WriteComponent(Component); StringStream := TStringStream.Create(''); try MemoryStream.Seek(0, soFromBeginning); ObjectBinaryToText(MemoryStream, StringStream); Result := StringStream.DataString; finally StringStream.Free; end; finally MemoryStream.Free; end; end; class function TComponentDfm.LoadFromDfm(const FileName: string): TComponent; var DfmStrings: TStrings; begin DfmStrings := TStringList.Create; try DfmStrings.LoadFromFile(FileName); Result := TComponentDfm.FromDfm(DfmStrings.Text); finally DfmStrings.Free; end; end; class procedure TComponentDfm.SaveToDfm(const Component: TComponent; const FileName: string); var DfmStrings: TStrings; begin DfmStrings := TStringList.Create; try DfmStrings.Text := TComponentDfm.GetDfm(Component); DfmStrings.SaveToFile(FileName); finally DfmStrings.Free; end; end; end.
And then this example should work (first the form code, then the form dfm): The most important line is RegisterClass(TIntegerValueComponent);, as it is easy to forget. The rest of the code is pretty straight forward.
As a bonus, you also see how you can copy a component to the clipboard and paste it back. It streams to/from the clipboard using the binary format.
const FileName = 'IntegerValue.dfm'; procedure TStreamingDemoForm.ButtonEnabledTimerTimer(Sender: TObject); begin SaveButton.Enabled := SaveStyleRadioGroup.ItemIndex -1; LoadButton.Enabled := SaveButton.Enabled; if SaveStyleRadioGroup.ItemIndex = 0 then LoadButton.Enabled := FileExists(FileName); if SaveStyleRadioGroup.ItemIndex = 1 then LoadButton.Enabled := Clipbrd.Clipboard.HasFormat(CF_COMPONENT); end; procedure TStreamingDemoForm.LoadButtonClick(Sender: TObject); var IntegerValueComponent: TIntegerValueComponent; begin IntegerValueComponent := nil; if SaveStyleRadioGroup.ItemIndex = 0 then IntegerValueComponent := LoadUsingFileStream() else IntegerValueComponent := LoadUsingClipboard(); try if Assigned(IntegerValueComponent) then Log('Loaded: %d', [IntegerValueComponent.IntegerValue]) else Log('nil during Load'); finally IntegerValueComponent.Free; end; end; function TStreamingDemoForm.LoadUsingClipboard: TIntegerValueComponent; var Component: TComponent; begin Result := nil; RegisterClass(TIntegerValueComponent); Component := Clipboard.GetComponent(nil, nil); if Assigned(Component) then if Component is TIntegerValueComponent then Result := TIntegerValueComponent(Component); end; function TStreamingDemoForm.LoadUsingFileStream: TIntegerValueComponent; var Component: TComponent; begin Result := nil; RegisterClass(TIntegerValueComponent); Component := TComponentDfm.LoadFromDfm(FileName); if Assigned(Component) then if Component is TIntegerValueComponent then Result := TIntegerValueComponent(Component); end; procedure TStreamingDemoForm.Log(const Line: string); begin LogMemo.Lines.Add(Line); end; function TStreamingDemoForm.Log(const Mask: string; const Args: array of const): string; begin Log(Format(Mask, Args)); end; procedure TStreamingDemoForm.SaveButtonClick(Sender: TObject); var IntegerValueComponent: TIntegerValueComponent; begin IntegerValueComponent := TIntegerValueComponent.Create(nil); try IntegerValueComponent.IntegerValue := ValueToSaveSpinEdit.Value; if SaveStyleRadioGroup.ItemIndex = 0 then SaveUsingFileStream(IntegerValueComponent) else SaveUsingClipboard(IntegerValueComponent); Log('Saved: %d', [IntegerValueComponent.IntegerValue]) finally IntegerValueComponent.Free; end; end; procedure TStreamingDemoForm.SaveUsingClipboard(IntegerValueComponent: TIntegerValueComponent); begin Clipboard.SetComponent(IntegerValueComponent); end; procedure TStreamingDemoForm.SaveUsingFileStream(IntegerValueComponent: TIntegerValueComponent); begin TComponentDfm.SaveToDfm(IntegerValueComponent, Filename); end;
Finally the form dfm:
object StreamingDemoForm: TStreamingDemoForm Left = 0 Top = 0 Caption = 'StreamingDemoForm' ClientHeight = 348 ClientWidth = 643 Color = clBtnFace Font.Charset = DEFAULT_CHARSET Font.Color = clWindowText Font.Height = -11 Font.Name = 'Tahoma' Font.Style = [] OldCreateOrder = False DesignSize = ( 643 348) PixelsPerInch = 96 TextHeight = 13 object ValueToSaveLabel: TLabel Left = 14 Top = 8 Width = 65 Height = 13 Caption = '&Value to save' FocusControl = ValueToSaveSpinEdit end object ValueToSaveSpinEdit: TSpinEdit Left = 95 Top = 5 Width = 121 Height = 22 MaxValue = 0 MinValue = 0 TabOrder = 0 Value = 0 end object SaveButton: TButton Left = 14 Top = 33 Width = 75 Height = 25 Caption = '&Save' TabOrder = 1 OnClick = SaveButtonClick end object LoadButton: TButton Left = 14 Top = 64 Width = 75 Height = 25 Caption = '&Load' TabOrder = 2 OnClick = LoadButtonClick end object LogMemo: TMemo Left = 14 Top = 95 Width = 621 Height = 245 Anchors = [akLeft, akTop, akRight, akBottom] Lines.Strings = ( 'LogMemo') TabOrder = 3 end object SaveStyleRadioGroup: TRadioGroup Left = 95 Top = 30 Width = 121 Height = 59 Caption = 'Save st&yle' Items.Strings = ( 'Value.dfm' 'Clipboard') TabOrder = 4 end object ButtonEnabledTimer: TTimer Interval = 100 OnTimer = ButtonEnabledTimerTimer Left = 270 Top = 6 end end
Note that normally we do not use dfm files to stream configurations to and from, but it an interesting concept.
It would be even more interesting when Delphi used xml files to store components in. Even without an xsd file, you could check if the xml files were well formed. With an xsd, dtd or other kind of definition, you could actually perform a syntax check.
That’s why we use XML files to store configuration in: easy to check to be well formed and/or valid. I wrote a bit about this in my post on getting the sourcefile name from a source file with an Assert trick using EAssertionFailed, but will elaborate on it in a future blog post.
Finally I want to point you to the answer to Smashers‘ own answer to his question What is the best way to serialize Delphi application configuration?
That shows how to to RTTI stuff yourself: it might even be extended to record RTTI.
Have fun with this example!
–jeroen
Delphi/Fortran memory allocation difference: row/column major order makes a big difference. StackOverflow answer. « The Wiert Corner – irregular stream of Wiert stuff said
[…] solutions use generics that were introduced in Delphi 2009. Until Delphi 2010, RTTI on records was very minimal), so you got the right Delphi version for both […]
CR said
It would be even more interesting when Delphi used xml files to store components in. Even without an xsd file, you could check if the xml files were well formed. With an xsd, dtd or other kind of definition, you could actually perform a syntax check.
I fail to see the benefit. Firstly, XML is more verbose and less human readable, increasing the likelihood of syntax errors being created if edited by hand. Secondly, why do you need an explicit syntax check? If there’s an error in the DFM, an exception will be cleanly raised – what more ‘syntax checking’ do you actually need?
jpluimers said
For dfm errors, some have to do with the form, some of the content.
Some dfm file corruptions (VSS anyone?) give error messages that do not point you to the real error (just like in some situations the Delphi compiler does not point you in the right direction).
Since in XML you both have the concept of being well formed (is the basic XML ok?) and the concept of being valid (does it conform to a schema like XSD or DTD) it makes it easier to write parsers and generators.
In the past, some people in and close to the Delphi team hinted that when XML had been more popular earlier on, that it would have made a lot of sense that the dfm files were in fact in a form of XML.
So I still think it would be an interesting idea :-)
–jeroen
pkulek said
I can see no reason why it would be harder to write a parser for the DFM file rather than for XML, also the DFM file is much easier to read than an XML file.
jpluimers said
There are many more tools available for handling XML than for handling DFM. But back when DFM got designed, XML was not as well established as it is now.
Since XML defines both “well formed” and “valid”, it is also easier to write XML than DFM (there are quite a few subtle things in DFM that are easy to get wrong, and hard to detect).
What you see is that Delphi does more an more XML (for instance, the .dproj files are now XML, they were not in the past), and I think that is a good thing.
But of course that is a personal opinion :-)
–jeroen