The Wiert Corner – irregular stream of stuff

Jeroen W. Pluimers on .NET, C#, Delphi, databases, and personal interests

  • My badges

    Mastodon

  • Twitter Updates

  • My Flickr Stream

    20140508-Delphi-2007--Project-Options--Cannot-Edit-Application-Title-HelpFile-Icon-Theming20140430-Fiddler-Filter-Actions-Button-Run-Filterset-now20140424-VMware-Fusion-6.0.3.-no-reclaimable
    More Photos

  • Pages

  • All categories

  • Enter your email address to subscribe to this blog and receive notifications of new posts by email.

    Join 1,861 other subscribers

Archive for the ‘Event’ Category

« Previous Entries
Next Entries »

How to debug Delphi JSON export stack-overflows: watch the fields and their circular references

Posted by jpluimers on 2020/12/23

Unlike Delphi RTL XML support which is property based, the JSON support is field based.

By default, JSON uses all fields (no matter their protection level, so anything from strict private to published  is taken into account).

When there are cycles, they are not detected: it will just stack-overflow with a high set of entries like this:

REST.JsonReflect.{REST.JsonReflect}TTypeMarshaller.MarshalSimpleField($788BFB40,$78AB0150)
REST.JsonReflect.{REST.JsonReflect}TTypeMarshaller.MarshalData($78AB0150)
REST.JsonReflect.{REST.JsonReflect}TTypeMarshaller.MarshalValue((($A9B7E8, Pointer($AFE168) as IValueData, 80, 336, 2024472912, $78AB0150, TClass($78AB0150), 80, 336, 2024472912, 2,77471682335019e+34, 1,00022251675539e-314, 0,00000000737961e-4933, 2024472912, 202447,2912, 2024472912, 2024472912, ($78AB0150, nil), $78AB0150)),???)
REST.JsonReflect.{REST.JsonReflect}TTypeMarshaller.MarshalSimpleField($78A921D0,$78AA69C0)
REST.JsonReflect.{REST.JsonReflect}TTypeMarshaller.MarshalData($78AA69C0)
REST.JsonReflect.{REST.JsonReflect}TTypeMarshaller.Marshal(???)

The easiest way to debug this is to:

  1. Set breakpoints in procedure TTypeMarshaller<TSerial>.MarshalData(Data: TObject);
    1. First breakpoint on the for rttiField loop
      • Watch or log these values (the first two usually are the same, the last two too):
        1. ComposeTypeName(Data) which gives you the fully qualified name (including unit and class) of the type exposing the fields
        2. Data.ClassName as a sanity check
        3. rttiType.Name which should be the same as Data.ClassName
    2. Second breakpoint inside the for rttiField loop on the if not ShouldMarshalstatement
      • Watch or log these values:
        1. rttiType.Name inside the loop, it changes value to match rttiField.Name, because of a debugger bug not showing it as E2171 Variable 'rttiType' inaccessible here due to optimization.
        2. rttiField.Name the actual field name

Tricks to circumvent circular references:

  • remember that fields with a reference to function value are not marshaled, so they are an excellent way of shoehorning in a dependency in (the reference value will be a capture which includes the instance data of the function to be called)
  • applying a [JsonMarshalled(False)] attribute (be sure to use unit REST.Json.Types!) only works when used inside non-generic types:
    • a class like TMySoapHeaderValue<T: IInterface> = class will not expose these attributes
    • a class like TMySoapHeaderValue = class will expose these attributes

You can check the JsonMarshalled problem by setting a breakpoint inside function LazyLoadAttributes(var P: PByte): TFunc<TArray<TCustomAttribute>>; on the line Exit(nil); and watch the value for Handle^.

Posted in Conference Topics, Conferences, Delphi, Development, Event, Software Development | Leave a Comment »

I really wish people reporting bugs provide more factual details, especially when asked for

Posted by jpluimers on 2020/12/15

Basically the below thread goes like this: [WayBack] GExperts / Bugs / #92 Grep cannot handle UTF-16 and UTF-32 pas files

  1. There is a bug in UTF-16 and UTF-32 handling in your tool when running under AAA, but  not when running your tool under BBB; these files fail: xxx.txt  and yyy.txt
  2. Which version of our tool did you run under AAA and which version of our tool did you run under BBB
  3. It fails with your tool when running under AAA , but succeeds under BBB
  4. Repeat at step 2 until you fall asleep.

Part of the [WayBack] Short, Self Contained, Correct Example are indeed in it, but without more details it is hard to reproduce.

So without the reporter providing those details, nobody, especially not on open source projects, is going to fix it just on that bug report.

Via: [WayBack] It’s time for a gift to all Delphi developers, a new Release of GExperts. Happy Holidays! (But do spend some time with your family rather than testing G… – Thomas Mueller (dummzeuch) – Google+

Which highlights another conceptual problem from the same bug reporter: expecting a new version to have a regression of all open bugs against the new version.

That’s not how the world works, if it has ever worked that way. If your issue is not mentioned in any release notes, then assume nothing happened. If you want to bump it up, then provide more details.

–jeroen

Posted in Conference Topics, Conferences, Development, Event, How to report bugs, Issue/Bug tracking, Software Development | Leave a Comment »

In Delphi, avoid having a TComponent descendant implement interfaces, unless you are prepared to handle your refcounting

Posted by jpluimers on 2020/12/15

Every now and then I see code, where a class descending from TComponent implements some interfaces, only interface references are used to the instances, and the expected behaviour is that the instances will free themselves when all the references went out of scope.

Bummer: TComponent by default is not reference counted.

It is when you assign VCLComObject an IVCLComObject which is hell to implement (Delphi provides two of them: TVCLAutoObject and TActiveFormControl. They sound heavey, and are).

Do not go that way. If you need some form of ownership in a class implementing an interface, then descend the class from TInterfacedObject, and add a field of TComponent that is the owner of things that need to be freed later on. In the destructor, free that field.

Something like this:

unit InterfacedObjectWithRootUnit;

interface

type
   TInterfacedObjectWithRoot = class(TInterfacedObject)
   strict private
      FRoot: TComponent;
      function GetRoot: TComponent;
   strict protected
      property Root: TComponent read GetRoot;
   public
      destructor Destroy; override;
   end;

implementation

destructor TInterfacedObjectWithRoot.Destroy;
begin
  if Assigned(FRoot) then
  begin
    FRoot.Free();
    FRoot := nil;
  end;
  inherited Destroy();
end;

function TInterfacedObjectWithRoot.GetRoot: TComponent;
begin
  if FRoot = nil then
    FRoot := TComponent.Create(nil);
  Result := FRoot;
end;

end.

–jeroen

Posted in Conference Topics, Conferences, Delphi, Development, Event, Software Development | Leave a Comment »

Delphi Compiler Intrinsics can help you collapse generated code for generics a lot

Posted by jpluimers on 2020/12/01

On my reading list [WayBack] Delphi Corner Weblog: The current state of generics in Delphi because it explains in more detail why Delphi compiler generic type based intrinsics introduced some 5 years ago, but never been (accurately) documented, but listed at [WayBack] delphi – Undocumented intrinsic routines – Stack Overflow, are so important to help collapse decreasing code bloat:

  • Default
  • IsManagedType
  • HasWeakRef
  • GetTypeKind

There is also the TypeInfo which is somewhat documented on-line starting in the Delphi 2007 documentation:

 

And some non-generic intrinsics that are still undocumented:

  • IsConstValue
  • ReturnAddress

Spring4D already has been making use of this for a few years, and you can too!

Via [WayBack] rvelthuis.blogspot.com/2018/10/the-current-state-of-generics-in-delphi.html – Jacek Laskowski – Google+ that has some more information on Spring4D too.

A tiny bit more information on collapsing: [WayBack] TURBU Tech » Blog Archive » Wish list: Generics collapsing

–jeroen

Posted in Conference Topics, Conferences, Delphi, Development, Event, Software Development | Leave a Comment »

A bunch of Spring4D dependency injection container related questions

Posted by jpluimers on 2020/11/19

Since G+ is down, a lot of interesting questions have vanished.

Luckily I saved some by [WayBack] Jacek Laskowski – Google+ related to Spring4D and dependency injection:

[WayBack] Spring4D IoC and specific singleton… – Jacek Laskowski – Google+

First my Google Groups archival reminder:

  1. Rename from
  2. To
  3. Save that URL with Archive.is

Read the rest of this entry »

Posted in Conference Topics, Conferences, Delphi, Development, Event, Software Development | Leave a Comment »

Do not make methods protected unless you want them to be visible as public

Posted by jpluimers on 2020/11/18

One of the protection levels in Delphi is protected. Originally meant for the class itself, that level is also visible to “friends”: anything in the same unit, for example:

unit BusinessLogicUnit;

interface

type
  TBusinessLogic = class(TObject)
  protected
     Procedure Foo();
     // ...
  public
     // ...
  end;

implementation

// ...

end.

You can even access them from outside that unit by using a trick like below.

Some people use the protected section so that unit tests can assess them using the below trick.

Do not do that!

It means anyone can use that trick, often doing more damage than good.

In this case, the trick was abused by a clever programmer that was relatively new to the code base. It resulted in unintended side effects.

unit HackUnit;

interface

implementation

uses
  BusinessLogicUnit;

type
  TBusinessLogicHack = class(TBusinessLogic);

procedure Hack;
var
  Instance: TBusinessLogicHack;
begin
  Instance := TBusinessLogicHack.Create();
  try
    Instance.Foo();
  finally
    Instance.Free();
  end;
end;

end.

Of course you can still access it like below.

It is slightly longer, but more importantly: much better shows the intent and how that intent is accomplished.

unit GoodUsageUnit;

interface

implementation

uses
  BusinessLogicUnit;

type
  TBusinessLogicDescendant = class(TBusinessLogic)
  public
    procedure Foo();
  end;

procedure TBusinessLogicDescendant.Foo();
begin
  inherited Foo();
  // ...
end;

procedure Usage;
var
  Instance: TBusinessLogicDescendant;
begin
  Instance := TBusinessLogicDescendant.Create();
  try
    Instance.Foo();
  finally
    Instance.Free();
  end;
end;

end.

–jeroen

Posted in Conference Topics, Conferences, Delphi, Development, Event, Software Development | 1 Comment »

delphi – What are the list of all possible values for DVCLAL? – Stack Overflow

Posted by jpluimers on 2020/11/17

From a while ago, from notes even longer ago – around 1994 on DVCLAL the Delphi VCL Access License code which is actually a checking logic for determining the SKU (stock keeping unit) or Delphi license: [WayBack] delphi – What are the list of all possible values for DVCLAL? – Stack Overflow

There is no official documentation on this, so here is something from my notes of 15+ years ago:

The DVCLAL is there to check which SKU of Delphi you are using and it varies per SKU.

There are only checks for the Professional (RPR) and Client/Server (RCS) SKUs:

procedure RCS;

procedure RPR;

If they fail, they call this method:

procedure ALV;
begin
  raise Exception.CreateRes(@SNL);
end;

where

resourcestring
  SNL = 'Application is not licensed to use this feature';

Depending on the feature matrix and Delphi version, various components call RPR and RCS in their Create constructors to guarantee a minimum SKU.

Underneath, these RPR and RCS functions call the function  GDAL . Their names are historic and got documented around Delphi 2007:

  • [WayBack] GDAL (Get Delphi Access License)
  • [WayBack] RPR will Restrict to PRofessional license and higher
  • [WayBack] RCS will Restrict to Client/Service license and higher

Historically you had these levels of Delphi editions that could be distinguished this way:

  1. Personal
  2. Professional
  3. Client/Server (or Enterprise)

This excludes Starter and Community (which are “just” Personal), Turbo (which was “just” Professional), Architect and Ultimate, which are “just” Client/Server with extra tools.

A few years ago, another answer got added to that question explaining more details:

I am just adding another answer to this question, for all the people who search the for actual DVCLAL (Delphi Visual Component Library Access License) values, as well as some other information for all people who are curious how stuff works.

1) Like Jeroen Wiert Pluimers said, if you want to check for “Professional or higher” or “Enterprise only” inside your Delphi application/library/package/component, you can use RPR (Require Professional) or RCS (“Require Client/Server”; Client/Server was the name for the Enterprise edition in early Delphi versions) respectively. If the requirement is not met, ALV (Access License Violation) will be called which will raise an Exception with the message defined in SysConst.SNL (S Not Licensed). In English:

Application is not licensed to use this feature

2) In case you want to check for one specific edition, you can use the output of the function GDAL (Get Delphi Access License), which is one of the following (AL1s array):

AL1s[0] = $FFFFFFF0; // Standard/Personal edition DVCLAL value
AL1s[1] = $FFFFEBF0; // Professional edition DVCLAL value
AL1s[2] = $00000000; // Enterprise/ClientServer edition DVCLAL value
AL1s[3] = $FFFFFFFF; // DVCLAL resource not existing

if the DVCLAL resource has an invalid value, GDAL will call ALVwhich will raise an Exception with message SysConst.SNL.

3) In case you want to check the DVCLAL value of a foreign EXE/DLL file (e.g. if you want to write a Resource Editor, decompiler etc), then you’ll have to query the DVCLAL resource directly.

There are only three official values:

Standard:      23 78 5D 23 B6 A5 F3 19 43 F3 40 02 26 D1 11 C7
Professional:  A2 8C DF 98 7B 3C 3A 79 26 71 3F 09 0F 2A 25 17
Enterprise:    26 3D 4F 38 C2 82 37 B8 F3 24 42 03 17 9B 3A 83

4) Just for fun: If you solve the formula 0 = (ROR(a,15) xor a) xor (ROR(b,10) xor b) xor (ROR(c,5) xor c) xor (AL1 xor AL2) you can define any DVCLAL value (tuple a, b, c, d) you want! (AL1 and AL2 are the values in the AL1s and AL2s arrays which describe the desired Delphi edition; ROR is rotate right through carry)

For example, here are alternative DVCLALs which work too:

Standard:      00 00 00 00 00 00 00 00 9B 70 0C 66 6B 8F F3 99
Professional:  00 00 00 00 00 00 00 00 9A DB 73 0F 6A 30 8C F0
Enterprise:    00 00 00 00 00 00 00 00 D8 B2 48 11 D8 B2 48 11

To validate a DVCLAL, you calculate

AL1 := DVCLAL[0] xor DVCLAL[1] xor DVCLAL[2] xor DVCLAL[3];
AL2 := ROR(DVCLAL[0],15) xor ROR(DVCLAL[1],10) xor ROR(DVCLAL[2],5) xor DVCLAL[3];

and look up AL1 and AL2 in the array AL1s and AL2s,

This way you can disguise the edition you have used a little.

5) In the meantime, an official documentation, at least for the functions GDAL, RPR and RCS, has been published.

6) Of course, everything works for C++ Builder, too.

In the mean time, new posts explaining bits of DVCLAL related resources (like PACKAGEINFO and CHARTABLE) and the TPF0 form/datamodule resource have appeared, of which this is a selection:

—jeroen

miscprogs.zip

Posted in Conference Topics, Conferences, Delphi, Development, Event, History, Software Development, Undocumented Delphi | Leave a Comment »

Archiving Google Product Forums URLs

Posted by jpluimers on 2020/11/13

Archiving Google Product Forum URLs is a pain in the butt for a couple of reasons:

So the trick for saving is:

  1. Get from the /forum/#!topic/ based URL to the /d/topic/ based one
  2. Put it after the archive.is/?run=1&url=, then save

--jeroen

 

Posted in Conference Topics, Conferences, Event, Internet, InternetArchive, Power User, WayBack machine | Leave a Comment »

Tracking down Delphi “abstract error” occassions.

Posted by jpluimers on 2020/11/12

A few tips.

  1. Set breakpoints in these units:
    • System.pas:
      procedure _AbstractError;
begin
  if Assigned(AbstractErrorProc) then
    AbstractErrorProc;
  RunErrorAt(210, ReturnAddress);
end;
    • System.SysUtils.pas:
      procedure AbstractErrorHandler;
begin
  raise EAbstractError.CreateRes(@SAbstractError);
end;
  2. add a watch for ClassName that has “Allow side effects and function calls” enabled
  3. run your code until it breaks or stops (usually in the first method above)
  4. double click on each entry in the “Call Stack” pane, and for each entry:
    • set a break point on the call to the method above it in the stack trace
    • note that sometimes this is on the line you arrived, but sometimes one more more lines higher in your code
    • the cause is that the stack trace will show you where your code will RETURN to, not the place it was CALLED FROM.
  5. abort your program
  6. run your program again
  7. on each hit breakpoint, watch the value of ClassName:
    • if it becomes [WayBack] E2003: Undeclared identifier: 'ClassName', then:
      • observe the method directives
      • if it includes static, but it is actually inside a class, then remove the static

Sometimes you cannot perform the last step: class property definitions have to be backed by static class methods.

This usually means you have a bad design anyway: you depend on global/singleton kind of behaviour that is almost impossible to properly test.

A better approach is to have regular object instances for that, then use constructor dependency injection (maybe combined with factory or dependency injection container) to setup the structures of dependencies.

TODO

Create a conference summary on the use of class versus instance methods, and static/regular.

Base materials:

–jeroen

Posted in Conference Topics, Conferences, Delphi, Development, Event, Software Development | Leave a Comment »

Be careful changing the DataContext of a TRemotable

Posted by jpluimers on 2020/11/11

Still not sure why the RTL code is like this:

procedure TRemotable.SetDataContext(Value: TDataContext);
begin
  if (RemotableDataContext <> nil) and (RemotableDataContext = Self.DataContext) then
  begin
    TDataContext(RemotableDataContext).RemoveObjectToDestroy(Self);
  end;
  FDataContext := Value;
end;

It means that if you ever have to change the DataContext property from the default global RemotableDataContext, it will be removed, but not added to the new DataContext.

When you assign it nil (so you can dump it to JSON, which often is easier to read than XML), and back to the old value, this results in a memory leak:

function TNotificationKeyPortTypeImplementation.prematchChanged(const prematchChangedRequest: prematchChangedRequest): prematchChangedResponse;
var
  DataContext: TDataContext;
begin
  // ...
  DataContext := prematchChangedRequest.DataContext;
  try
    prematchChangedRequest.DataContext := nil; // otherwise the JSON serializer will stackoverflow because DataContext points back to the TRemotable instance.
    Result := Format('prematchChanged: prematchChangedRequest=%s', [TJson.ObjectToJsonString(prematchChangedRequest)]);
  finally
    // `prematchChangedRequest.DataContext := nil` removed `prematchChangedRequest` from `DataContext.FObjsToDestroy`
    DataContext.AddObjectToDestroy(prematchChangedRequest);
    prematchChangedRequest.DataContext := DataContext; // does not add `prematchChangedRequest` to `DataContext.FObjsToDestroy`
  end;
end;

or when you are outside an incoming SOAP call where DataContext might not be assigned at all:

function ToJSON(const Value: TRemotable): string;
var
  DataContext: TDataContext;
  RemotableDataContext: Pointer;
begin
  if Assigned(Value) then
  begin
    DataContext := Value.DataContext;
    try
      Value.DataContext := nil; // otherwise the JSON serializer will stackoverflow because DataContext points back to the TRemotable instance.
      Result := TJson.ObjectToJsonString(Value);
      Result := TRegExSanitiser.ReplaceSecretInText(Result, [rttJSON]);
    finally
      // `Value.DataContext := nil` removed `Value` from `DataContext.FObjsToDestroy`
      if Assigned(DataContext) then
      begin
        RemotableDataContext := GetRemotableDataContext();
        if Assigned(RemotableDataContext) and (RemotableDataContext = DataContext) then
          DataContext.AddObjectToDestroy(Value);
        Value.DataContext := DataContext; // does not add `Value` to `DataContext.FObjsToDestroy`
      end;
    end;
  end
  else
    Result := '';
end;

–jeroen

Posted in Conference Topics, Conferences, Delphi, Development, Event, Software Development | Leave a Comment »

« Previous Entries
Next Entries »