The Wiert Corner – irregular stream of stuff

Jeroen W. Pluimers on .NET, C#, Delphi, databases, and personal interests

  • My badges

  • Twitter Updates

  • My Flickr Stream

  • Pages

  • All categories

  • Enter your email address to subscribe to this blog and receive notifications of new posts by email.

    Join 1,861 other subscribers

Archive for the ‘Conference Topics’ Category

DUnit testing code that should raise a specific exception

Posted by jpluimers on 2021/02/17

A while back, I was writing some code to demonstrate a few inner workings of TInterfacedObject, interface reference counting and mixing object references with interface references.

One way to show this is through a test case that expects a certain exception to happen, but I forgot how to do that in DUnit. Luckily this pointed me on the right track: [WayBack] delphi – CheckException only accepts 0-parameter methods; how do I test that other methods throw exceptions? – Stack Overflow.

The solution shows that DUnit has had support for something similar as DUnitX: now has a WillRaiseAttribute to ease defining tests around code that should throw exceptions for a very long time (I think this was introduced around Delphi 2005).

You can do it in a property way:

unit InterfacedObjectTestCaseUnit;

interface

uses
  TestFramework;

type
  TDebuggableInterfacedObjectTestCase = class(TTestCase)
  published
    procedure System_TInterfacedObject_Free_Before_RefCount_Should_Raise_EInvalidPointer();
  end;

implementation

uses
  System.SysUtils,

procedure TInterfacedObjectTestCase.System_TInterfacedObject_Free_Before_RefCount_Should_Raise_EInvalidPointer();
var
  ObjectReference:    System.TInterfacedObject;
  InterfaceReference: IInterface;
begin
  ObjectReference    := System.TInterfacedObject.Create();
  InterfaceReference := ObjectReference;
  ExpectedException  := System.SysUtils.EInvalidPointer;
  ObjectReference.Free(); // this should raise an exception in System.TInterfacedObject.BeforeDestruction, as it checks the RefCount to be zero
  // the below is optional; should not be reached. If it is reached, it will fail earlier than the encompassing `RunTest` method would
  ExpectedException := nil; // or `StopExpectingException();`
end;

end.

or in a method way for an exception that happens in the current method:

procedure TInterfacedObjectTestCase.System_TInterfacedObject_Free_Before_RefCount_Should_Raise_EInvalidPointer();
var
  ObjectReference:    System.TInterfacedObject;
  InterfaceReference: IInterface;
begin
  ObjectReference    := System.TInterfacedObject.Create();
  InterfaceReference := ObjectReference;
  StartExpectingException(System.SysUtils.EInvalidPointer);
  ObjectReference.Free(); // this should raise an exception in System.TInterfacedObject.BeforeDestruction, as it checks the RefCount to be zero
  // the below is optional; should not be reached. If it is reached, it will fail earlier than the encompassing `RunTest` method would
  StopExpectingException();
end;

The alternative using CheckException that will raise earlier, but also tests the results of a complete method which also has to be parameterless:

procedure TDebuggableInterfacedObjectTestCase.System_TInterfacedObject_Free_Before_RefCount();
var
  ObjectReference:    System.TInterfacedObject;
  InterfaceReference: IInterface;
begin
  ObjectReference    := System.TInterfacedObject.Create();
  InterfaceReference := ObjectReference;
  ObjectReference.Free(); // this should raise an exception in System.TInterfacedObject.BeforeDestruction, as it checks the RefCount to be zero
end;

procedure TDebuggableInterfacedObjectTestCase.System_TInterfacedObject_Free_Before_RefCount_Should_Raise_EInvalidPointer_TTestMethod_Based();
begin
  CheckException(System_TInterfacedObject_Free_Before_RefCount, System.SysUtils.EInvalidPointer);
end;

So I wrote a class helper based on TProc that allows you to test an anonymous method which usually has more fine grained testing potential.

Because of type compatibility, you have to call the inherited version of CheckException inside the new one:

unit TestCaseHelperUnit;

interface

uses
  System.SysUtils,
  TestFramework;

type
  TTestCaseHelper = class helper for TTestCase
  public
    procedure CheckException(const AProc: TProc; const AExceptionClass: TClass; const msg: string = '');
  end;

implementation

type
  TTestCaseInvoker = class
  strict private
    FProc: TProc;
  public
    constructor Create(const AProc: TProc);
    procedure Execute();
  end;

{ TTestCaseInvoker }

constructor TTestCaseInvoker.Create(const AProc: TProc);
begin
  inherited Create();
  FProc := AProc;
end;

procedure TTestCaseInvoker.Execute();
begin
  if Assigned(FProc) then
    FProc();
end;

procedure TTestCaseHelper.CheckException(const AProc: TProc; const AExceptionClass: TClass; const msg: string = '');
var
  TestCaseInvoker: TTestCaseInvoker;
begin
  TestCaseInvoker := TTestCaseInvoker.Create(AProc);
  try
    inherited CheckException(TestCaseInvoker.Execute, AExceptionClass, msg); // `inherited`, to avoid stack overflow because `TProc` is compatible with `TTestMethod`
  finally
    TestCaseInvoker.Free();
  end;
end;

end.

The test then becomes this:

procedure TDebuggableInterfacedObjectTestCase.System_TInterfacedObject_Free_Before_RefCount_Should_Raise_EInvalidPointer_TProc_Based();
var
  ObjectReference:    System.TInterfacedObject;
  InterfaceReference: IInterface;
begin
  ObjectReference    := System.TInterfacedObject.Create();
  InterfaceReference := ObjectReference;
  CheckException(procedure ()
  begin
    ObjectReference.Free(); // this should raise an exception in System.TInterfacedObject.BeforeDestruction, as it checks the RefCount to be zero
  end,
  System.SysUtils.EInvalidPointer);
end;

DUnit code snippets

Read the rest of this entry »

Posted in Agile, Conference Topics, Conferences, Delphi, Development, Event, Software Development, Unit Testing | Leave a Comment »

Checking expiration dates for your certbot certificates

Posted by jpluimers on 2021/02/16

I have these two little aliases in my toolbox:

alias "certbot-check-all-by-file=bash <(curl -fsSL https://raw.githubusercontent.com/srvrco/checkssl/master/checkssl) --location /etc/letsencrypt/live"
alias "certbot-check-all-by-config=bash <(curl -fsSL https://raw.githubusercontent.com/srvrco/checkssl/master/checkssl) --server ISPconfig"

First a big fat warning

do not run just any script downloaded through curl. Plenty of reasons why this is dangerous:

  1. [WayBack] Detecting the use of “curl | bash” server side | Application Security
  2. [WayBack] One way “curl pipe sh” install scripts can be dangerous [proof of concept] / Jordan Eldredge: script content differs depending on user agent
  3. [WayBack] sean cassidy : Don’t Pipe to your Shell: scripts having different behaviour when executed partially
  4. [WayBack] Why using curl | sudo sh is not advised? – Stack Overflow:

    You can proof your scripts against partial execution by putting the whole thing into the body of a function, and executing that function on the last line. If you know a script is defined like that, it’s exactly as secure as downloading and then executing some installer.

The first three can mostly prevented by using your own fork of the script repository, then checking each modification of the script, combined with ensuring your fork location does not throw tricks 1 or 2 on you.

That’s why I run the above alias only from a checkssl fork which I can inspect.

Back to the alias

The aliases use quite a few tricks:

  1. Having curl download a command minimising
    curl -fsSL https://raw.githubusercontent.com/srvrco/checkssl/master/checkssl

    Via: [WayBack] The missing package manager for macOS (or Linux) — The missing package manager for macOS (or Linux)

  2. Running that command through bash as if the download were a file by wrapping wrapping it in parenthesis and a less than sign .
    bash <(curl -fsSL https://raw.githubusercontent.com/srvrco/checkssl/master/checkssl)

    Via: [WayBack] linux – Execute bash script from URL – Stack Overflow

  3. Passing arguments to the bash command by appending this to the command just like from the regular command-line:
    bash <(curl -fsSL https://raw.githubusercontent.com/srvrco/checkssl/master/checkssl) --location /etc/letsencrypt/live
  4. An alias [WayBack] with double-quotes around the whole statement:
    alias "certbot-check-all-by-file=bash <(curl -fsSL https://raw.githubusercontent.com/srvrco/checkssl/master/checkssl) --location /etc/letsencrypt/live"
  5. Either use the certbot configuration file or apache2 (via ISPconfig as the apache2 parameter value is not yet supported) domain configuration:
    alias "certbot-check-all-by-file=bash <(curl -fsSL https://raw.githubusercontent.com/srvrco/checkssl/master/checkssl) --location /etc/letsencrypt/live"
    alias "certbot-check-all-by-config=bash <(curl -fsSL https://raw.githubusercontent.com/srvrco/checkssl/master/checkssl) --server ISPconfig"

Using source instead of bash

Note that an alternative alias is this one:

alias "certbot-check-all-by-file=(source <(curl -s https://raw.githubusercontent.com/srvrco/checkssl/master/checkssl) --location /etc/letsencrypt/live)"

However, that needs an extra set of parenthesis, otherwise you will get bumped out of your current shell.

The reason is that bash runs in a [WayBack] subshell, whereas [WayBack] source (and the equivalent [WayBack] “dot” command .) runs in the current shell, but the script performs a gracefull_exit or error_exit which end in an exit [WayBack] terminating the current shell.

The [WayBack] () parenthesis around the source command ensure it runs in a sub-shell.

In this case, you can still pass the --location /etc/letsencrypt/live parameters, as bash is the only shell allowing this: [WayBack] source – Passing variables to a bash script when sourcing it – Unix & Linux Stack Exchange.

References

Related [WayBack] Advanced Bash-Scripting Guide topics:

Related cURL options from [WayBack] curl – How To Use:

  • -f: [WayBack] -f, --fail

    (HTTP) Fail silently (no output at all) on server errors. This is mostly done to better enable scripts etc to better deal with failed attempts. In normal cases when an HTTP server fails to deliver a document, it returns an HTML document stating so (which often also describes why and more). This flag will prevent curl from outputting that and return error 22.

    This method is not fail-safe and there are occasions where non-successful response codes will slip through, especially when authentication is involved (response codes 401 and 407).

  • -s: [WayBack] -s, --silent:

    Silent or quiet mode. Don’t show progress meter or error messages. Makes Curl mute. It will still output the data you ask for, potentially even to the terminal/stdout unless you redirect it.

    Use -S, –show-error in addition to this option to disable progress meter but still show error messages.

    See also -v, –verbose and –stderr.

  • -S: [WayBack] -S, --show-errors:

    When used with -s, –silent, it makes curl show an error message if it fails.

  • -L: [WayBack] -L, --location:

    (HTTP) If the server reports that the requested page has moved to a different location (indicated with a Location: header and a 3XX response code), this option will make curl redo the request on the new place. If used together with -i, –include or -I, –head, headers from all requested pages will be shown. When authentication is used, curl only sends its credentials to the initial host. If a redirect takes curl to a different host, it won’t be able to intercept the user+password. See also –location-trusted on how to change this. You can limit the amount of redirects to follow by using the –max-redirs option.

    When curl follows a redirect and the request is not a plain GET (for example POST or PUT), it will do the following request with a GET if the HTTP response was 301, 302, or 303. If the response code was any other 3xx code, curl will re-send the following request using the same unmodified method.

    You can tell curl to not change the non-GET request method to GET after a 30x response by using the dedicated options for that: –post301–post302 and –post303.

Reminders

Reminder to self: see if JSON output is viable. This commit might help.

–jeroen

Posted in bash, Conference Topics, Conferences, Development, Event, Scripting, Software Development | Leave a Comment »

The magic “procedure Touch(var …” construct

Posted by jpluimers on 2021/02/16

procedure Touch(var Argument);  
begin
end;

I included the above code in my blog a long time ago (2014 in fact: Delphi: always watch the compiler Warnings), but never got around to explain the why and how I got it, why it works and why it will likely work forever.

Background

Ever since the early Delphi days, there are three hints about “never used” of which the second often gets in the way during debugging:

(note that these %s only hold for non-managed types, which I also addressed in Why don’t I get the warning W1036 Variable “‘MyStrings’ might not have been initialized”… and Delphi 10.3 Rio got released; I’ll wait a while hoping to see more positive comments).

Usually the compiler is right, but sometimes it is not: [WayBack] Check your compiler warnings and hints. They may still be errors. | Shiftkey Software Blog

So once every while, you need this workaround:

Solution

The solution is to have a method with one untyped var parameter (a var parameter without any type: this way you can pass any field or variable to it) that just does nothing. Often I included only at the place I need it as this single line fragment: procedure Touch(var Argument); begin end;.

Former Delphi compiler engineer and Borland Chief Schientist Danny Thorpe handed this solution, I think it was during or around his famous BorCon99 in Philadelphi (and later BorCon2005 in San Jose) Reading Tea Leaves: The Fine Art of Debugging talk. The talk is not-online, but luckily there are notes and a StackOverflow post:

The session had seemingly simple things like this [WayBack] Shenoy At Work: Set Next Statement in Delphi? with the picture on the right.

Voiding the solution

I’ve seen teams making this method inline, but that voids it. Usually they do not see it as they already resolved the “never used” problem in another way.

Why it still works

Read the rest of this entry »

Posted in Conference Topics, Conferences, Delphi, Development, Event, Software Development | 1 Comment »

One second code: Do YOU know how much your computer can do in a second?

Posted by jpluimers on 2021/02/11

[WayBack] One second code: Do YOU know how much your computer can do in a second? is a quiz version of the [WayBack] Numbers Every Programmer Should Know By Year.

[WayBack] About this game revealed it was made by 3 people curious in the speed of their hardware which – not surprisingly – has been relatively stable over the last decade or so.

Source code is at [WayBack] GitHub – kamalmarhubi/one-second: Fun performance game!

I bumped into it via these tweets:

I like games like this (ever played the The Deadlock Empire multi-threading game?), so I played the computers-are-fast.github.io tests, and learned a lot:

Read the rest of this entry »

Posted in Conference Topics, Conferences, Development, Event, Hardware, Power User, Software Development | Leave a Comment »

The Delphi interface reference counting compiler bug that will never get fixed as it is marked “as designed”

Posted by jpluimers on 2021/02/10

A long time ago, I write a question [WayBack] delphi – Should the compiler hint/warn when passing object instances directly as const interface parameters? – Stack Overflow in 2010.

It was marked by (now former) compiler engineer Barry Kelly [WayBack1/WayBack2] as bug:

It’s a bug. The conversion from instance to interface reference in RunLeakCrash should be to a temporary variable, keeping it alive for the duration of RunLeakCrash.

Added to that was a comment that this has happened since at least Delphi 6, so I filed a bug WayBack QualityCentral Report #: 90482 The compiler should keep a hidden reference when passing freshly created object instances directly as const interface parameters.

Just for years later, it was marked with “As Designed” which means it is never going to be fixed, especially since in the mean time Embarcadero got rid of most the senior Delphi R&D team members and went down the path of hiring contractors.

The problem is that I run into the two manifestations every now and then, and it usually takes a long time debugging to zoom into the actual location of the spots.

First manifestation: crash

This is the bug in the first manifestation (by now the root interface is IInterface instead of IUnknown, and you usually have an interface derived from it):

Read the rest of this entry »

Posted in Conference Topics, Conferences, Delphi, Development, Event, Software Development, Undocumented Delphi | Leave a Comment »

If you use an implementation of TNonRefCountInterfacedObject, then document in the descendants how lifetime management is arranged for

Posted by jpluimers on 2021/01/21

There are a few TNonRefCountInterfacedObject (or maybe better named TNonReferenceCountedInterfacedObject) implementations around (see list of links below).

They can be used to expose interfaces, but do not provide interface reference counting. This means you have to do your own lifetime management, which can bring quite a few headaches.

So each class you descend from it must have proper motivation on why, and how lifetime management is performed.

One thing you can do is mark the class with a hint directive like [WayBack] library.

In addition, [Archive.is] TNonRefCountInterfacedObject / [Archive.is] TNonReferenceCountedInterfacedObject implementations should at least implement [WayBack] IInterface (or [WayBack] IUnknown in really old Delphi versions); I have seen implementations that don’t but just provide almost empty [WayBackQueryInterface, [WayBack] _AddRef and [WayBack] _Release methods).

Some examples via  “TNonRefCountInterfacedObject” – Google Search:

Delphi RTL/VLC/FMX

I used this GExperts Grep Search expression to find the entries below: (_AddRef|_Release|QueryInterface)

Delphi itself has a few implementations of non-standard interface management that have good documentation on their use cases. So take a look at at least these before doing something odd with interface implementations yourself:

  • [WayBack] TAggregatedObject in System
    • This redirects all IInterface implementations to a controller
    • It does not implement IInterface itself, so a descendent must add the interface reference
    • Descendants are TContainedObject and TPropertyPageImpl (the latter used by TActiveXPropertyPage)
  • [WayBack] TContainedObject in System
    • This redirects all IInterface implementations except QueryInterface to a controller
    • Descendants are for instance TSOAPHeaders (via TSOAPHeadersBase) used by TSoapPascalInvoker, TInvokableClass and TRIO, and TConnectionPoint used by TConnectionPoints
  • [WayBack] TInterfacedPersistent in System.Classes
    • This supports the notion of (potentially) being owned by another TPersistent. Classes like TCollectionItem, TFieldOptions and TEditButton implement this ownership behaviour.
    • When owned, then redirect reference counting to the owner (if that owner implements IInterface), but not QueryInterface
    • When not owned, then it is effectively non-reference counted
  • [WayBack] TComponent in System.Classes
    • This supports the notion of (potentially) being owned by another TComponent. Classes like TComponent and TCollectionItem implement this ownership behaviour.
    • When owned, then redirects all IInterface calls to the owner (including QueryInterface).
  • [Archive.is] TXPEditReaderStream in DUnit XP_OTAEditorUtils. It is largely undocumented.
  • TXPInterfacedObject in DUnit XPInterfacedObject. It is largely undocumented too.

Not so good examples:

  • [WayBack] TCustomVariantType in System.Variants (which is basically TSingletonImplementation with a lot of extra methods)
  • [WayBack] TSingletonImplementation in System.Generics.Defaults(which is basically what most TNonRefCountInterfacedObject implementations do; this is sort of OK as it is meant to be used only in [WayBack] TCustomComparer<T> and descendants that are supposed to be singletons).
  • IUnknown in Winapi.Ole2 (this whole unit is undocumented; the class only has virtual abstract methods; the unit – together with Winapi.OleCtl – seems to be only meant to support the depcrecated Vcl.OleAuto unit.)

And of course there is the standard implementation with proper multi-threading support:

There are quite a few classes that implement reference counting while descending from things like TComponent, usually without the proper multi-threading support that TInferfacedObject has:

Read the rest of this entry »

Posted in Conference Topics, Conferences, Delphi, Development, Event, Software Development | Leave a Comment »

Don’t use global state to manage a local problem – The Old New Thing

Posted by jpluimers on 2021/01/20

The 20081211 article [WayBack] Don’t use global state to manage a local problem – The Old New Thing reminds me of a process I went through with a programming team a few years ago.

A lot of their source base came from the procedural era: global variables and global methods. No classes. No methods on records.

Taking them to the level of reference counted immutable instances that used dependency injection as an architectural design was a long journey.

Early in their journey, they would create a lot of methods on classes and records at the class level.

Then they started introducing instances that were basically singletons.

Finally they made real instances that could have more than one available at run-time. These would still create other instances when they needed, often through a few singletons that were still left (for instance session state, database connection state, etc).

Then they introduced caches and pools to keep things alive longer than as to speed up things. It also highly complicated life-time management.

Finally they backed down, and started hooking up things through dependency injection.

A lot of the above caused global state to be used for solving local problems.

It was a tough, but fun time, to get them on the right path, tickling them with the right puzzles at the right time to steer them through their journey..

–jeroen

Posted in Conference Topics, Conferences, Development, Event, Software Development | Leave a Comment »

GitHub – pierrejean-coudert/delphi-libraries-list: List of Delphi Libraries and Frameworks

Posted by jpluimers on 2021/01/14

For my link archive: [WayBackGitHub – pierrejean-coudert/delphi-libraries-list: List of Delphi Libraries and Frameworks

–jeroen

Posted in Conference Topics, Conferences, Delphi, Development, Event, Software Development | Leave a Comment »

The difference between thread-safety and re-entrancy – The Old New Thing

Posted by jpluimers on 2021/01/13

Just in case I need a nice example about the difference: [WayBack] The difference between thread-safety and re-entrancy – The Old New Thing.

It has both that and the definition:

An operation is “thread-safe” if it can be performed from multiple threads safely, even if the calls happen simultaneously on multiple threads.

An operation is re-entrant if it can be performed while the operation is already in progress (perhaps in another context). This is a stronger concept than thread-safety, because the second attempt to perform the operation can even come from within the same thread.

–jeroen

Posted in Conference Topics, Conferences, Development, Event, Software Development | Leave a Comment »

Want to use SmartPointers in Delphi? Use the Spring4D one: it is unit tested and optimised

Posted by jpluimers on 2021/01/06

A while ago, there was a nice discussion on smart pointers at [WayBack] Smart Pointers – Generics vrs non-generic implementastion – RTL and Delphi Object Pascal – Delphi-PRAXiS [en].

Conclusion from that:

  • many people think that reference counted interfaces are the same as Smart Pointers
    (basically Smart Pointers are the next level and of course they are based on reference counting)
  • there are a lot of Smart Pointer implementations, but few have a test suite, nor are optimised , nor easy to use
  • The combo Shared/IShared<T>/TShared<T> from Spring4D has all of the above advantages
  • in order to optmise Smart Pointer implementations, you really have to well know the effects of modern Delphi language constructs on the compiler in various target platforms

The discussion mentioned above includes both feature and speed comparisons.

I was a bit amazed that at CodeRage 2018, Marco Cantu introduced yet another smart pointer implementation: one worse than existing implementations, and one with only basic demonstration code, leaving out a test suite.

There have many posts on my blog about smart pointers (see the list below), but Spring4D smart pointer implementation has been around for such a long time that any well respected Delphi developer by now should use them. The source is at  Shared/IShared (search for {$REGION 'Shared smart pointer'} at the current repository).

This list below on my Smart Pointer related blog posts might not be fully complete, but at least mentions that by now you should be using Spring4D.

Some comments on the CodeRage 2018 demos

Read the rest of this entry »

Posted in Conference Topics, Conferences, Delphi, Development, Event, Software Development, Spring4D | Leave a Comment »