The Wiert Corner – irregular stream of stuff

Jeroen W. Pluimers on .NET, C#, Delphi, databases, and personal interests

  • My badges

  • Twitter Updates

  • My Flickr Stream

  • Pages

  • All categories

  • Enter your email address to subscribe to this blog and receive notifications of new posts by email.

    Join 1,861 other subscribers

Archive for the ‘Conferences’ Category

Delphi: workaround doing math with generic types preventing “E2015 Operator not applicable to this operand type” with TValue (as there is no way to constraint the generic type to be floating point or ordinal)

Posted by jpluimers on 2021/12/14

A while ago on Facebook (it’s a private group, so you cannot see the posts unless you both have a Facebook account and are member of the group), [Archive.is] Niels Tjørnhøj-Thomsen (coming from a C++ templates background) asked why the below method would throw a E2015 Operator not applicable to this operand type in the complex expression:

function TAxis<t>.Calc(const AScalar: T): single;
begin
  Result := fStart + ( ( ( AScalar - fMin ) / fRange ) * fExtent );
end;

The type itself was very simple:

TAxis<T> = record
  fMin, fMax, fRange: T;
  fStart, fEnd, fExtent: single;
  function Calc( const AScalar: T ): single;
end;

He used these small example specialisations that put me on the wrong foot, as the order was TDateTime followed by single:

var
  rXAxis: TAxis<TDateTime>;
  rYAxis: TAxis<single>;

So at first I thought this might be caused by TDateTime to be defined in the System unit as a typed type:

type
  TDateTime = type Double;

It wasn’t.

Splitting the code in 4 lines with assignments of single expression operations would make the error appear in all expressions.

Casting parts of the expression to simple would not help either.

A small test program [Archive.is] might put you, like me, on the wrong foot because the specialisation is in the same source file as the generic type:

program DelphiMathAndGenerics;

type
  TAxis<T> = record
    fMin, fMax, fRange: T;
    fStart, fEnd, fExtent: single;
    function CalcCasted( const AScalar: T ): single;
    function CalcPlain( const AScalar: T ): single;
  end;

function TAxis<T>.CalcCasted(const AScalar: T): single;
var
  Offset: single;
  NormalisedOffset: single;
  ScaledOffset: single;
begin
  // First 2 lines give the same error: E2089 Invalid typecast
  Offset := single(AScalar) - fMin;
  NormalisedOffset := Offset / single(fRange);
  ScaledOffset := NormalisedOffset * fExtent;
  Result := fStart + ScaledOffset;
end;

function TAxis<T>.CalcPlain(const AScalar: T): single;
var
  Offset: T;
  NormalisedOffset: T;
  ScaledOffset: T;
begin
  // All 4 lines give the same error: E2015 Operator not applicable to this operand type
  Offset := AScalar - fMin;
  NormalisedOffset := Offset / fRange;
  ScaledOffset := NormalisedOffset * fExtent;
  Result := fStart + ScaledOffset;
end;

var
  rXAxis: TAxis<TDateTime>;
  rYAxis: TAxis<single>;

begin
end.

Splitting this in two files [Archive.is], a AxisUnit unit having only the TAxis<T> type, and a main program (even without having the specialisations) shows that even the unit itself would not compile.

This shows a major difference between Delphi (and similar C#) generics and C++ templates:

  • generics are compiled and fully verified at the generic stage
  • templates are pre-processed, then finally verified at specialisation stage

A solution would be that Delphi could constraint the generic type T into something like float or ordinal so the compiler would know that more operators are allowed in the code. But alas, Delphi – like C# – has a very limited number of constraints (C# only would allow a constraint for enumerations in version 7.3): Delphi Constraints in Generics – RAD Studio XE documentation wiki.

This StackOverflow question is very similar, and has the same answer (generics in Delphi work differently than templates in C++): [Source] templates – Arithmetic operations with generic types in Delphi – Stack Overflow

I’m new in Delphi. For a project required by my company, I need to translate some code from our existing C++ classes to Delphi. Some of these classes are templates, such …

Workaround: use the TValue.From<T>() function

There is a workaround though, but it is slow, as you need to convert from the generic T type to the actual (in this case floating point) type you can apply the operators on.

This is possible with the (Delphi 2010 introduced) TValue.From<T>() method which returns a TValue record. That TValue record has instance methods like AsExtended to extract or convert the embedded value as a specific type.

Initially, [Wayback] Delphi 2010 Rtti.TValue documentation had the From method signature wrong, maybe because of many wiki and blog HTML editors kill angle bracket pairs < and > in code blocks:

function From(const Value: T): TValue; static;

Since the [Wayback] Delphi XE System.Rtti.TValue documentation, the From method signature is fixed (see the bold parts):

class function From<T>(const Value: T): TValue; static;

With the [Wayback] Delphi XE2 Rtti.TValue documentation, the unit got renamed from Rtti into System.Rtti and has not changed further.

When using TValue.From<T>(), the AxisUnit becomes this:

unit AxisUnit;

interface

type
  TAxis<T> = record
    fMin, fMax, fRange: T;
    fStart, fEnd, fExtent: single;
    function Calc( const AScalar: T ): single;
  strict private
    function AsSingle(const Value: T): single;
  end;

implementation

uses
  System.Rtti;

function TAxis<T>.AsSingle(const Value: T): single;
begin
  Result := TValue.From<T>(Value).AsExtended
end;

function TAxis<T>.Calc(const AScalar: T): single;
var
  Offset: single;
  NormalisedOffset: single;
  ScaledOffset: single;
begin
  Offset := AsSingle(AScalar) - AsSingle(fMin);
  NormalisedOffset := Offset / AsSingle(fRange);
  ScaledOffset := NormalisedOffset * fExtent;
  Result := fStart + ScaledOffset;
end;

end.

–jeroen

Read the rest of this entry »

Posted in .NET, C#, Conference Topics, Conferences, Delphi, Development, Event, Software Development | Leave a Comment »

The IDEA project – an ongoing series of nonverbal algorithm assembly instructions

Posted by jpluimers on 2021/12/10

I wonder how many new algorithms were added, as the first 6 were really impressive: [WayBackIDEA on Twitter: “Excited to announce the IDEA project – an ongoing series of nonverbal algorithm assembly instructions: https://t.co/zOAyfOAv3l… https://t.co/epQfBBdzdF”

While originally scheduling this, these were added:

Read the rest of this entry »

Posted in Algorithms, Conference Topics, Conferences, Development, Event, IKEA hacks, LifeHacker, Power User, Software Development | Leave a Comment »

On TStrings (and TStringList) sorting: what the default Sort behaviour is and how to change sorting order

Posted by jpluimers on 2021/12/07

Because I need this eventually, here the full quote of my answer in [Wayback] sorting – How can I get TStringList to sort differently in Delphi – Stack Overflow (The default Sort behaviour is to accommodate i18n sorting in natural order):

Read the rest of this entry »

Posted in Conference Topics, Conferences, Delphi, Development, Event, Software Development, Undocumented Delphi | Leave a Comment »

Katie Anderson on Twitter: “Saw this on Facebook and it’s my new favorite PaaS (Pizza as a Service) breakdown”

Posted by jpluimers on 2021/11/30

Legacy/IaaS/PaaS/SaaS explained by “Pizza as a service”: from home made, take and bake, pizza delivery to full dining out.

[Archive.is] Katie Anderson on Twitter: “Saw this on Facebook and it’s my new favorite PaaS (Pizza as a Service) breakdown https://t.co/INKKG9UOAK” / Twitter

–jeroen

Read the rest of this entry »

Posted in Cloud, Conference Topics, Conferences, Development, Event, Infrastructure, Software Development, Systems Architecture | Leave a Comment »

AgileTD Keynote | Happiness is Quality

Posted by jpluimers on 2021/09/16

Hopefully this becomes available on-line, as this will be a great talk, but Potsdam is out of my reach in my current physical condition.

[Wayback] AgileTD Keynote | Happiness is Quality

In Gwen Diagram’s keynote go on a journey of looking at how to improve happiness in teams for the goal of building quality systems.


How can we build teams that strive to build quality systems? By building in happiness early and often.

Each organisation has a culture which directly affects the quality of their work. Organisations with a lower level of quality; that is, systems that fail more often, longer times to fix and longer times to build often have a few things in common; frustration, apathy and cynicism.

On the other side of the coin, teams with slack, time to learn and time to reflect also have the space to improve their systems. Quality is everyone’s responsibility is an oft quoted phrase. But, how do you actually engage people to build quality systems?

Improving quality should not weigh heavy on the shoulders of the test specialist. Their main role should not be attempting to convince people that unit tests should exist, that systems should be testable and observable or that automated tests will speed up development. Instead, we should be building teams that want to find out together how to build a system that breathes quality.

The way to do this is by improving the happiness of the engineers on the team with learning, autonomy and coaching. So, how can we build teams that strive to build quality systems? By building in happiness early and often. In this talk, we will go on a journey of looking at how to improve happiness in teams for the end goal of building quality systems.

Via: [Wayback] Agile Testing Days on Twitter: “☀️ Here comes the happiness! ☀️ @gwendiagram will take us on a journey of looking at how to improve happiness in teams for the end goal of building quality systems. ➡️ Learn more about the keynote and the speaker: … “

–jeroen

Posted in Agile, Conferences, Development, Event, Power User, Software Development, Testing | Leave a Comment »

DUnit with FastMM: Detecting memory leaks in individual tests.

Posted by jpluimers on 2021/08/05

Steps:

  1. Add the conditional define FASTMM to your project
  2. Ensure you have $(BDS)\source\DUnit\src  in your search path in your project (as otherwise Delphi will pick the pre-built TestFramework.dcu file which was compiled without the FASTMM conditional define)
  3. Inside a test method, or the SetUp method of a class, set FailsOnMemoryLeak to True
  4. If the SetUp method of the class allocates memory, ensure the TearDown de-allocates it. Otherwise you will have leaks:
    • DUnit will check memory differences from the start of the SetUp until the end of the TearDown
    • DUnit will not take into account what is freed in the destructor or by automatic finalization after the destructor!
  5. Re-build your application (otherwise the DUnit TestFramework unit will not take into account the FASTMM conditional define)

Depending in your test framework, FailsOnMemoryLeak might be by default be False or True:

  • TestInsight by default has FailsIfMemoryLeaked set to True for the root test suite (which is then applied to FailsOnMemoryLeak of any test method).
    procedure RunRegisteredTests(const baseUrl: string);
    var
      suite: ITestSuite;
      result: TTestResult;
      listener: ITestListener;
    begin
      suite := RegisteredTests;
      if not Assigned(suite) then Exit;
      result := TTestResult.Create;
      result.FailsIfNoChecksExecuted := True;
      result.FailsIfMemoryLeaked := True;
      listener := TTestInsightListener.Create(baseUrl, suite.CountEnabledTestCases);
      result.AddListener(listener);
      try
        suite.Run(result);
      finally
        result.Free;
      end;
    end;
  • Console DUnit runners (Text, or XML) by default have FailsIfMemoryLeaked set to False.
  • GUI DUnit runner has FailsIfMemoryLeaked depending on the options:

DUnit source differences

Note that recent Delphi versions (I think XE and up) ship with almost the same sources as https://sourceforge.net/code-snapshots/svn/d/du/dunit/svn/dunit-svn-r44-trunk.zip, with these Embarcadero changes:

  • all SVN timestamps are based on time zone -0400 instead of +0000:
    • $Date: 2008-04-24 07:59:47 -0400 (Thu, 24 Apr 2008) $
    • $Date: 2008-04-24 11:59:47 +0000 (Thu, 24 Apr 2008) $
  • Embarcadero removed:
    • all comment lines having TODO in them
    • all files of types .dof, .cfg, and.wmz
    • the files NGUITestRunner.nfm and NGUITestRunner.pas
    • the file /etc/usermap
    • the directory trees /private and /projects
  • Embarcadero changed
    • file /src/versioninfo.res from 9.3.0 to 9.2.1 (which is odd, as all files are from 9.3.0)
    • unit TextTestRunner to support:
      • CLR (used last in Delphi 2007)
      • FailureCount
      • ErrorCount
      • IndentLevel
      • PrefixChars
      • conditional defines ADDITIONAL_INFO, BASIC_INFO
      • output of UnitName
    • unit TestExtensions to support:
      • CLR (used last in Delphi 2007)
      • conditional defines ANDROID_FIXME and LINUX
      • compiler directive LEGACYIFEND
    • GUITestRunner.dfm to have ResultsView: TListView property Height value 45 instead of 39
    • the below methods to use ReturnAddress in stead of CallerAddr:
      • TGUITestCase.FindControl
      • TGUITestCase.Click
      • TGUITestCase.EnterKeyInto
      • TGUITestCase.EnterTextInto
      • TGUITestCase.Show
      • TGUITestCase.CheckTabTo overloads
      • TGUITestCase.CheckFocused overloads
      • TGUITestCase.CheckEnabled overloads
      • TGUITestCase.SetFocus overloads
      • TGUITestCase.CheckVisible overloads
    • method TGUITestRunner.RunTest
      • from
        class procedure TGUITestRunner.RunTest(test: ITest);
        begin
          with TGUITestRunner.Create(nil) do
          begin
           try
            suite := test;
            ShowModal;
           finally
            Free;
           end;
          end;
        end;
      • to
        class procedure TGUITestRunner.RunTest(test: ITest);
        var
          GUI :TGUITestRunner;
        begin
          Application.CreateForm(TGUITestRunner, GUI);
          with GUI do
          begin
           try
            suite := test;
            ShowModal;
           finally
            Free;
           end;
          end;
        end;
    • unit GUITestRunner:
      • from
        procedure RunTest(test: ITest);
        begin
          with TGUITestRunner.Create(nil) do
          begin
            try
              Suite := test;
              ShowModal;
            finally
              Free;
            end;
          end;
        end;
      • to
        procedure RunTest(test: ITest);
        var
          GUI :TGUITestRunner;
        begin
          Application.CreateForm(TGUITestRunner, GUI);
          with GUI do
          begin
            try
              Suite := test;
              if Suite <> nil then
                ShowModal;
            finally
              Free;
            end;
          end;
        end;
      • method TGUITestRunner.SetUp not to set SubItems[0] := ''; when there is no test suite.
      • method TGUITestRunner.FormCreate to use FormatSettings.TimeSeparator instead of TimeSeparator.
  • unit TestFramework to support:
    • HPPEMIT
    • LEGACYIFEND
    • CLR and _USE_SYSDEBUG_
    • AUTOREFCOUNT
    • NEXTGEN
    • ANDROID
    • the RunCountAttribute and RunCount support
    • a CheckEquals overload for uint64
    • a CheckNotEquals overload for TCharArray
    • CheckCircularRef
    • CompareFloatRelative
    • NotSameErrorMessage with WideString arguments instead of string
    • a TestDataDir function
    • ReturnAddress for older compilers
  • a new unit DUnitTestRunner
  • a new Makefile file
  • a new UnitTests.log file

–jeroen

Posted in Conference Topics, Conferences, Delphi, Development, DUnit, Event, FastMM, Software Development, TestInsight | 1 Comment »

“Unexpected Memory Leak” after “MyTestApplication.exe: Memory Leak Detected”

Posted by jpluimers on 2021/07/28

Sometimes after a regular FastMM memory leak test report you get a dialog like this:

---------------------------
Unexpected Memory Leak
---------------------------
An unexpected memory leak has occurred. The unexpected small block leaks are:

61 - 68 bytes: Unknown x 1
---------------------------
OK   
---------------------------

The normal FastMM memory leak reporting dialog looks like this:

---------------------------
MyTestApplication.exe: Memory Leak Detected
---------------------------
...
Note: Memory leak detail is logged to a text file in the same folder as this application. To disable this memory leak check, undefine "EnableMemoryLeakReporting".
---------------------------
OK   
---------------------------

Searching for “Unexpected Memory Leak” in RTL or FastMM code did not reveal results, but that might be me not doing a proper search.

One big problem is that the regular memory leak dialog is being suppressed by setting SuppressMessageBoxes to True (see for instance my blog post Application shutdown: wait for all threads to terminate or not? or [WayBack] delphi – Generating a FASTMM report WITHOUT the shutdown dialog – Stack Overflow).

However the “Unexpected Memory Leak” message box is always shown.

Research

Location of the error message caption

  • GETMEM.INC has an identifier LeakMessageTitle:
    LeakMessageTitle: _PAnsiChr = 'Unexpected Memory Leak';
  • Unlike FastMM4.pas, GETMEM.INC does not take into account SuppressMessageBoxes.
  • Like FastMM4.pas, GETMEM.INC does take into account ReportMemoryLeaksOnShutdown.

More on GETMEM.INC: [WayBack] delphi – Reporting memory leaks on shutdown with a console application – Stack Overflow

Versioned FastMM4.pas.

Complicating the hunt

Not all allocations go via the memory manager. A complicating factor is that:

  • ALL TMonitor.Create allocations go through SysAllocMem:
class function TMonitor.Create: PMonitor;
begin
  if CacheLineSize = 0 then
    AtomicExchange(CacheLineSize, GetCacheLineSize);
  if (CPUCount > 1) and (FDefaultSpinCount = 0) then
    AtomicExchange(FDefaultSpinCount, 1000);
  if CacheLineSize > SizeOf(Result^) then
    Result := SysAllocMem(CacheLineSize)
  else
    Result := SysAllocMem(SizeOf(Result^));
  Result.FSpinCount := FDefaultSpinCount;
end;
  • ALL TMonitor.Destroy deallocations go through SysFreeMem:
procedure TMonitor.Destroy;
begin
  if (MonitorSupport <> nil) and (FLockEvent <> nil) then
    MonitorSupport.FreeSyncObject(FLockEvent);
  SysFreeMem(@Self);
end;

Debugging this is easiest to set a breakpoint in the FastMM4.pas finalization section enabling a breakpoint group that has breakpoints on these methods inside GETMEM.INC:

  • function SysGetMem(Size: NativeInt): Pointer;
  • function SysFreeMem(P: Pointer): Integer;
  • function SysReallocMem(P: Pointer; Size: NativeInt): Pointer;
  • function SysAllocMem(Size: NativeInt): Pointer;

For inspecting for instance an asm construct like TSmallPoolBlockPoolHeader[edx], use a conversion PSmallBlockPoolHeader(Pointer(EDX))^,r

Poor mans shotgun approach

This will hide all the SysAllocMem related leaks:

unit FastMM4WrapperUnit;

interface

{$ifdef UseFastMM4}
uses
  FastMM4,
  FastMM4Messages;
{$endif UseFastMM4}

implementation

initialization
//
finalization
// Disable GETMEM.INC reports that neglect SuppressMessageBoxes and NoErrMsg; this will effectively hide the leak as GETMEM.INC also does not write the leaks to a log file
{$ifdef UseFastMM4}
  if SuppressMessageBoxes then
  begin
    {$WARN SYMBOL_PLATFORM OFF} NoErrMsg := True {$WARN SYMBOL_PLATFORM ON}; // no ShowMessage from the RTL (except for memory leaks)
    ReportMemoryLeaksOnShutdown := False; // No ShowMessage from the RTL in the GETMEM.INC teardown
  end;
{$endif UseFastMM4}
end.

IsConsole: needs linker option

Note that I tried embedding this in the then portion, but enabling IsConsole fails:

    IsConsole := True; // can only force run-time error messages to be written to the console if the "Project/Options/Linking/Generate console application" is set.

The reason is that this does not allocate a console handle, so you really need to ensure the linker has allocated a console for you by enabling Project/Options/Linking/Generate console application (or option DCC_ConsoleTarget in the .dproj file)

For more details, see my post console – When is System.IsConsole true in Delphi? – Stack Overflow.

–jeroen

Posted in Conference Topics, Conferences, Delphi, Development, Event, FastMM, Software Development | Leave a Comment »

console – When is System.IsConsole true in Delphi? – Stack Overflow

Posted by jpluimers on 2021/07/22

Some highlights from [WayBack] console – When is System.IsConsole true in Delphi? – Stack Overflow:

  • Project/Options/Linking/Generate console application and {$APPTYPE CONSOLE} are two separate things. – [WayBack] Andreas Rejbrand.
  • The -cc commandline parameter for dcc32.exe (which could be in your ProjectX.cfg file!) is equivalent to “Generate console application):

    Found it: the executable has been created using dcc32.exe and the dpr / cfg file, the cfg contains a line

    -cc

    which creates a console application. – [WayBack] mjn

  • Note that in Delphi XE2, OSX applications can not use this variable as it is always true. See QC Entry 98956 and Why Does My OSX FireMonkey App Think It Is a Console App ? – [WayBack] mjn

A few notes I have found out myself

The linker option Project/Options/Linking/Generate console application” is the same as “DCC_ConsoleTarget” in the .dproj file:

    <PropertyGroup Condition="'$(Base_Win32)'!=''">
        <DCC_ConsoleTarget>true</DCC_ConsoleTarget>
    </PropertyGroup>

Even setting IsConsole to True does not allocate a new console, so the STD_OUTPUT_HANDLE handle does not work!

The below method in the System unit, then does not show output:

procedure WriteErrorMessage;
{$IFDEF MSWINDOWS}
var
  Dummy: Cardinal;
begin
  if IsConsole then
  begin
    with TTextRec(Output) do
    begin
      if (Mode = fmOutput) and (BufPos > 0) then
        TTextIOFunc(InOutFunc)(TTextRec(Output));  // flush out text buffer
    end;
    // Leave #0 off end of runErrMsg
    WriteFile(GetStdHandle(STD_OUTPUT_HANDLE), @runErrMsg, Sizeof(runErrMsg) - 1, Dummy, nil);
    WriteFile(GetStdHandle(STD_OUTPUT_HANDLE), @sLineBreak[1], 2, Dummy, nil);
  end
  else if not NoErrMsg then
    MessageBoxA(0, runErrMsg, errCaption, 0);
end;

For console output, you have to enable the linker option DCC_ConsoleTarget allocates a console (if not yet allocated), enables the “STD_OUTPUT_HANDLE” handle, and sets IsConsole to True.

–jeroen

Posted in Conference Topics, Conferences, Delphi, Development, Event, Software Development | Leave a Comment »

Filezilla: figuring out the cause of “Connection timed out after 20 seconds of inactivity”

Posted by jpluimers on 2021/07/21

On one of my Raspberry Pi boxes, somehow I could not access files over SFTP (SSH File Transfer Protocol) via FileZilla.

I would consistently get this error:

"Connection timed out after 20 seconds of inactivity"

Figuring the exact cause took a while.

TL;DR: SFTP uses an interactive non-login shell, then interprets the output from that shell. For that kind of shell, ensure few or none scripts run that output text.

These links finally got me to the cause

Read the rest of this entry »

Posted in *nix, *nix-tools, bash, bash, Communications Development, Conference Topics, Conferences, Development, Event, Internet protocol suite, Power User, Scripting, SFTP, Software Development, SSH, TCP | Leave a Comment »

Some links for detecting memory leaks from individual DUnit test methods

Posted by jpluimers on 2021/07/20

–jeroen

Posted in Conferences, Delphi, Event, Software Development, Development, FastMM, DUnit, Conference Topics | Leave a Comment »