The Wiert Corner – irregular stream of stuff

Jeroen W. Pluimers on .NET, C#, Delphi, databases, and personal interests

  • My badges

  • Twitter Updates

  • My Flickr Stream

  • Pages

  • All categories

  • Enter your email address to subscribe to this blog and receive notifications of new posts by email.

    Join 1,861 other subscribers

Archive for the ‘Delphi’ Category

console – When is System.IsConsole true in Delphi? – Stack Overflow

Posted by jpluimers on 2021/07/22

Some highlights from [WayBack] console – When is System.IsConsole true in Delphi? – Stack Overflow:

  • Project/Options/Linking/Generate console application and {$APPTYPE CONSOLE} are two separate things. – [WayBack] Andreas Rejbrand.
  • The -cc commandline parameter for dcc32.exe (which could be in your ProjectX.cfg file!) is equivalent to “Generate console application):

    Found it: the executable has been created using dcc32.exe and the dpr / cfg file, the cfg contains a line

    -cc

    which creates a console application. – [WayBack] mjn

  • Note that in Delphi XE2, OSX applications can not use this variable as it is always true. See QC Entry 98956 and Why Does My OSX FireMonkey App Think It Is a Console App ? – [WayBack] mjn

A few notes I have found out myself

The linker option Project/Options/Linking/Generate console application” is the same as “DCC_ConsoleTarget” in the .dproj file:

    <PropertyGroup Condition="'$(Base_Win32)'!=''">
        <DCC_ConsoleTarget>true</DCC_ConsoleTarget>
    </PropertyGroup>

Even setting IsConsole to True does not allocate a new console, so the STD_OUTPUT_HANDLE handle does not work!

The below method in the System unit, then does not show output:

procedure WriteErrorMessage;
{$IFDEF MSWINDOWS}
var
  Dummy: Cardinal;
begin
  if IsConsole then
  begin
    with TTextRec(Output) do
    begin
      if (Mode = fmOutput) and (BufPos > 0) then
        TTextIOFunc(InOutFunc)(TTextRec(Output));  // flush out text buffer
    end;
    // Leave #0 off end of runErrMsg
    WriteFile(GetStdHandle(STD_OUTPUT_HANDLE), @runErrMsg, Sizeof(runErrMsg) - 1, Dummy, nil);
    WriteFile(GetStdHandle(STD_OUTPUT_HANDLE), @sLineBreak[1], 2, Dummy, nil);
  end
  else if not NoErrMsg then
    MessageBoxA(0, runErrMsg, errCaption, 0);
end;

For console output, you have to enable the linker option DCC_ConsoleTarget allocates a console (if not yet allocated), enables the “STD_OUTPUT_HANDLE” handle, and sets IsConsole to True.

–jeroen

Posted in Conference Topics, Conferences, Delphi, Development, Event, Software Development | Leave a Comment »

Some useful FastMM related methods to track memory usage

Posted by jpluimers on 2021/07/21

Below, for my link archive, some searches and relevant posts on FastMM related method calls to track or report memory usage.

Searches:

  • LogMemoryManagerStateToFile
  • FastGetHeapStatus {Returns summarised information about the state of the memory manager. (For
    backward compatibility.)}
  • GetMemoryManagerState (InternalBlockSize, UseableBlockSize, AllocatedBlockCount, ReservedAddressSpace) {Returns statistics about the current state of the memory manager}GetMemoryManagerUsageSummary {Returns a summary of the information returned by GetMemoryManagerState}
  • GetMemoryMap {Non-POSIX only; Gets the state of every 64K block in the 4GB address space}
  • ScanMemoryPoolForCorruptions; {Scans the memory pool for any corruptions. If a corruption is encountered an “Out of Memory” exception is raised.}
    • It is very costly in CPU usage, but helps finding heap corruption quickly.
  • function GetCurrentAllocationGroup: Cardinal;
    • {Returns the current “allocation group”. Whenever a GetMem request is serviced
      in FullDebugMode, the current “allocation group” is stored in the block header.
      This may help with debugging. Note that if a block is subsequently reallocated
      that it keeps its original “allocation group” and “allocation number” (all
      allocations are also numbered sequentially).}
  • procedure PushAllocationGroup(ANewCurrentAllocationGroup: Cardinal);
    procedure PopAllocationGroup;

    • {Allocation groups work in a stack like fashion. Group numbers are pushed onto
      and popped off the stack. Note that the stack size is limited, so every push
      should have a matching pop.}
  • LogAllocatedBlocksToFile
    • {Logs detail about currently allocated memory blocks for the specified range of
      allocation groups. if ALastAllocationGroupToLog is less than
      AFirstAllocationGroupToLog or it is zero, then all allocation groups are
      logged. This routine also checks the memory pool for consistency at the same
      time, raising an “Out of Memory” error if the check fails.}
  • SetMMLogFileName
    • {Specify the full path and name for the filename to be used for logging memory
      errors, etc. If ALogFileName is nil or points to an empty string it will
      revert to the default log file name.}

Posts (note that not all of them get their calculations right):

These help you track leaks that do not appear as leaks during shutdown: memory allocations that will be released at the end of your application, but are mostly unused while your application is still alive.

A few things to take away from these:

  1. “Out of Memory” (or exception EOutOfMemor) could mean that the memory manager structures are hosed, but memory is still available.
  2. You can specify the FastMM log file used (for instance to include a PID or run-time ID in them so each run gets a separate filename)
  3. When carefully setting up allocation groups, you are able to zoom in at allocations

A gist with a MemoryManagerUnit showing a few of these calls is below.

An example of its usage is this:

procedure TMyTestClass.TestMethod();
begin
   TLogMemoryStatesHelper.DumpMemoryStatesBeforeAndAfter('TestMethod',
     TLogging.LogDir,
     TFileLogger.GetLogFileName,
     procedure (const AFormat: string; const Args: array of const)
     begin
       TLogging.LogEvent(ltInfoHigh, aFormat, Args);
     end,
     procedure()
     begin
       try
         // Given
         BuildSomeTestScenario();
         // When
         InitializeTestScenario();
         // Then
         CheckEquals(0, TestScenarioSummary());
       finally
         // Cleanup
         CleanUpTestScenario();
       end;
     end
  );
end;

–jeroen

Read the rest of this entry »

Posted in Delphi, Development, FastMM, Software Development | Leave a Comment »

Some links for detecting memory leaks from individual DUnit test methods

Posted by jpluimers on 2021/07/20

–jeroen

Posted in Conferences, Delphi, Event, Software Development, Development, FastMM, DUnit, Conference Topics | Leave a Comment »

inheritance – Delphi: How to call inherited inherited ancestor? – Stack Overflow

Posted by jpluimers on 2021/07/15

Officially, the answer to [WayBack] inheritance – Delphi: How to call inherited inherited ancestor? – Stack Overflow is that you can’t on the language level as I explained in my answer

You can’t in a regular language way, as this would break the object oriented aspects of the language.

You can fiddle around with pointers and clever casts to do this, but before even starting to answer that: is this really what you want?

As others mentioned: your need sounds like a serious “design smell” (which is similar to code smell, but more severe.

Edit:

Going down the pointer fiddling road might save you work in the short term, and cost you weeks of work in the long term.
This makes for some good reading on that: Upstream decisions, downstream costs.

If you really want, then there is a clever hack around this by [WayBack] User kludg – Stack Overflow.

His hack is centered around understanding what what the [WayBack] System.TMethod Record essentially is:

Read the rest of this entry »

Posted in Conference Topics, Conferences, Delphi, Development, Event, Software Development | Leave a Comment »

Some OmniThreadLibrary notes and links

Posted by jpluimers on 2021/07/14

For my link archive:

IOmniTimedTask and the Timed task name

You create a reference like this:

  TimedTask := Parallel.TimedTask.Execute(
    procedure(const task: IOmniTask)
    begin
      // ...
    end);

In the background, Parallel.TimedTask calls TOmniTimedTask.Create() which calls CreateTask(TOmniTimedTaskWorker.Create(), 'Timed task').Unobserved.Run

The problem is that TOmniTimedTaskWorker is private to the OtlParallel unit, which means you cannot take that call out without also copying that class.

There might be a workaround which I need to research based on the Apply method of IOmniTaskConfig, maybe through Parallel.ApplyConfig. These links might help:

–jeroen

Read the rest of this entry »

Posted in Delphi, Development, Multi-Threading / Concurrency, Software Development | Leave a Comment »

Chromium Embedded Framework – some links

Posted by jpluimers on 2021/07/13

For my link archive:

–jeroen

Read the rest of this entry »

Posted in .NET, Delphi, Development, Software Development | Leave a Comment »

Delphi Tokyo binary BPLs not finding “@System@Net@Httpclient@Win@TCertificateStore@$bcctr$qqrv” means you likely have Delphi 10.2.2 installed

Posted by jpluimers on 2021/07/13

If you get errors loading BPLs that indicate this method cannot be found: “@System@Net@Httpclient@Win@TCertificateStore@$bcctr$qqrv” then it is likely you run Delphi 10.2.2.

Upgrade to Delphi 10.2.3 to resolve this problem.

This happens for instance with binary versions of these below libraries.

They cannot help, as Embarcadero removed the TCertificateStore class (present in Delphi 10.1.2) in 10.2.0, then re-added it in 10.2.3.

 

Libraries that I bumped into that were affected:

Note that some reports talk about “@System@Net@Httpclient@Win@TCertificateStore@$bcdtr$qqrv” in stead of “@System@Net@Httpclient@Win@TCertificateStore@$bcctr$qqrv“. I am not sure what causes this discrepancy yet.

–jeroen

Posted in Delphi, Delphi 10.2 Tokyo (Godzilla), Development, Software Development | Leave a Comment »

delphi – How to enable wirecompression on Firebird 3.0 – Stack Overflow

Posted by jpluimers on 2021/07/08

For my link archive:

–jeroen

Posted in Database Development, Delphi, Development, Firebird, Software Development | Leave a Comment »

E2213 Bad packaged unit format: ..\xxxxx.dcp.yyyy – Expected version: 32.0, Windows Unicode(x86) Found version: 105.101, Unk(CIL)

Posted by jpluimers on 2021/07/08

On my research list I have seen this happen with various libraries, but only libraries that deliver a single set of .bpl/.dcp files that you require for both debug and release builds.

The error occurs both ways:

  • building a “debug” build is fine, but after it a “release” build fails
  • building a “release” build is fine, but after it a “debug” build fails

This is with all separate DCU output directories for any permutation in any project: .\$(Platform)\$(Config)\$(SanitizedProjectName)

Example (where xxxx.pas is the first unit depending on Qrctrls, but I have seen it with other libraries that provide .bpl/.dcp combinations as well):

[dcc32 Fatal Error] xxxx.pas(14): E2213 Bad packaged unit format: ..\3rdParty\Quickrep506\19.0\Lib\Win32\Release\QR506RunDXE10_2.dcp.Qrctrls - Expected version: 32.0, Windows Unicode(x86) Found version: 105.101, Unk(CIL)

My best guess is that something is being cached in between builds, but not marked for the correct build configuration well enough.

–jeroen

Posted in Delphi, Delphi 10.2 Tokyo (Godzilla), Development, Software Development | Leave a Comment »

SetProcessWorkingSetSize: you hardly – if ever – need to call this from your process

Posted by jpluimers on 2021/07/07

There are quite a few posts that recommend using SetProcessWorkingSetSize to trim your process working set, usually in the SetProcessWorkingSetSize(ProcessHandle, -1, -1) form:

[WayBack] SetProcessWorkingSetSize function (winbase.h) | Microsoft Docs

Sets the minimum and maximum working set sizes for the specified process.

BOOL SetProcessWorkingSetSize(
  HANDLE hProcess,
  SIZE_T dwMinimumWorkingSetSize,
  SIZE_T dwMaximumWorkingSetSize );

The working set of the specified process can be emptied by specifying the value (SIZE_T)–1 for both the minimum and maximum working set sizes. This removes as many pages as possible from the working set. The [WayBack] EmptyWorkingSet function can also be used for this purpose.

In practice you hardly ever have to do this, mainly because this will write – regardless of (dis)usage – all of your memory to the pagefile, even the memory your frequently use.

Windows has way better heuristics to do that automatically for you, skipping pages you frequently use.

It basically makes sense in a few use cases, for instance when you know that most (like 90% or more) of that memory is never going to be used again.

Another use case (with specific memory sizes) is when you know that your program is going to use a defined range of memory, which is outside what Windows will heuristically expect from it.

A few more links that go into more details on this:

  • [WayBack] windows – Pros and Cons of using SetProcessWorkingSetSize – Stack Overflow answers by:
    • Hans Passant:

      SetProcessWorkingSetSize() controls the amount of RAM that your process uses, it doesn’t otherwise have any affect on the virtual memory size of your process. Windows is already quite good at dynamically controlling this, swapping memory pages out on demand when another process needs RAM.

      By doing this manually, you slow down your program a lot, causing a lot of page faults when Windows is forced to swap the memory pages back in.

      SetProcessWorkingSetSize is typically used to increase the amount of RAM allocated for a process. Or to force a trim when the app knows that it is going to be idle for a long time. Also done automatically by old Windows versions when you minimize the main window of the app.

    • Zack Yezek:

      The only good use case I’ve seen for this call is when you KNOW your process is going to hog a lot of the system’s RAM and you want to reserve it for the duration. You use it to tell the OS “Yes, I’m going to eat a lot of the system RAM during my entire run and don’t get in my way”.

    • Maxim Masiutin:

      We have found out that, for a GUI application written in Delphi for Win32/Win64 or written in a similar way that uses large and heavy libraries on top of the Win32 API (GDI, etc), it is worth calling SetProcessWorkingSetSize once.

      We call it with -1, -1 parameters, within a fraction of second after the application has fully opened and showed the main window to the user. In this case, the SetProcessWorkingSetSize(... -1, -1) releases lots of startup code that seem to not needed any more.

  • [WayBack] c# – How to set MinWorkingSet and MaxWorkingSet in a 64-bit .NET process? – Stack Overflow answer by Hans Passant:

    Don’t pinvoke this, just use the Process.CurrentProcess.MinWorkingSet property directly.

    Very high odds that this won’t make any difference. Soft paging faults are entirely normal and resolved very quickly if the machine has enough RAM. Takes ~0.7 microseconds on my laptop. You can’t avoid them, it is the behavior of a demand_paged virtual memory operating system like Windows. Very cheap, as long as there is a free page readily available.

    But if it “blips” you program performance then you need to consider the likelihood that it isn’t readily available and triggered a hard page fault in another process. The paging fault does get expensive if the RAM page must be stolen from another process, its content has to be stored in the paging file and has to be reset back to zero first. That can add up quickly, hundreds of microseconds isn’t unusual.

    The basic law of “there is no free lunch”, you need to run less processes or buy more RAM. With the latter option the sane choice, 8 gigabytes sets you back about 75 bucks today. Complete steal.

  • [WayBack] c++ – SetProcessWorkingSetSize usage – Stack Overflow answer by MSalters:

    I had an application which by default would close down entirely but keep listening for certain events. However, most of my code at that point would not be needed for a long time. To reduce the impact my process made, I called SetProcessWorkingSetSize(-1,-1);. This meant Windows could take back the physical RAM and give it to other apps. I’d get my RAM back when events did arrive.

    That’s of course unrelated to your situation, and I don’t think you’d benefit.

  • [WayBack] delphi – When to call SetProcessWorkingSetSize? (Convincing the memory manager to release the memory) – Stack Overflow

    If your goal is for your application to use less memory you should look elsewhere. Look for leaks, look for heap fragmentations look for optimisations and if you think FastMM is keeping you from doing so you should try to find facts to support it. If your goal is to keep your workinset size small you could try to keep your memory access local. Maybe FastMM or another memory manager could help you with it, but it is a very different problem compared to using to much memory.

    you can check the FasttMM memory usage via FasttMM calls GetMemoryManagerState and GetMemoryManagerUsageSummary before and after calling API SetProcessWorkingSetSize.

    I don’t need to use SetProcessWorkingSetSize. FastMM will eventually release the RAM.


    To confirm that this behavior is generated by FastMM (as suggested by Barry Kelly) I crated a second program that allocated A LOT of RAM. As soon as Windows ran out of RAM, my program memory utilization returned to its original value.

  • [WayBack] delphi – SetProcessWorkingSetSize – What’s the catch? – Stack Overflow answer by Rob Kennedy:

    Yes, it’s a bad thing. You’re telling the OS that you know more about memory management than it does, which probably isn’t true. You’re telling to to page all your inactive memory to disk. It obeys. The moment you touch any of that memory again, the OS has to page it back into RAM. You’re forcing disk I/O that you don’t actually know you need.

    If the OS needs more free RAM, it can figure out which memory hasn’t been used lately and page it out. That might be from your program, or it might be from some other program. But if the OS doesn’t need more free RAM, then you’ve just forced a bunch of disk I/O that nobody asked for.

    If you have memory that you know you don’t need anymore, free it. Don’t just page it to disk. If you have memory that the OS thinks you don’t need, it will page it for you automatically as the need arises.

    Also, it’s usually unwise to call Application.ProcessMessages unless you know there are messages that your main thread needs to process that it wouldn’t otherwise process by itself. The application automatically processes messages when there’s nothing else to do, so if you have nothing to do, just let the application run itself.

–jeroen

Posted in .NET, C, C++, Delphi, Development, Software Development, Windows Development | Leave a Comment »