The Wiert Corner – irregular stream of stuff

Jeroen W. Pluimers on .NET, C#, Delphi, databases, and personal interests

  • My badges

  • Twitter Updates

  • My Flickr Stream

  • Pages

  • All categories

  • Enter your email address to subscribe to this blog and receive notifications of new posts by email.

    Join 1,862 other subscribers

Archive for the ‘Delphi’ Category

Work around for running out of memory when building multiple projects in Delphi < 10.2

Posted by jpluimers on 2018/08/16

I found out this is a problem until at least Delphi 10.1 Berlin, but my original question was for XE8:

Is there any tool that clears the Delphi memory overhead when a “build all” switches to the next project in a project group?

XE8 constantly runs out of memory with large project groups as the memory usage keeps increasing for each project it builds in the group.

There are actually two answers:

Uwe Raabe:  Have you tried with Use MSBuild checked for each project?

Jeroen Wiert Pluimers: +Uwe Raabe Much better. Much slower too (:

and:

Torbjörn Olsson: Have you tried DDevExtensions?
There is a setting named “Release compiler unit cache before compiling”

I’ve opted for the [Archive.is] DDevExtensions; I thought this VM had it installed, but apparently I forgot installing it as [Archive.is] IDE FixPack by the same author was installed.

This is how you configure DDevExtensions:

  1. Run Delphi
  2. Choose menu “Tools”, submenu “DDevExtensions Options…”
  3. In the dialog, select “Compilation” in the list on the left
  4. Ensure “Release compiler unit cache of other projects before compiling” is checked
  5. Confirm any changes

Source: [WayBackIs there any tool that clears the Delphi memory overhead when a “build all” s…

In the comparison below:

  • watch the scale of the graphs
  • observe that DDevExtensions uses about the same memory as msbuild, but builds much faster
With "Use MSBuild externally to compile"

With “Use MSBuild externally to compile”

Regular "Build all"

Regular “Build all”

With DDevExtensions

With DDevExtensions

 

Posted in Conference Topics, Conferences, Delphi, Development, Event, Software Development | Leave a Comment »

A success story – kudos to ScaleMM, +André Mussche… – Ondrej Kelle – Google+

Posted by jpluimers on 2018/08/15

Though no maintenance for over 2 years, it ScaleMM seems among the fastest of Delphi memory managers: [WayBack] GitHub – andremussche/scalemm: Fast scaling memory manager for Delphi

Full story via [WayBackA success story – kudos to ScaleMM, +André Mussche… – Ondrej Kelle – Google+ at [WayBack] just tried Scaled MM – cut the time for 100k down from 51 minutes to 40 minutes. That’s with 18 (9+9) cores allocated to the app and 2 to everything else. Wow, allocating 8 + 8 and the time drops even more… – Russell Weetch

Two lessons on multi-threading here:

  • use a memory manager that copes well with threads
  • do not allocate more busy threads than available (hyper-threaded) cores

Some history (as ScaleMM, TopMM and FastMM seem to be related):

  • A bit of history here: at the time we had the mm contest. Not many people had access to 8 cores+, we had…. Andre and Ivo were both colleagues of mine at that point, although working for different companies. We needed (stock trading, Ivo and me) something to scale and Ivo wrote topmm, which still performs better on multi-cores than fastmm and has less assembler code. Andre improved on Ivo’s concepts and yes, it really outperforms fastmm on multi-cores today.
    It also out-performs most C family provided mm’s. Note I was not massively involved, but both Ivo and Andre were. And both did a proper job. So KUDOS to them.
    Note these (fastmm and topmm) were written with multi-core in mind. The practical results at the time were often under-estimated, because few people had access to the real hardware. Most of us running two cores at most. Nice to see that the concept those two programmers pursued still pays dividend in 2018!
  • Read: ScaleMM and TopMM and FastMM… Keyboard left me..
    (I wrote COMMM as a joke..!).
  • [WayBack] TopMemory v3.55. High Performance. Fully Scalable. Free. Memory Manager. for. Delphi – PDF
  • [WayBack] FPC Anagrams

There are also some IntelTBB memory manager references at https://plus.google.com/+RussellWeetch/posts/W4EQgLme5ud [WayBack]

–jeroen

Posted in Delphi, Development, FastMM, Software Development | Leave a Comment »

zlib win32/i386/x86 and win64/x64_86 builds for Windows

Posted by jpluimers on 2018/08/15

In the past, the DLL for zlib was called zlib.dll, but as of a few years back it is called zlib1.dll as it switched to static linking. There are may obscure download sites for zlib1.dll, but the master site is referenced from these documents:

–jeroen

Posted in Delphi, Development, Software Development | Leave a Comment »

Delphi Build Groups: building all the Configuration/Target permutations of projects in your project group from inside the IDE

Posted by jpluimers on 2018/08/14

An automated build system for your Delphi applications is gold, and straightforward to setup using the Delphi support for msbuild.

Sometimes however, it is convenient to build any or all Configuration/Target permutations of all your projects from inside the Delphi IDE.

This has been possible since Delphi XE introduced the [Archive.is] Build Groups – RAD Studio XE. It is a really nifty feature which you can activate by clicking on the “people” icon in the project manager: .

Uwe Raabe has witten an excellent blog post on this a few years ago: [WayBack] Do you know Build Groups? | The Art of Delphi Programming

Since you can add multiple Build Groups, you can select which one contains what Configuration/Target permutations of any number of projects.

This for instance allows you to:

  • build all your server projects at once
  • only build debug versions of all projects

The Build Groups settings are saved as part of your groupproj settings and are a pretty stable feature.

Two extra tips:

–jeroen

Related:

Posted in Conference Topics, Conferences, Delphi, Development, Event, Software Development | Leave a Comment »

Optimized Pascal compilation time | Fredrik Haglund’s blog

Posted by jpluimers on 2018/08/14

Long uses lists especially of units circularly referencing each other are killing for compiler performance both in CPU and memory consumption.

To solve it:

Ingredients

  • ICARUS – a free tool from the maker of Pascal Analyzer. [WayBackhttp://www.peganza.com/products.htm. Icarus parses Delphi or Borland Pascal source code and generates a Uses Report. This report will help you remove unneeded units from your uses lists. You will also know which units that can be moved from the interface uses list to the implementation uses list.
  • Uses Cleanup Tool – by Oleg Zhukov [WayBackhttp://cc.borland.com/Item.aspx?id=23199.

Step-by-step

  1. Run ICARUS on your delphi project to get a report of which units depends on each other.
  2. Input this report in the Uses Cleanup tool to automatically update all uses-clauses in your source code.
  3. Delete all DCU-files and recompile.

Source: [WayBackOptimized Pascal compilation time | Fredrik Haglund’s blog

Edit

Bit-rot galore!: Since Fredrik published his article, some links have vanished, so they are invalid in the above quote as well:

Uwe Raabe was the first to notice on [WayBack] G+ Long uses lists especially of units circularly referencing each other are killing for compiler performance both in CPU and memory consumption… – Jeroen Wiert Pluimers – Google+ that the Uses Cleanup Tool by Oleg Zhukov can now be downloaded from [WayBackFiles as [WayBackUsesCleanupceaf.zip.

Several people suggest using cnPack in the above G+ thread, but [WayBacktheir Delphi parser is far behind [WayBack] Delphi AST.

Cleaning uses lists is a tough thing, it very much depends in getting the dependencies really right, then interpreting them correctly.

–jeroen

Posted in Delphi, Development, Software Development | 7 Comments »

Why Skylake CPUs Are Sometimes 50% Slower – How Intel Has Broken Existing Code – Alois Kraus

Posted by jpluimers on 2018/08/09

[WayBack] Why Skylake CPUs Are Sometimes 50% Slower – How Intel Has Broken Existing Code – Alois Kraus reports that the PAUSE instruction on Intel Skylake architecture takes an order of magnitude longer than on previous architectures.

This impacts spinlock code in .NET 4.x and .NET Core 2, and likely impacts other spinlock code as well. The .NET core fix gets back-ported to .NET 4.x.

Since Delphi XE, the Delphi RTL code has borrowed ideas from .NET implementing this kind of code, so I filed [WayBack] QualityCentral: 144063 PAUSE instruction on Intel Skylake takes order of magnitude longer: important for SpinWait/SpinLock code (because Google can index it).

It is no coincidence that a Senior Scaleability Engineer at Booking.com mentioned it on his G+ stream ([WayBack] The “Pause” instruction changed timing dramatically in Skylake. Spinlock implementation based on pause will need adjustments. – Kristian Köhntopp – Google+) as changes like this can heavily impact server systems.

–jeroen

via: [WayBack] The “Pause” instruction changed timing dramatically in Skylake. Spinlock implementation based on pause will need adjustments.I hope a fix for this will be back-ported for many Delphi versions. – Jeroen Wiert Pluimers – Google+

Posted in .NET, .NET 4.0, Delphi, Development, Software Development | Leave a Comment »

Reminder to self: write up which refactoring options might work under what circumstances

Posted by jpluimers on 2018/08/09

Reminder to self to elaborate on:

–jeroen

Posted in Delphi, Development, Software Development | 2 Comments »

Delphi 2007 – when you lost your Borland.*.Targets file during a Windows upgrade

Posted by jpluimers on 2018/08/09

When upgrading Windows, it will create a fully new %windir% for you.

Any files an application installed files in them will be gone, and appear in a %HomeDrive%\Windows.old

If you’ve deleted that directory because you thought all your applications worked, or the Windows 10 upgrade automagically deleted the Windows.old directory by itself after some grace period, then you might be in trouble.

One of those applications is Delphi 2007. I wrote about a different issue on x64 Windows on that product before: Delphi on Windows x64: solution when project compiles fine under IDE, but not using MSBuild.

Basically it puts these files in %windir%\Microsoft.NET\Framework\v2.0.50727:

  • Borland.Common.Targets
  • Borland.Cpp.Targets
  • Borland.Delphi.Targets
  • Borland.Group.Targets

Those will be gone if you deleted the Windows.old directory.

If you are lucky, the files are also in this directory to copy back to %windir%\Microsoft.NET\Framework\v2.0.50727 and %windir%\Microsoft.NET\Framework64\v2.0.50727:

C:\ProgramData\{B59CE2E6-B15A-4F23-BD0E-72BF2ADDC3C7}\core\7EFD2DA3\6C948720

And if you’re not, you can find them here too: https://gist.github.com/anonymous/ab801cd925e5e634518fd5592eb2a46e

–jeroen

via:

Read the rest of this entry »

Posted in Delphi, Delphi 2007, Development, Software Development | Leave a Comment »

ngHttp2 and OpenSSL win32/i386/x86 and win64/x64_86 (a.k.a. x86_64) builds for Windows

Posted by jpluimers on 2018/08/08

[WayBack] ngHttp2 DLLs has a simple a version scheme. The build inside the Windows PHP distribution includes these version numbers. The TreadSafe versions work as plug in replacement for github.com/grijjy/DelphiScalableClientSockets/tree/master/Bin which is used by github.com/grijjy/GrijjyFoundation/blob/master/Grijjy.Http.pas#L11. You obtain them via [WayBack] PHP For Windows: Binaries and sources Releases; and at the time of writing these were the most recent versions:

[WayBack] OpenSSL DLLs has a much more complex version scheme, as they are numeric but OpenSSL releases are not.

  • DLLs have four numbers a.b.c.d
  • OpenSSL versions have three numbers and a letter a.b.c.x
  • The letter matches the fourth digit, though the ones marked with * have not been used yet:
    # letter # letter # letter # letter # letter remark
    1 a 6 f 11 k 16 p 21 u
    2 b 7 g 12 l 17 q 22 v *
    3 c 8 h 13 m 18 r 23 w *
    4 d 9 i 14 n 19 s 24 x *
    5 e 10 j 15 o 20 t 25 y *
    26 z *

[WayBack] Index of /SSL has “Pre-compiled Win32/64 libraries without external dependencies to the Microsoft Visual Studio Runtime DLLs, except for the system provided msvcrt.dll.”

These work no matter what development/deployment stacks you use (including a Visual Studio based stack).

The most recent version as of writing is 1.0.2o, which maps to 1.0.2.20 which contains libeay32.dll and ssleay32.dll for both the i386-win32 and x86_64-win64 build (not sure why they both use 32 in the name):

openssl-1.0.2o-i386-win32.zip
openssl-1.0.2o-x64_86-win64.zip which supports x86_64 as this site is about the only one using x64_86 in the name

Background reading:

These binaries are for instance used by (most of them are behind or far behind on the OpenSSL version):

  • Avira Antivirus
  • subversion
  • git mingw64
  • VMware Tools
  • Microsoft OneDrive
  • Delphi Indy communications library

Speaking of which: this is a recent Delphi wrapper around libeay32.dll: [WayBack] GitHub – lminuti/Delphi-OpenSSL: Delphi implementation of OpenSSL

–jeroen

Posted in Delphi, Development, OpenSSL, Power User, Security, Software Development | 2 Comments »

Some tips for your Delphi project options hierarchy

Posted by jpluimers on 2018/08/08

Over the years, various Delphi versions have cluttered the Project Options hierarchy quite a bit. This means that when you walk this hierarchy in the Project Options dialog, you will see many bold entries, indicating they have been changed for that level in the hierarchy.

The most important tip is to keep as much as possible at the top level, preferably using relative paths or paths containing $(...) macros.

I usually fill these entries there:

  • “All configurations – All platforms”
    • “Conditional defines” having the base conditional defines
    • “Output directory” (usually including a relative path that includes .\$(Platform)\$(Config)
    • “Search path” with .\$(Platform) when you nave libraries containing DCU files per platform
      • You can even do $(Platform)\$(Config) here if these DCU files are different for configurations (DEBUG/RELEASE) as well.
    • “Unit output directory” so each application gets a unique path (so I include .\$(SanitizedProjectName)\$(Platform)\$(Config) or .\$(Platform)\$(Config)\$(SanitizedProjectName) in the path)
    • Unit scope names (based onSource: Delphi unit prefixes for VCL applications)
  •  “Debug configuration – All platforms”
    • “Conditional defines”: add DEBUG
  • “Release configuration – All platforms”
    • “Conditional defines”: add RELEASE

I include the $(SanitizedProjectName) because it has the name of your project, which I documented at Source: Delphi XE8 does not adhere the $(PROJECTNAME) in a “Unit Output Directory”, but does recognise $(SanitizedProjectName) including many other macros.

Note:

A long time ago, +Uwe Raabe explained that you can use $(SanitizedProjectName) in your project settings to include the actual project name in https://plus.google.com/+DavidNottageDelphiExpert/posts/PcW8CwMQGmH

This works great for DCU output, but not so well for EXE output: when using for instance C:\Binaries\$(SanitizedProjectName) as target, the debugger cannot find the executable as the compiler puts it in the wrong place.

Example with a fresh console project:

0. Ensure C:\Binaries exists and you have write access to it
1. Set Output directory to C:\Binaries\$(SanitizedProjectName)
2. Set Unit output directory to .\$(SanitizedProjectName)\$(Platform)\$(Config)
3. Run:

---------------------------
Error
---------------------------
Could not find program, 'C:\Binaries\%SanitizedProjectName%\Project1.exe'.
---------------------------
OK
---------------------------

The executable is actually at C:\Binaries\Project1.exe, so the compiler does not take the “\$(SanitizedProjectName)” bit into account.

Anyone with a fix for that?

Via [WayBack] A long time ago, +Uwe Raabe explained that you can use $(SanitizedProjectName) in your project settings to include the actual project name in https://pl… – Jeroen Wiert Pluimers – Google+

Uwe Raabe

To make the compiler use the correct path the SanitizedProjectName item must be located before its use in the dproj file. DprojNormalizer assures that since version 2.2.1 and ProjectMagician does so, too. Thus my Project1.exe is written correctly to C:\Binaries\Project1\Project1.exe here on my system.
Unfortunately that won’t fix the main problem, which is that the call to the debugger doesn’t resolve this variable in the first place. Currently I don’t know how this could be fixed.
You may try
$(MSBuildProjectName)

It looks like the .dproj is this using itself too, for instance as this part:

    <Import Project="$(MSBuildProjectName).deployproj" Condition="Exists('$(MSBuildProjectName).deployproj')"/>

–jeroen

Posted in Conference Topics, Conferences, Delphi, Development, Event, Software Development | Leave a Comment »