The Wiert Corner – irregular stream of stuff

Jeroen W. Pluimers on .NET, C#, Delphi, databases, and personal interests

  • My badges

  • Twitter Updates

  • My Flickr Stream

  • Pages

  • All categories

  • Enter your email address to subscribe to this blog and receive notifications of new posts by email.

    Join 4,262 other subscribers

Archive for February 20th, 2020

Delphi memory allocators and configuration notes in multi-threaded environments with high allocation/deallocation rates

Posted by jpluimers on 2020/02/20

From an interesting discussion at [Archive.is/WayBack] FaceBook: Delphi developer thread by Jarto Tarpio with some measurements by Jarto Tarpio and André Mussche.

  • Manipulation of strings and lists in Delphi have high memory allocation/deallocation rates, so HTTP related services with high call rates are affected more than regular services
  •  FastMM:
    • conditional defines that can help are NeverSleepOnThreadContention, UseSwitchToThread, and UseReleaseStack then measure.
    • has one huge advantage: It’s very, very good at keeping memory fragmentation at bay
    • default settings are for applications that use lots of CPU, but have no really high memory allocation/deallocation rates
    • has very good debugging facilities
    • Under FullDebugMode address space is never released back to the operating system so once the address space has been exhausted there is very little room to manoeuvre.
  • TCMalloc:
    • is very good at multi-threaded memory management with high allocation/deallocation rates
    • needs to be persuaded to releases memory to the OS:
      it only releases to the system under two occasions: Freeing another part of the memory, or asking it to release all parts marked as freed.

    • has no debugging facilities

The differences make it a challenge to integrate in your development and deployment process: because of the debugging facilities, you’d like FastMM in all your environments, but TCMalloc in multi-threaded environments with high allocation/deallocation rates.

One possibility is to have your CI environment deliver both in all stages, run all tests on both, then choose the final one depending on your run-time configuration.

That gives a burden on configuring your Continuous Integration, but the gain might outweigh this cost.

Relevant links from the Facebook thread:

–jeroen

Posted in Delphi, Development, FastMM, Software Development | 5 Comments »

More great free Python books

Posted by jpluimers on 2020/02/20

A while ago I wrote about Until 20171201 you can get free access to “Automate the Boring Stuff with Python”?.

That book is still free. And these great books are too:

Via: [WayBack] Cracking Codes with Python by @AlSweigart teaches complete beginners how to program in Python. The book features the source code to several ciphers and … – ThisIsWhyICode – Google+

–jeroen

Posted in Development, Python, Scripting, Software Development | Leave a Comment »

Writing a Time Series Database from Scratch | Fabian Reinartz

Posted by jpluimers on 2020/02/20

Long read but very much worth it: [WayBack] Writing a Time Series Database from Scratch | Fabian Reinartz.

On using the file systems strengths for a flexible time series database that is both fast to write, and read millions of data points per second.

via:

Similarly interesting: [WayBack] Translations: This article was translated to Simplified Chinese by Xiong Duo and to Korean by Matt Lee (이 성욱). Introduction I want to make solid-state

–jeroen

Posted in Development, Power User, Software Development, Systems Architecture | Leave a Comment »