The Wiert Corner – irregular stream of stuff

Jeroen W. Pluimers on .NET, C#, Delphi, databases, and personal interests

  • My badges

  • Twitter Updates

  • My Flickr Stream

  • Pages

  • All categories

  • Enter your email address to subscribe to this blog and receive notifications of new posts by email.

    Join 1,860 other subscribers

Archive for the ‘C++’ Category

Valetudo | Cloud-free control webinterface for Xiaomi vacuum robots

Posted by jpluimers on 2022/01/04

[Wayback] Valetudo | Cloud-free control webinterface for vacuum robots

Valetudo is a standalone binary which runs on rooted Vacuums of the Xiaomi ecosystem and aims to enable the user to operate the robot vacuum without any Cloud Connection whatsoever.

You can even use it to make a WiFi heat map of the floors in your home: the Valeroni app can do that for you.

The Valeroni app is open source too: [Wayback/Archive.is] ccoors/Valeronoi: A WiFi mapping companion app for Valetudo

Via [Archive.is] Kristian Köhntopp on Twitter: “Paging @sys_adm_ama: … Alternative Firmware für Robot-Staubsauger … Alternative Firmware für Robot-Staubsauger verwenden, um das Wifi im Haus zu mappen (und Staub zu saugen)”

–jeroen

Read the rest of this entry »

Posted in C++, Development, Hardware, LifeHacker, Power User, Software Development, WiFi | Leave a Comment »

The Charlie Calvert “Here’s to Good Friends and New Adventures” article

Posted by jpluimers on 2021/10/14

Since this contains a list of contains from back then (20+ years ago!), I save it for future reference: [WayBack] Here’s to Good Friends and New Adventures

I would like to make a very short list of the other people at Borland who I have had the privilage of working with very closely. These people are Jason Sprenger, Xavier Pacheco, Steve Teixeira, John Kaster, Lar Mader and Rich Jones. Each of these people I worked with every day over a period of years, and they never showed me anything but the very best and most admirable human traits. I hope I am so lucky as to work with such fine people again in my life.

As for all the others, there is no way even to begin to thank them. A few of these people are Karen Giles, Lino Tadros, Steve Trefethen, Christine Ellis, Paolo Ciccone, Yolanda Davis, Blake Stone, Bruneau Babet, Dave Marancik, Anders Ohlsson, Dave Powell, Claudio Briceno, Joe Manzone, Terri Bartos, Dave Wilhelm, Andrea Ginsberg, Jason Vokes, Ludo Neveu, Martin Pamdeth, Martin Raim, Ernesto Franchini, Edwin Desouza, Zack Urlocker, Rosemary Abell, Robert Warren, Scott Bussinger, Richard Morris, Paul Beach, Jeremy McGee, Nimish Vora, Michael Swindell, Lorie Hull, Kendyll Upstrom, Kari Gallant, Allen Bauer, Josh Dahlby, Jose Rubens, John Thomas, John Williams, J.D. Hildebrand, Hizo Jozsef, Goran Kallmark, Ben Riga, George Cross, Gary Benner, Fred Felman, Erik Jakowitz, Danny Thorpe, Craig Farrell, Claudia Currie, Bill Weber, Lance Devon, Robert West, Amber Hein, Richard Kubat, Jeff Peters, Ellie Peters, Krystyna Niedzwiedzka, Kathy Berkland, Kelly Welty, Tom Lam, Nester Miranda (and Carlos!), Dana Kaufman, Pawal Ksiezyk, Jim Wright, Lori and Ellen from travel, Sergey Orlik and many others who I just don’t happen to recall right now, or who I liked very much but only met a few times.

I’m also indebted to Ray Kanopka, Mark Miller, Dick Malley, Dan Horn, Taco Oosterkamp, Bob Swart, Ann Lynnworth, Marco Cantu, Jeroen Pluimers, and many more who worked in the Borland community and brought me great joy. It’s amazing to consider how many talented and remarkable people have been drawn to this company.

jeroenhttps://www.facebook.com/groups/137012246341854/permalink/2895795467130171/

Posted in Borland C++, C++, C++ Builder, Delphi, Development, Pascal, Software Development, Turbo Pascal | Leave a Comment »

SetProcessWorkingSetSize: you hardly – if ever – need to call this from your process

Posted by jpluimers on 2021/07/07

There are quite a few posts that recommend using SetProcessWorkingSetSize to trim your process working set, usually in the SetProcessWorkingSetSize(ProcessHandle, -1, -1) form:

[WayBack] SetProcessWorkingSetSize function (winbase.h) | Microsoft Docs

Sets the minimum and maximum working set sizes for the specified process.

BOOL SetProcessWorkingSetSize(
  HANDLE hProcess,
  SIZE_T dwMinimumWorkingSetSize,
  SIZE_T dwMaximumWorkingSetSize );

The working set of the specified process can be emptied by specifying the value (SIZE_T)–1 for both the minimum and maximum working set sizes. This removes as many pages as possible from the working set. The [WayBack] EmptyWorkingSet function can also be used for this purpose.

In practice you hardly ever have to do this, mainly because this will write – regardless of (dis)usage – all of your memory to the pagefile, even the memory your frequently use.

Windows has way better heuristics to do that automatically for you, skipping pages you frequently use.

It basically makes sense in a few use cases, for instance when you know that most (like 90% or more) of that memory is never going to be used again.

Another use case (with specific memory sizes) is when you know that your program is going to use a defined range of memory, which is outside what Windows will heuristically expect from it.

A few more links that go into more details on this:

  • [WayBack] windows – Pros and Cons of using SetProcessWorkingSetSize – Stack Overflow answers by:
    • Hans Passant:

      SetProcessWorkingSetSize() controls the amount of RAM that your process uses, it doesn’t otherwise have any affect on the virtual memory size of your process. Windows is already quite good at dynamically controlling this, swapping memory pages out on demand when another process needs RAM.

      By doing this manually, you slow down your program a lot, causing a lot of page faults when Windows is forced to swap the memory pages back in.

      SetProcessWorkingSetSize is typically used to increase the amount of RAM allocated for a process. Or to force a trim when the app knows that it is going to be idle for a long time. Also done automatically by old Windows versions when you minimize the main window of the app.

    • Zack Yezek:

      The only good use case I’ve seen for this call is when you KNOW your process is going to hog a lot of the system’s RAM and you want to reserve it for the duration. You use it to tell the OS “Yes, I’m going to eat a lot of the system RAM during my entire run and don’t get in my way”.

    • Maxim Masiutin:

      We have found out that, for a GUI application written in Delphi for Win32/Win64 or written in a similar way that uses large and heavy libraries on top of the Win32 API (GDI, etc), it is worth calling SetProcessWorkingSetSize once.

      We call it with -1, -1 parameters, within a fraction of second after the application has fully opened and showed the main window to the user. In this case, the SetProcessWorkingSetSize(... -1, -1) releases lots of startup code that seem to not needed any more.

  • [WayBack] c# – How to set MinWorkingSet and MaxWorkingSet in a 64-bit .NET process? – Stack Overflow answer by Hans Passant:

    Don’t pinvoke this, just use the Process.CurrentProcess.MinWorkingSet property directly.

    Very high odds that this won’t make any difference. Soft paging faults are entirely normal and resolved very quickly if the machine has enough RAM. Takes ~0.7 microseconds on my laptop. You can’t avoid them, it is the behavior of a demand_paged virtual memory operating system like Windows. Very cheap, as long as there is a free page readily available.

    But if it “blips” you program performance then you need to consider the likelihood that it isn’t readily available and triggered a hard page fault in another process. The paging fault does get expensive if the RAM page must be stolen from another process, its content has to be stored in the paging file and has to be reset back to zero first. That can add up quickly, hundreds of microseconds isn’t unusual.

    The basic law of “there is no free lunch”, you need to run less processes or buy more RAM. With the latter option the sane choice, 8 gigabytes sets you back about 75 bucks today. Complete steal.

  • [WayBack] c++ – SetProcessWorkingSetSize usage – Stack Overflow answer by MSalters:

    I had an application which by default would close down entirely but keep listening for certain events. However, most of my code at that point would not be needed for a long time. To reduce the impact my process made, I called SetProcessWorkingSetSize(-1,-1);. This meant Windows could take back the physical RAM and give it to other apps. I’d get my RAM back when events did arrive.

    That’s of course unrelated to your situation, and I don’t think you’d benefit.

  • [WayBack] delphi – When to call SetProcessWorkingSetSize? (Convincing the memory manager to release the memory) – Stack Overflow

    If your goal is for your application to use less memory you should look elsewhere. Look for leaks, look for heap fragmentations look for optimisations and if you think FastMM is keeping you from doing so you should try to find facts to support it. If your goal is to keep your workinset size small you could try to keep your memory access local. Maybe FastMM or another memory manager could help you with it, but it is a very different problem compared to using to much memory.

    you can check the FasttMM memory usage via FasttMM calls GetMemoryManagerState and GetMemoryManagerUsageSummary before and after calling API SetProcessWorkingSetSize.

    I don’t need to use SetProcessWorkingSetSize. FastMM will eventually release the RAM.


    To confirm that this behavior is generated by FastMM (as suggested by Barry Kelly) I crated a second program that allocated A LOT of RAM. As soon as Windows ran out of RAM, my program memory utilization returned to its original value.

  • [WayBack] delphi – SetProcessWorkingSetSize – What’s the catch? – Stack Overflow answer by Rob Kennedy:

    Yes, it’s a bad thing. You’re telling the OS that you know more about memory management than it does, which probably isn’t true. You’re telling to to page all your inactive memory to disk. It obeys. The moment you touch any of that memory again, the OS has to page it back into RAM. You’re forcing disk I/O that you don’t actually know you need.

    If the OS needs more free RAM, it can figure out which memory hasn’t been used lately and page it out. That might be from your program, or it might be from some other program. But if the OS doesn’t need more free RAM, then you’ve just forced a bunch of disk I/O that nobody asked for.

    If you have memory that you know you don’t need anymore, free it. Don’t just page it to disk. If you have memory that the OS thinks you don’t need, it will page it for you automatically as the need arises.

    Also, it’s usually unwise to call Application.ProcessMessages unless you know there are messages that your main thread needs to process that it wouldn’t otherwise process by itself. The application automatically processes messages when there’s nothing else to do, so if you have nothing to do, just let the application run itself.

–jeroen

Posted in .NET, C, C++, Delphi, Development, Software Development, Windows Development | Leave a Comment »

DCOM calls from thread pool threads: CoInitialize/CoUnitialize location and expensiveness?

Posted by jpluimers on 2021/06/24

Interesting takeaway from [WayBack] DCOM calls from thread pool threads

call CoInitialize* at the start, and call CoUninitialize before returning. Expensive, but necessary

Related:

–jeroen

Posted in .NET, C, C++, COM/DCOM/COM+, Delphi, Development, Software Development, Windows Development | Leave a Comment »

“No mapping for the Unicode character exists in the target multi-byte code page”

Posted by jpluimers on 2021/06/24

Usually when I see this error [Wayback] “No mapping for the Unicode character exists in the target multi-byte code page” – Google Search, it is in legacy code that uses string buffers where decoding or decompressing data into.

This is almost always wrong no matter what kind of data you use, as it will depend in your string encoding.

I have seen it happen especially in these cases:

  • base64 decoding from string to string (solution: decode from a string stream into a binary stream, then post-process from there)
  • zip or zlib decompress from binary stream to string stream, then reading the string stream (solution: decompress from binary stream to binary stream, then post-process from there)

Most cases I encountered were in Delphi and C code, but surprisingly I also bumped into C# exhibiting this behaviour.

I’m not alone, just see these examples from the above Google search:

–jeroen

Posted in .NET, base64, C, C#, C++, Delphi, Development, Encoding, Software Development, Unicode | Leave a Comment »

Chocolatey: vcredist 2005/2008/2010 updates failing as Microsoft rebuilt the installers multiple times causing new hashes each time

Posted by jpluimers on 2021/05/14

The Microsoft rebuilt move affects these Chocolatey Microsoft Visual C++ Redistributable packages, usually multiple times:

You get error messages like this:

ERROR: Checksum for 'C:\Users\saroot\AppData\Local\Temp\chocolatey\vcredist2010\10.0.40219.2\vcredist_x86.exe' did not meet '66B797B3B4F99488F53C2B676610DFE9868984C779536891A8D8F73EE214BC4B' for checksum type 'sha256'. Consider passing the actual checksums through with --checksum --checksum64 once you validate the checksums are appropriate. A less secure option is to pass --ignore-checksums if necessary

The cause is [Archive.is] vcredist-all fails to install due to broken checksum of vcredist2005, vcredist2008 and vcredist2010 · Issue #105 · jberezanski/ChocolateyPackages

jberezanskicommented

Great, Microsoft is apparently on an in-place installer update spree.

Actually, this is not a problem with vcredist-all, but rather with those specific packages and should be reported to maintainers of those packages. It so happens, however, that I’m also one of the maintainers of vcredist2008 and vcredist2010 (which live in https://github.com/chocolatey-community/chocolatey-coreteampackages/tree/master/manual). I’ve already prepared a fix for 2010 and I guess I’ll do 2008 tomorrow.

As for 2005, I can see that it has the same maintainer as vcredist2010 had until very recently – when we took over that package because the maintainer did not respond. So we probably should take over 2005, too.

Progress

The vcredist2010 package has been modified two times for this:

Hopefully the vcredist2008 and vcredist2005 packages will follow soon.

–jeroen

Posted in C++, Chocolatey, Development, Power User, Software Development, Visual Studio C++, Windows | Leave a Comment »

Delphi compile time assertions

Posted by jpluimers on 2021/02/24

My post on Delphi intrinsic functions that evaluate to consts as a step up to Delphi compile time assertions.

This is a corner case of Delphi language use, which can come in very handy when your code is changed in the future, and you want to be prepared to ensure that some changes do not violate some predefined boundaries.

Hopefully a future post will elaborate a bit more on actual usage, but for now, lets first show some examples, then some other languages that have a richer set of compile time assertions.

My original goal was to see if I could come up with a mechanism that allowed for better validation of generic types because Delphi generic constraints – still – are quite limited: Delphi Constraints in Generics – RAD Studio XE documentation wiki, so limiting or verifying the aspects of the concrete type often cannot be done by constraints.

C# had a similar limitation for constraining to enum, which finally got added some 13 years after adding generics, in 2018: [WayBack] Unmanaged, delegate and enum type constraints – C# 7.3 in Rider and ReSharper – .NET Tools Blog.NET Tools Blog.

Let’s start simple:

const
  // forbidden const values to check compile time assert:
  A = 0;
  B = 1;
  C = -1;
  // The below expressions all each generate a "[dcc32 Error] E2098 Division by zero" (so multiple errors in one compile)
  // Asserting at compile time using boolean expressions:
  BooleanAssertAIsNotZero =  1 div Ord(A <> 0);
  BooleanAssertBIsNotOne = 1 div Ord(B <> 1);
  // Asserting at compile time using numeric expressions:
  AssertAIsNotZero = 1 div A;
  AssertBIsNotOne =  1 div (B - 1);
  AssertBIsNotAbsOne =  1 div (Abs(B) - 1);
  AssertCIsNotAbsOne =  1 div (Abs(C) - 1);

This is all centered around generating a compile time error "[dcc32 Error] E2098 Division by zero", of which multiple can occur in one compile go (after compilation, the cursor focus will be at the first error) and which has been in the language for a very long time [WayBack] E2098: Division by zero.

The conversion of Boolean to Integer is done using Ord, a very powerful compile time intrinsic that evaluates to a constant.

You can use this for other intrinsics as well, for example:

type
  TDigits = 0..9;

const
  DigitsAreInteger = GetTypeKind(TDigits) = tkInteger;
  DigitsAreIntegerIsTrue = 1 div Ord(DigitsAreInteger);
  DigitsAreEnumeration = GetTypeKind(TDigits) = tkEnumeration; // compiles fine
  DigitsAreEnumerationIsTrue = 1 div Ord(DigitsAreEnumeration); // [dcc32 Error] E2098 Division by zero

The above learns that integer subranges are not enumerations, but stay integers.

You can now extend this to check longer boolean expressions, for instance to check if a record size matches certain criteria. For this we create records having zero to four bytes in size (yes, you can have empty record in Delphi, it in fact the only data structure that can be zero bytes in length, though the documentation [WayBack] Structured Types: record types does not state this is in fact possible ), then validate the sizes:

type
  TRecord0 = record
  end;

  TRecord1 = packed record
    FByte0: Byte;
  end;

  TRecord2 = packed record
    FByte0: Byte;
    FByte1: Byte;
  end;

  TRecord3 = packed record
    FByte0: Byte;
    FByte1: Byte;
    FByte2: Byte;
  end;

  TRecord4 = packed record
    FByte0: Byte;
    FByte1: Byte;
    FByte2: Byte;
    FByte3: Byte;
  end;

const
  AssertTRecord0SizeOf0 = 1 div Ord(SizeOf(TRecord0) = 0); // When expression is false: [dcc32 Error] E2098 Division by zero
  AssertTRecord0SizeOf1 = 1 div Ord(SizeOf(TRecord1) = 1); // When expression is false: [dcc32 Error] E2098 Division by zero
  AssertTRecord0SizeOf2 = 1 div Ord(SizeOf(TRecord2) = 2); // When expression is false: [dcc32 Error] E2098 Division by zero
  AssertTRecord0SizeOf3 = 1 div Ord(SizeOf(TRecord3) = 3); // When expression is false: [dcc32 Error] E2098 Division by zero
  AssertTRecord0SizeOf4 = 1 div Ord(SizeOf(TRecord4) = 4); // When expression is false: [dcc32 Error] E2098 Division by zero
  AssertTRecord0SizeOfMultipleOf4 = 1 div Ord(SizeOf(TRecord0) mod 4 = 0); // When expression is false: [dcc32 Error] E2098 Division by zero
  AssertTRecord4SizeOfMultipleOf4 = 1 div Ord(SizeOf(TRecord4) mod 4 = 0); // When expression is false: [dcc32 Error] E2098 Division by zero
  AssertTRecord0SizeOfMultipleOf4AndGreaterThan0 = 1 div Ord((SizeOf(TRecord0) mod 4 = 0) and (SizeOf(TRecord0) > 0)); // When expression is false: [dcc32 Error] E2098 Division by zero
  AssertTRecord4SizeOfMultipleOf4AndGreaterThan0 = 1 div Ord((SizeOf(TRecord4) mod 4 = 0) and (SizeOf(TRecord4) > 0)); // When expression is false: [dcc32 Error] E2098 Division by zero

That’s how far I got in my first experiments using this mechanism. Hopefully it gave you some inspiration too, so I welcome any usages you made with it.

Inline use of intrinsics can lead to no generated code at all

Since Delphi has no macro language, you cannot create your own intrinsic functions that evaluate to const. You could use a pre-processor though, as described in [WayBack] How to write Delphi compile-time functions – Stack Overflow.

The answer by Johan there however mentions clever use of in-line functions that do not generate any code at all (so effectively evaluate to a const). More on that in a future post.

Compile time assertions in other languages

Many languages support a form of [WayBack] Compile-time calculation – Rosetta Code. If such a language can errors out on compiling such a calculation, then you can have compile time assertions.

Compile time assertions are very much used in C and C++, where they are often called static assertions. Often they depend on macros, but C11 (C standard revision 11) has it built-in.

Since I also do quite a bit of .NET: [WayBack] Can C# Provide a static_assert? – Stack Overflow

Some links on how they work in C and C++, and what you can do with them:

–jeroen

Posted in .NET, C, C#, C++, Conference Topics, Conferences, Delphi, Development, Event, Software Development | Leave a Comment »

A garbage collector for C and C++ (and a wrapper for Delphi): The Boehm-Demers-Weiser conservative C/C++ Garbage Collector

Posted by jpluimers on 2020/12/30

I bumped into [WayBackA garbage collector for C and C++ a while ago, for which the source is at [WayBack] GitHub – ivmai/bdwgc: The Boehm-Demers-Weiser conservative C/C++ Garbage Collector (libgc, bdwgc, boehm-gc).

There is a (very old!) wrapper for Delphi too: [WayBack] 21646 API for Boehm Garbage Collector DLL

Barry Kelly <barry_j_kelly@hotmail.com>,
19 April 2004
——————————————————
This archive contains a simple API unit for the Boehm Garbage Collector DLL, along with another unit which makes it easier to use with classes, and a demonstration application. Also included is the Boehm GC DLL binary, along with source code in the gc_dll_src directory.

The files:

BoehmGc.pas
———–
This unit exports a dozen or so routines from the Boehm GC dll. Since the GC integrates with and replaces the Delphi default memory manager, you probably don’t need to use this unit unless you want to fine-tune the behaviour of the DLL. The DLL exports more routines than are in this unit; the C prototypes are in the gc_dll_src/gc.h header file, and can be imported as needed. If you allocate large chunks of memory (>100K) which don’t contain references to other chunks (and thus don’t need to be scanned for pointers), there are routines in this unit which you can use to increase performance.

General advice: don’t tweak until you need to tweak.

Gc.pas
——
This is the main unit. Put this unit first in the uses clause of you project and the project will automatically use garbage collection. If you want to use objects which require finalization and you don’t want to have to call TObject.Free / TObject.Destroy on them manually, you can use the MarkForFinalization(TObject) function. The basic pattern is to register the object for finalization in its constructor and unregister it with UnmarkForFinalization in its destructor. This handles the two most common use cases for finalization: GC-invoked finalization and manual finalization. Note that it’s always safe to behave as if GC doesn’t exist, and use GetMem/FreeMem, New/Dispose, Create/Free etc. The use of these units simply allows you to also program with garbage collection.

GcTest.dpr & GcTest.exe
———————–
This program contains simple sample code demonstrating the garbage collector in action.

BoehmGC.dll
———–
This contains the implementation of the garbage collector itself. The DLL can be recompiled from the source in gc_dll_src with various options, including multithreaded support, different pointer alignment granularities, etc.

****
The original Boehm GC source comes from: http://www.hpl.hp.com/personal/Hans_Boehm/gc/

I’m Barry Kelly: barry_j_kelly@hotmail.com

You can do anything you like with my source code (*.pas, *.dpr).

See the file gc_dll_src/LICENSEa for permissions for the GC itself.

</barry_j_kelly@hotmail.com>

Although when trying to download, I got this for both cc.embarcadero.com/Download.aspx?id=21646 and cc.embarcadero.com/Download.aspx?id=21646&prot=ftp:

Access to the path ‘\\etnaedndb02.embarcadero.com\f\webcache\cc\2004\4\19\21646.zip’ is denied.

An error has occurred while processing the page.

Please try to refresh the page, or return to the home page.

: ETNACDC04

and [WayBackJeroen Pluimers auf Twitter: “It looks like the @EmbarcaderoTech code central file cc.embarcadero.com/Item/21646 is broken: “Access to the path ‘\https://t.co/3f3blXN9mp\f\webcache\cc\2004\4\19\https://t.co/0UJUtWvxVV’ is denied.” when exploring or downloading.…”

 Explore the files in this upload

File Exploration is Disabled

We’re sorry, but errors in the uploaded zip file prevent it from being explored.

The error generated by the Zip attachment is:

Access to the path ‘\\etnaedndb02.embarcadero.com\f\webcache\cc\2004\4\19\21646.zip’ is denied.You may still be able to repair the zip file contents if you download the entire zip locally. You may also want to ask the author to repost the attachment.

Via [WayBack] delphi – Reference-counting for objects – Stack Overflow which also points to:

Downloads of stable versions: [WayBack] Download · ivmai/bdwgc Wiki · GitHub

–jeroen

Read the rest of this entry »

Posted in C, C++, Delphi, Development, Software Development | Leave a Comment »

List Of Windows Messages – WineHQ Wiki

Posted by jpluimers on 2020/12/17

Easiest way to find which message # (decimal or hexadecimal) belongs to which message and vice versa:

None of the lists are completely accurate, but they get you going.

For comparison: an early Windows 10 SDK WinUser.h and [Archive.is] NativeMethods.cs

Translations:

–jeroen

Posted in C, C++, Delphi, Development, Software Development | Leave a Comment »

Fl Ko: Getting the pixel position of the current editor cursor position for an OTA wizard

Posted by jpluimers on 2020/09/08

Since WayBack cannot save G+ photos, here is one from [WayBack] So, now I’m in front of a problem similar to a former one: I need to get the pixel position of the current editor cursor position for my OTA wizard. I’… – Fl Ko – Google+:

Related:

–jeroen

Posted in C++, C++ Builder, Delphi, Development, Software Development | Leave a Comment »