The Wiert Corner – irregular stream of stuff

Jeroen W. Pluimers on .NET, C#, Delphi, databases, and personal interests

  • My badges

    Mastodon

  • Twitter Updates

  • My Flickr Stream

    20140508-Delphi-2007--Project-Options--Cannot-Edit-Application-Title-HelpFile-Icon-Theming20140430-Fiddler-Filter-Actions-Button-Run-Filterset-now20140424-VMware-Fusion-6.0.3.-no-reclaimable
    More Photos

  • Pages

  • All categories

  • Enter your email address to subscribe to this blog and receive notifications of new posts by email.

    Join 1,835 other subscribers

Archive for the ‘C’ Category

« Previous Entries
Next Entries »

NTFS Sparse Files For Programmers

Posted by jpluimers on 2024/09/25

Need to check this out some day: cs.exe compiled from [Wayback] sparse.zip which you can download from [Wayback/Archive] NTFS Sparse Files For Programmers

Read the rest of this entry »

Posted in C, C++, Development, NTFS, Power User, RoboCopy, Software Development, Visual Studio C++, Windows, Windows 10, Windows 11 | Leave a Comment »

ladislav-zezula/FileTest: Source code for File Test – Interactive File System Test Tool

Posted by jpluimers on 2024/08/14

Cool tool to peek around in the Windows File System API and fiddle around uncharted territory: [Wayback/Archive] ladislav-zezula/FileTest: Source code for File Test – Interactive File System Test Tool.

It is written in C++ using Visual Studio and has build-instructions in [Wayback/Archive] FileTest/README.md at master · ladislav-zezula/FileTest.

Via [Wayback/Archive] “create reparse point” “query reparse point” – Google Search (which also found [Wayback/Archive] Free File Utilities – Free download and software reviews – CNET Download [Wayback download]) after reading a tweet thread having these highlights:

Read the rest of this entry »

Posted in C, C++, Development, Polyglot, Software Development, Visual Studio C++, Windows Development | Leave a Comment »

Code Visualisation through Python Tutor – Visualise Python, Java, C, C++, JavaScript, TypeScript, and Ruby code execution

Posted by jpluimers on 2024/04/18

The final visualisation post of this week (themed Data Visualisation and Code Visualisation) is about [Wayback/Archive] Python Tutor – Visualize Python, Java, C, C++, JavaScript, TypeScript, and Ruby code execution.

Languages covered in these visualisers:

Earlier posts in the series:

–jeroen

Posted in C, C++, Development, Java, Java Platform, JavaScript/ECMAScript, Python, Ruby, Scripting, Software Development | Leave a Comment »

XZ 5.6.x are backdoored and present in many systems: downgrade to 5.4.x or earlier now; consider libarchive compromised until proven otherwise

Posted by jpluimers on 2024/03/30

Edit 20240331: because of

https://mastodon.social/@kobold/112183756981119562

Debian is working on reverting back to even earlier than 5.4.x

[Wayback/Archive] #1068024 – revert to version that does not contain changes by bad actor – Debian Bug report logs

> I'd suggest reverting to 5.3.1. Bearing in mind that there were security
> fixes after that point for ZDI-CAN-16587 that would need to be reapplied.

Note that reverted to such an old version will break packages that use
new symbols introduced since then. From a quick look, this is at least:
- dpkg
- erofs-utils
- kmod

Having dpkg in that list means that such downgrade has to be planned
carefully.

Original post:

Everything I know about the XZ backdoor

Note that because of the Wayback Machine limit of 5 archivals per URL per day, the archived versions are rapidly getting out-of-date.

It is way worse:

[Wayback/Archive] Thread by @_ruby on Thread Reader App – Thread Reader App

@_ruby: The setup behind the CVE-2024-3094 supply-chain attack is fascinating. I originally wanted to finish and share a tool to audit other OSS projects for anomalous contributor behavior, but I feel what I found tr……

How it was found:

Analogy on how it was found:

Via:

Related:

If you are running homebrew on a Mac, then update too:

Of course this “XKCD dependency” adoption applies:

[Wayback/Archive] GJ4KvbeWIAAS_mu (535×680)

Posted in C, Compression, Development, Infosec (Information Security), Power User, Security, Software Development, xz | Leave a Comment »

When Microsoft download URLs time out: check if it other IP addresses for the same host do work fine (it might be a regional Microsoft CDN issue)

Posted by jpluimers on 2024/03/28

A while back, early in the Wednesday morning after Patch Tuesday I performed regular updates of all the systems noticing some updates failed because timeouts on the Microsoft download servers.

Note I perform the manual steps on Wednesday as Patch Tuesday as it starts at 10:00 AM PST which is in the evening in Amsterdam. The automated steps are automated and kick in when Microsoft tells the Windows machines to update themselves.

See [Wayback/Archive] Security Update Guide FAQs

Microsoft schedules the release of security updates on “Patch Tuesday,” the second Tuesday of each month at 10:00 AM PST.

Depending on time zone(s) in which the organization operates, IT pros should plan their deployment schedules accordingly. Please note that there are some products that do not follow the Patch Tuesday schedule.

I posted a gist and a Tweet, but didn’t immediately thought of a good resolution so I postponed that until Thursday and found it:

Read the rest of this entry »

Posted in C, C++, Development, Power User, Software Development, Visual Studio and tools, Visual Studio C++, vscode Visual Studio Code, Windows, Windows Development | Leave a Comment »

BitSavers added some more Borland documentation in 2022 (most relevant for me: Assembler, Debugger, Profiler)

Posted by jpluimers on 2024/02/28

Below is a list of the Borland documentation that BitSavers added in 2022, ordered by relevance to me (and how I finally asked Peter Sawatzki if he still had the monochrome TDVIDEO.DLL he wrote for Turbo Debugger 3.0 for Windows):

Read the rest of this entry »

Posted in 8087, Algorithms, Assembly Language, Borland C++, C, C++, Debugging, Development, Floating point handling, Profiling, Software Development, Turbo Assembler, Turbo C, Turbo Debugger, Turbo Profiler, x86 | Leave a Comment »

That new Carbon language, it has a net zero footprint, right?

Posted by jpluimers on 2022/08/25

No, Carbon is not the same thing as the same named MacOS Carbon API it .

Someone had to make it: [Wayback/Archive] Jeroen Wiert Pluimers on Twitter: “@d_feldman That has a net zero Carbon footprint, right? (OK, I’ll show myself out)”.

Luckily Googlers themselves are joking about it as well:

Read the rest of this entry »

Posted in C, C++, Carbon, Development, Software Development | Leave a Comment »

When floating point code suddenly becomes orders magnitudes slower (via C++ – Why does changing 0.1f to 0 slow down performance by 10x? – Stack Overflow)

Posted by jpluimers on 2022/01/26

When working with converging algorithms, sometimes floating code can become very slow. That is: orders of magnitude slower than you would expect.

A very interesting answer to [Wayback] c++ – Why does changing 0.1f to 0 slow down performance by 10x? – Stack Overflow.

I’ve only quoted a few bits, read the full question and answer for more background information.

Welcome to the world of denormalized floating-point! They can wreak havoc on performance!!!

Denormal (or subnormal) numbers are kind of a hack to get some extra values very close to zero out of the floating point representation. Operations on denormalized floating-point can be tens to hundreds of times slower than on normalized floating-point. This is because many processors can’t handle them directly and must trap and resolve them using microcode.

If you print out the numbers after 10,000 iterations, you will see that they have converged to different values depending on whether 0 or 0.1 is used.

Basically, the convergence uses some values closer to zero than a normal floating point representation dan store, so a trick is used called “denormal numbers or denormalized numbers (now often called subnormal numbers)” as described in Denormal number – Wikipedia:

…

In a normal floating-point value, there are no leading zeros in the significand; rather, leading zeros are removed by adjusting the exponent (for example, the number 0.0123 would be written as 1.23 × 10−2). Denormal numbers are numbers where this representation would result in an exponent that is below the smallest representable exponent (the exponent usually having a limited range). Such numbers are represented using leading zeros in the significand.

…

Since a denormal number is a boundary case, many processors do not optimise for this.

–jeroen

Posted in .NET, Algorithms, ARM, Assembly Language, C, C#, C++, Delphi, Development, Software Development, x64, x86 | Leave a Comment »

Ken Thompson’s 1980 Unix password got cracked only a while ago: he used much entropy in his password

Posted by jpluimers on 2021/07/22

A few years back, Ken Thompson’s 1980 Unix password got cracked.

It took that long because his password p/q2-q4! had enough entropy by using uncommon characters so the crypt(3) based hash ZghOT0eRm4U9s was hard to crack.

The password was an opening chess move noted in the variety of the descriptive notation. A shorter notation would have been P-Q4, which would require months to crack in that era.

In modern chess notation, it would be 1. d4, moving the Queen’s Pawn from d2 to d4.

References (many interesting messages in the TUHS thread below):

Read the rest of this entry »

Posted in *nix, B, C, Development, Power User, Security, Software Development | Leave a Comment »

SetProcessWorkingSetSize: you hardly – if ever – need to call this from your process

Posted by jpluimers on 2021/07/07

There are quite a few posts that recommend using SetProcessWorkingSetSize to trim your process working set, usually in the SetProcessWorkingSetSize(ProcessHandle, -1, -1) form:

[WayBack] SetProcessWorkingSetSize function (winbase.h) | Microsoft Docs

Sets the minimum and maximum working set sizes for the specified process.

BOOL SetProcessWorkingSetSize(
  HANDLE hProcess,
  SIZE_T dwMinimumWorkingSetSize,
  SIZE_T dwMaximumWorkingSetSize );

The working set of the specified process can be emptied by specifying the value (SIZE_T)–1 for both the minimum and maximum working set sizes. This removes as many pages as possible from the working set. The [WayBack] EmptyWorkingSet function can also be used for this purpose.

In practice you hardly ever have to do this, mainly because this will write – regardless of (dis)usage – all of your memory to the pagefile, even the memory your frequently use.

Windows has way better heuristics to do that automatically for you, skipping pages you frequently use.

It basically makes sense in a few use cases, for instance when you know that most (like 90% or more) of that memory is never going to be used again.

Another use case (with specific memory sizes) is when you know that your program is going to use a defined range of memory, which is outside what Windows will heuristically expect from it.

A few more links that go into more details on this:

  • [WayBack] windows – Pros and Cons of using SetProcessWorkingSetSize – Stack Overflow answers by:
    • Hans Passant:

      SetProcessWorkingSetSize() controls the amount of RAM that your process uses, it doesn’t otherwise have any affect on the virtual memory size of your process. Windows is already quite good at dynamically controlling this, swapping memory pages out on demand when another process needs RAM.

      By doing this manually, you slow down your program a lot, causing a lot of page faults when Windows is forced to swap the memory pages back in.

      SetProcessWorkingSetSize is typically used to increase the amount of RAM allocated for a process. Or to force a trim when the app knows that it is going to be idle for a long time. Also done automatically by old Windows versions when you minimize the main window of the app.

    • Zack Yezek:

      The only good use case I’ve seen for this call is when you KNOW your process is going to hog a lot of the system’s RAM and you want to reserve it for the duration. You use it to tell the OS “Yes, I’m going to eat a lot of the system RAM during my entire run and don’t get in my way”.

    • Maxim Masiutin:

      We have found out that, for a GUI application written in Delphi for Win32/Win64 or written in a similar way that uses large and heavy libraries on top of the Win32 API (GDI, etc), it is worth calling SetProcessWorkingSetSize once.

      We call it with -1, -1 parameters, within a fraction of second after the application has fully opened and showed the main window to the user. In this case, the SetProcessWorkingSetSize(... -1, -1) releases lots of startup code that seem to not needed any more.

  • [WayBack] c# – How to set MinWorkingSet and MaxWorkingSet in a 64-bit .NET process? – Stack Overflow answer by Hans Passant:

    Don’t pinvoke this, just use the Process.CurrentProcess.MinWorkingSet property directly.

    Very high odds that this won’t make any difference. Soft paging faults are entirely normal and resolved very quickly if the machine has enough RAM. Takes ~0.7 microseconds on my laptop. You can’t avoid them, it is the behavior of a demand_paged virtual memory operating system like Windows. Very cheap, as long as there is a free page readily available.

    But if it “blips” you program performance then you need to consider the likelihood that it isn’t readily available and triggered a hard page fault in another process. The paging fault does get expensive if the RAM page must be stolen from another process, its content has to be stored in the paging file and has to be reset back to zero first. That can add up quickly, hundreds of microseconds isn’t unusual.

    The basic law of “there is no free lunch”, you need to run less processes or buy more RAM. With the latter option the sane choice, 8 gigabytes sets you back about 75 bucks today. Complete steal.

  • [WayBack] c++ – SetProcessWorkingSetSize usage – Stack Overflow answer by MSalters:

    I had an application which by default would close down entirely but keep listening for certain events. However, most of my code at that point would not be needed for a long time. To reduce the impact my process made, I called SetProcessWorkingSetSize(-1,-1);. This meant Windows could take back the physical RAM and give it to other apps. I’d get my RAM back when events did arrive.

    That’s of course unrelated to your situation, and I don’t think you’d benefit.

  • [WayBack] delphi – When to call SetProcessWorkingSetSize? (Convincing the memory manager to release the memory) – Stack Overflow

    If your goal is for your application to use less memory you should look elsewhere. Look for leaks, look for heap fragmentations look for optimisations and if you think FastMM is keeping you from doing so you should try to find facts to support it. If your goal is to keep your workinset size small you could try to keep your memory access local. Maybe FastMM or another memory manager could help you with it, but it is a very different problem compared to using to much memory.

    …

    you can check the FasttMM memory usage via FasttMM calls GetMemoryManagerState and GetMemoryManagerUsageSummary before and after calling API SetProcessWorkingSetSize.

    …

    I don’t need to use SetProcessWorkingSetSize. FastMM will eventually release the RAM.

    To confirm that this behavior is generated by FastMM (as suggested by Barry Kelly) I crated a second program that allocated A LOT of RAM. As soon as Windows ran out of RAM, my program memory utilization returned to its original value.

  • [WayBack] delphi – SetProcessWorkingSetSize – What’s the catch? – Stack Overflow answer by Rob Kennedy:

    Yes, it’s a bad thing. You’re telling the OS that you know more about memory management than it does, which probably isn’t true. You’re telling to to page all your inactive memory to disk. It obeys. The moment you touch any of that memory again, the OS has to page it back into RAM. You’re forcing disk I/O that you don’t actually know you need.

    If the OS needs more free RAM, it can figure out which memory hasn’t been used lately and page it out. That might be from your program, or it might be from some other program. But if the OS doesn’t need more free RAM, then you’ve just forced a bunch of disk I/O that nobody asked for.

    If you have memory that you know you don’t need anymore, free it. Don’t just page it to disk. If you have memory that the OS thinks you don’t need, it will page it for you automatically as the need arises.

    Also, it’s usually unwise to call Application.ProcessMessages unless you know there are messages that your main thread needs to process that it wouldn’t otherwise process by itself. The application automatically processes messages when there’s nothing else to do, so if you have nothing to do, just let the application run itself.

–jeroen

Posted in .NET, C, C++, Delphi, Development, Software Development, Windows Development | Leave a Comment »

« Previous Entries
Next Entries »