The Wiert Corner – irregular stream of stuff

Jeroen W. Pluimers on .NET, C#, Delphi, databases, and personal interests

  • My badges

  • Twitter Updates

  • My Flickr Stream

  • Pages

  • All categories

  • Enter your email address to subscribe to this blog and receive notifications of new posts by email.

    Join 1,854 other subscribers

Archive for the ‘Delphi’ Category

Delphi Tokyo binary BPLs not finding “@System@Net@Httpclient@Win@TCertificateStore@$bcctr$qqrv” means you likely have Delphi 10.2.2 installed

Posted by jpluimers on 2021/07/13

If you get errors loading BPLs that indicate this method cannot be found: “@System@Net@Httpclient@Win@TCertificateStore@$bcctr$qqrv” then it is likely you run Delphi 10.2.2.

Upgrade to Delphi 10.2.3 to resolve this problem.

This happens for instance with binary versions of these below libraries.

They cannot help, as Embarcadero removed the TCertificateStore class (present in Delphi 10.1.2) in 10.2.0, then re-added it in 10.2.3.

 

Libraries that I bumped into that were affected:

Note that some reports talk about “@System@Net@Httpclient@Win@TCertificateStore@$bcdtr$qqrv” in stead of “@System@Net@Httpclient@Win@TCertificateStore@$bcctr$qqrv“. I am not sure what causes this discrepancy yet.

–jeroen

Posted in Delphi, Delphi 10.2 Tokyo (Godzilla), Development, Software Development | Leave a Comment »

delphi – How to enable wirecompression on Firebird 3.0 – Stack Overflow

Posted by jpluimers on 2021/07/08

For my link archive:

–jeroen

Posted in Database Development, Delphi, Development, Firebird, Software Development | Leave a Comment »

E2213 Bad packaged unit format: ..\xxxxx.dcp.yyyy – Expected version: 32.0, Windows Unicode(x86) Found version: 105.101, Unk(CIL)

Posted by jpluimers on 2021/07/08

On my research list I have seen this happen with various libraries, but only libraries that deliver a single set of .bpl/.dcp files that you require for both debug and release builds.

The error occurs both ways:

  • building a “debug” build is fine, but after it a “release” build fails
  • building a “release” build is fine, but after it a “debug” build fails

This is with all separate DCU output directories for any permutation in any project: .\$(Platform)\$(Config)\$(SanitizedProjectName)

Example (where xxxx.pas is the first unit depending on Qrctrls, but I have seen it with other libraries that provide .bpl/.dcp combinations as well):

[dcc32 Fatal Error] xxxx.pas(14): E2213 Bad packaged unit format: ..\3rdParty\Quickrep506\19.0\Lib\Win32\Release\QR506RunDXE10_2.dcp.Qrctrls - Expected version: 32.0, Windows Unicode(x86) Found version: 105.101, Unk(CIL)

My best guess is that something is being cached in between builds, but not marked for the correct build configuration well enough.

–jeroen

Posted in Delphi, Delphi 10.2 Tokyo (Godzilla), Development, Software Development | Leave a Comment »

SetProcessWorkingSetSize: you hardly – if ever – need to call this from your process

Posted by jpluimers on 2021/07/07

There are quite a few posts that recommend using SetProcessWorkingSetSize to trim your process working set, usually in the SetProcessWorkingSetSize(ProcessHandle, -1, -1) form:

[WayBack] SetProcessWorkingSetSize function (winbase.h) | Microsoft Docs

Sets the minimum and maximum working set sizes for the specified process.

BOOL SetProcessWorkingSetSize(
  HANDLE hProcess,
  SIZE_T dwMinimumWorkingSetSize,
  SIZE_T dwMaximumWorkingSetSize );

The working set of the specified process can be emptied by specifying the value (SIZE_T)–1 for both the minimum and maximum working set sizes. This removes as many pages as possible from the working set. The [WayBack] EmptyWorkingSet function can also be used for this purpose.

In practice you hardly ever have to do this, mainly because this will write – regardless of (dis)usage – all of your memory to the pagefile, even the memory your frequently use.

Windows has way better heuristics to do that automatically for you, skipping pages you frequently use.

It basically makes sense in a few use cases, for instance when you know that most (like 90% or more) of that memory is never going to be used again.

Another use case (with specific memory sizes) is when you know that your program is going to use a defined range of memory, which is outside what Windows will heuristically expect from it.

A few more links that go into more details on this:

  • [WayBack] windows – Pros and Cons of using SetProcessWorkingSetSize – Stack Overflow answers by:
    • Hans Passant:

      SetProcessWorkingSetSize() controls the amount of RAM that your process uses, it doesn’t otherwise have any affect on the virtual memory size of your process. Windows is already quite good at dynamically controlling this, swapping memory pages out on demand when another process needs RAM.

      By doing this manually, you slow down your program a lot, causing a lot of page faults when Windows is forced to swap the memory pages back in.

      SetProcessWorkingSetSize is typically used to increase the amount of RAM allocated for a process. Or to force a trim when the app knows that it is going to be idle for a long time. Also done automatically by old Windows versions when you minimize the main window of the app.

    • Zack Yezek:

      The only good use case I’ve seen for this call is when you KNOW your process is going to hog a lot of the system’s RAM and you want to reserve it for the duration. You use it to tell the OS “Yes, I’m going to eat a lot of the system RAM during my entire run and don’t get in my way”.

    • Maxim Masiutin:

      We have found out that, for a GUI application written in Delphi for Win32/Win64 or written in a similar way that uses large and heavy libraries on top of the Win32 API (GDI, etc), it is worth calling SetProcessWorkingSetSize once.

      We call it with -1, -1 parameters, within a fraction of second after the application has fully opened and showed the main window to the user. In this case, the SetProcessWorkingSetSize(... -1, -1) releases lots of startup code that seem to not needed any more.

  • [WayBack] c# – How to set MinWorkingSet and MaxWorkingSet in a 64-bit .NET process? – Stack Overflow answer by Hans Passant:

    Don’t pinvoke this, just use the Process.CurrentProcess.MinWorkingSet property directly.

    Very high odds that this won’t make any difference. Soft paging faults are entirely normal and resolved very quickly if the machine has enough RAM. Takes ~0.7 microseconds on my laptop. You can’t avoid them, it is the behavior of a demand_paged virtual memory operating system like Windows. Very cheap, as long as there is a free page readily available.

    But if it “blips” you program performance then you need to consider the likelihood that it isn’t readily available and triggered a hard page fault in another process. The paging fault does get expensive if the RAM page must be stolen from another process, its content has to be stored in the paging file and has to be reset back to zero first. That can add up quickly, hundreds of microseconds isn’t unusual.

    The basic law of “there is no free lunch”, you need to run less processes or buy more RAM. With the latter option the sane choice, 8 gigabytes sets you back about 75 bucks today. Complete steal.

  • [WayBack] c++ – SetProcessWorkingSetSize usage – Stack Overflow answer by MSalters:

    I had an application which by default would close down entirely but keep listening for certain events. However, most of my code at that point would not be needed for a long time. To reduce the impact my process made, I called SetProcessWorkingSetSize(-1,-1);. This meant Windows could take back the physical RAM and give it to other apps. I’d get my RAM back when events did arrive.

    That’s of course unrelated to your situation, and I don’t think you’d benefit.

  • [WayBack] delphi – When to call SetProcessWorkingSetSize? (Convincing the memory manager to release the memory) – Stack Overflow

    If your goal is for your application to use less memory you should look elsewhere. Look for leaks, look for heap fragmentations look for optimisations and if you think FastMM is keeping you from doing so you should try to find facts to support it. If your goal is to keep your workinset size small you could try to keep your memory access local. Maybe FastMM or another memory manager could help you with it, but it is a very different problem compared to using to much memory.

    you can check the FasttMM memory usage via FasttMM calls GetMemoryManagerState and GetMemoryManagerUsageSummary before and after calling API SetProcessWorkingSetSize.

    I don’t need to use SetProcessWorkingSetSize. FastMM will eventually release the RAM.


    To confirm that this behavior is generated by FastMM (as suggested by Barry Kelly) I crated a second program that allocated A LOT of RAM. As soon as Windows ran out of RAM, my program memory utilization returned to its original value.

  • [WayBack] delphi – SetProcessWorkingSetSize – What’s the catch? – Stack Overflow answer by Rob Kennedy:

    Yes, it’s a bad thing. You’re telling the OS that you know more about memory management than it does, which probably isn’t true. You’re telling to to page all your inactive memory to disk. It obeys. The moment you touch any of that memory again, the OS has to page it back into RAM. You’re forcing disk I/O that you don’t actually know you need.

    If the OS needs more free RAM, it can figure out which memory hasn’t been used lately and page it out. That might be from your program, or it might be from some other program. But if the OS doesn’t need more free RAM, then you’ve just forced a bunch of disk I/O that nobody asked for.

    If you have memory that you know you don’t need anymore, free it. Don’t just page it to disk. If you have memory that the OS thinks you don’t need, it will page it for you automatically as the need arises.

    Also, it’s usually unwise to call Application.ProcessMessages unless you know there are messages that your main thread needs to process that it wouldn’t otherwise process by itself. The application automatically processes messages when there’s nothing else to do, so if you have nothing to do, just let the application run itself.

–jeroen

Posted in .NET, C, C++, Delphi, Development, Software Development, Windows Development | Leave a Comment »

Delphi: bumped into System.Classes.TLoginCredentialService while doing a “DefaultUsrPwDm” – Google Search

Posted by jpluimers on 2021/07/06

Doing a  “DefaultUsrPwDm” – Google Search, I bumped into the System.Classes.TLoginCredentialService class.

There is hardly any documentation.

It was introduced as [WayBack] System.Classes.TLoginCredentialService – XE2 API Documentation with the standard TObject template:

TObject is the ultimate ancestor of all objects and components.

System.Classes.TLoginCredentialService inherits from System.TObject. All content below this line refers to System.TObject.

TObject is the ultimate ancestor of all objects and components.

Although TObject is the based object of a component framework, not all objects are components. All component classes descend from TComponent.

Note: TObject is never directly instantiated. Although it does not use programming language features that prevent instantiation, TObject is an abstract class.

The documentation was updated in Delphi XE4 to [WayBack] System.Classes.TLoginCredentialService – RAD Studio API Documentation

TLoginCredentialService provides functionality for login action, regardless of framework.

TLoginCredentialService represents an extensible, framework agnostic, login credential service with support for callbacks (success, failure). It includes methods for:

  • Getting user credentials: DomainUsername and Password.
  • Registering and unregistering credentials from handlers.

See Also

That version also added [WayBack] System.Classes.ELoginCredendialError – RAD Studio API Documentation

ELoginCredendialError is the exception class for handling invalid login credentials.

ELoginCredendialError is raised when an application attempts to get specified, but unexisting, login credentials.

Nothing ever got added, no examples, no documentation, just look at for instance the Delphi Rio documentation:

Luckily, I found this for my research list (code at [WayBack] tlogincredentialservice.zip) : [WayBack] delphi – TLoginCredentialService usage example – Stack Overflow which uses globals, but that’s not the point of the demo (you can use different storage in your actual code)

I’ve just created a small demo of how to use it

Click here to download the code

In the following I’ll show some of the code:

First I need a record to hold Credentials, and a list of them :

Type
  TCredential = record
    Username, Password, Domain: string;
    constructor Create(const aUsername, aPassword, aDomain: string);
    function AreEqual(const aUsername, aPassword, aDomain: string): Boolean;
  end;

  TCredentialList = class(TList<TCredential>)
  public
    function IsValidCredential(const aUsername, aPassword, aDomain: string): Boolean;
  end;

then we need to define a context in wich we are calling this. Thats just a application unique string wich identifyes each Loginfunction

const
  Context = 'TForm1';

In Form create I create my list and add dummy data to it

procedure TForm1.FormCreate(Sender: TObject);
begin
  CredentialList := TCredentialList.Create;
  //Add Dummy data
  CredentialList.Add(TCredential.Create('AA', 'AA', 'DomainAA'));
  CredentialList.Add(TCredential.Create('BB', 'BB', 'DomainAA'));
  CredentialList.Add(TCredential.Create('CC', 'CC', 'DomainAA'));

  // Register your Login handler in a context.
  // This method is called when you try to login
  // by caling TLoginCredentialService.GetLoginCredentials();
  TLoginCredentialService.RegisterLoginHandler(Context, LoginCredentialEvent);
end;

I have placed a button on my form from wich I make my call to login :

procedure TForm1.Button1Click(Sender: TObject);
begin
  // The actual call to login
  // First param is the context
  // Second Parameres is a callback function given to the event handler.
  TLoginCredentialService.GetLoginCredentials(Context,
    function { LoginFunc } (const Username, Password, Domain: string): Boolean
    begin
      //The actual user validation
      Result := CredentialList.IsValidCredential(Username, Password, Domain);
    end);
end;

Finally I just need to implement my loginhandler:

//This is the "onLogin" event handler.
//This is called durring a login attempt
//The purpose of this event handler are to call tha callBack function with correct information
//and handle the result
procedure TForm1.LoginCredentialEvent(Sender: TObject; Callback: TLoginCredentialService.TLoginEvent; var Success: Boolean);
begin
  //Call the callback
  Callback(Sender, LabeledEdit1.Text, LabeledEdit2.Text, LabeledEdit3.Text, Success);

  //Handle the success.
  if Success then
    Label1.Caption := 'Yes'
  else
    Label1.Caption := 'No';
end;

Hope this answers the question.Dont forget to download the complete code here

–jeroen

Posted in Delphi, Development, Software Development | Leave a Comment »

Delphi: TArray of T versus TArray; a function can return the latter, but not the former

Posted by jpluimers on 2021/07/05

I wish I had blogged about this a lot sooner, as then far less people would use var aFoo: array of TFoo as method parameters for results that are just out parameters and could be a function result.

You cannot have array of TFoo as function result, but you can have TArray<TFoo>. The former would be an open array, the latter is a proper array type.

I see many people use var aFoo: array of TFoo with all sorts of SetLength calls before calling and inside a method where a function returning a TArray<TFoo> would be far more appropriate, both in the sense of readability and maintainability.

–jeroen

Posted in Delphi, Development, Software Development | 2 Comments »

Tuple class and records in the Spring4D framework

Posted by jpluimers on 2021/07/01

The Spring Framework for Delphi has some cool tuple support.

The [WayBack] Tuple class has 3 create methods that build generic Tuple<> records with 2, 3 and 4 fields:

–jeroen

Posted in Delphi, Development, Software Development, Spring4D | Leave a Comment »

Delphi interface generic function – Is there a work around? – Stack Overflow

Posted by jpluimers on 2021/06/30

The documentation simply states “no”, without explaining why.

Luckily, there is [WayBack] David Heffernan at [WayBack] Delphi interface generic function – Is there a work around? – Stack Overflow

Interfaces do not support generic parameterized methods, as the compiler says.

There is no workaround because it’s a fundamental limitation. Parameterized methods in classes are implemented by adding one method per instantiation to the class. That works for classes since they are concrete, but is not viable for interfaces. That’s because interfaces are a table of functions and the size of that table cannot vary depending on which generic method instantiations happen to be present elsewhere in the code. For similar reasons, generic methods cannot be virtual or dynamic.

In any case, it’s not possible. One option is to use a class instead. I agree that this is a bind.

If you want to do it in an interface, the best you can do is:

function CastAs(const IID: TGUID): IInterface;

But you’d have to call it like this:

MyIntf := ProxyIntf.CastAs(IMyIntf) as IMyIntf;

which feels somewhat foul.

Choose your poison!

In the background, for every distinct use of generic method parameters, the compiler generates an overloaded concrete version of the method. That is possible for classes and records, but not for interfaces.

In practice, I either go for the last option, or have the interface expose a record or class type that can have the required generic method (so the compiler can generate the overloads at run-time).

All options feel somewhat ugly, so I like the comment by [WayBack] Jason:

Wish I could “Program into Your Language, Not in it” programmers.stackexchange.com/questions/2777/…

Which leads to Code Complete

[WayBack] What are the key points of Code Complete? – Software Engineering Stack Exchange answer by [WayBack] limist:

Code Complete is about software craftsmanship; it is an advanced-beginner/intermediate-level book, written for the working programmer, but it would still be very useful to someone who’s been programming for at least a year.

Thus the key points of Code Complete (2nd ed.) are nicely summarized in its Chapter 34, Themes in Software Craftsmanship. As paraphrased from my notes:

  1. Conquer Complexity: reduce the cognitive load on your mind via discipline, conventions, and abstraction.
  2. Pick Your Process: be conscious of quality from start (requirements) to finish (deployment) and beyond (maintenance).
  3. Write Programs for People First, Computers Second: code readability is hugely important for comprehensibility, review-ability, error-rate, error-correction, modifiability, and the consequent development time and quality.
  4. Program into Your Language, Not in it: think of the What? and Why? before the How?
  5. Focus Your Attention with the Help of Conventions: conventions manage complexity by providing structure where it’s needed, so that the ultimate resource – your attention – can be effectively used.
  6. Program in Terms of the Problem Domain: work at the highest level of abstraction possible; top-level code should describe the problem being solved. Distinguish OS level, programming language level, low-level implementation structures, low-level problem domain terms, and finally, high-level problem-domain terms that would make total sense to the (non-coder) user.
  7. Watch for Falling Rocks: as programming merges art and science, good judgement is vital, including heeding warning signs.
  8. Iterate, Repeatedly, Again and Again: iterate requirements, design, estimates, code, code tuning.
  9. Thou Shalt Render Software and Religion Asunder: be eclectic and willing to experiment. Don’t be an inflexible zealot, it precludes curiosity and learning. Go beyond having just a hammer in your toolbox.

But the most important take-aways are in Chapter 33, Personal Character: once you consciously seek to improve as a coder, you can and will. The fastest way to do so is to take on the the attitudes of master coders(humility, curiosity, intellectual honesty, discipline, creativity), while also practicing their habits (many good habits are listed in the book, e.g. choosing good variable/value names).

Also, the book makes clear that the gap between average and excellent in software is immense; that fact alone should drive the conscientious coder to better himself.

That’s the short of it; the long version is in the book. :) I can also send you my not-so-long, not-so-short notes if you want more details. But the book is certainly money and time well-spent, even if the writing style is tiresome at times.

Beyond Code Complete, I’d highly recommend The Pragmatic Programmer. It’s for intermediate-level programmers, nicely-written and a great mix of high, medium, and low-level advice.

There are more limitations on generics in Delphi

The documentation on limitations has not changed much since Delphi 2009.

Delphi 2009

[WayBack] Overview of Generics: Platform Requirements and Differences

Generics are supported by the Delphi for Win32 compiler.

Runtime type identification (RTTI)

In Win32, generics and methods do not have RTTI, but instantiated types do have RTTI. An instantiated type is the combination of a generic with a set of parameters.

Interface GUID

In Win32, an instantiated interface type does not have an interface GUID.

Parameterized method in interface

A parameterized method (method declared with type parameters) cannot be declared in an interface.

Instantiation timing

Instantiation is processed by the compiler. All instantiated objects are emitted into .obj files.

Dynamic instantiation

Dynamic instantiation at runtime is not supported.

Interface constraints

The Win32 interface is not a “light” interface. This means all type parameters with interface constraints always support the COM IUnknown methods _AddRef_Release and QueryInterface or inherit from TInterfacedObject. Record types cannot specify an interface constraint parameter.

Delphi 10.3 Rio

[WayBack] Overview of Generics – RAD Studio: Platform Requirements and Differences

Generics are supported by the Delphi compilers.

Run-time type identification

In Win32, generics and methods do not have run-time type information (RTTI), but instantiated types do have RTTI. An instantiated type is the combination of a generic with a set of parameters. The RTTI for methods of a class is a subset of the RTTI for that class as a whole. If a non-generic class has a generic method, that method will not be included in the RTTI generated for the class because generics are instantiated at compile time, not at run time.

Interface GUID

In Win32, an instantiated interface type does not have an interface GUID.

Parameterized method in interface

A parameterized method (method declared with type parameters) cannot be declared in an interface.

Instantiation timing

Generic types are instantiated at compile time and emitted into executables and relocatable files. Instance variables of a generic type are instantiated at run time for classes and at compile time for generic records. The RTTI for generic classes is only generated when the classes are instantiated. RTTI for instantiated classes follows just as for non-generic classes. If the generic class has a generic method, then the instantiated generic class will not have RTTI for that generic method.

Dynamic instantiation

Dynamic instantiation at run time is not supported.

Interface constraints

The Win32 interface is not a “light” interface. This means all type parameters with interface constraints always support the COM IUnknown methods _AddRef_Release, and QueryInterface or inherit from TInterfacedObject. Record types cannot specify an interface constraint parameter.

–jeroen

Posted in Conference Topics, Conferences, Delphi, Development, Event, Software Development | Leave a Comment »

DCOM calls from thread pool threads: CoInitialize/CoUnitialize location and expensiveness?

Posted by jpluimers on 2021/06/24

Interesting takeaway from [WayBack] DCOM calls from thread pool threads

call CoInitialize* at the start, and call CoUninitialize before returning. Expensive, but necessary

Related:

–jeroen

Posted in .NET, C, C++, COM/DCOM/COM+, Delphi, Development, Software Development, Windows Development | Leave a Comment »

“No mapping for the Unicode character exists in the target multi-byte code page”

Posted by jpluimers on 2021/06/24

Usually when I see this error [Wayback] “No mapping for the Unicode character exists in the target multi-byte code page” – Google Search, it is in legacy code that uses string buffers where decoding or decompressing data into.

This is almost always wrong no matter what kind of data you use, as it will depend in your string encoding.

I have seen it happen especially in these cases:

  • base64 decoding from string to string (solution: decode from a string stream into a binary stream, then post-process from there)
  • zip or zlib decompress from binary stream to string stream, then reading the string stream (solution: decompress from binary stream to binary stream, then post-process from there)

Most cases I encountered were in Delphi and C code, but surprisingly I also bumped into C# exhibiting this behaviour.

I’m not alone, just see these examples from the above Google search:

–jeroen

Posted in .NET, base64, C, C#, C++, Delphi, Development, Encoding, Software Development, Unicode | Leave a Comment »