The Wiert Corner – irregular stream of stuff

Jeroen W. Pluimers on .NET, C#, Delphi, databases, and personal interests

  • My badges

  • Twitter Updates

  • My Flickr Stream

  • Pages

  • All categories

  • Enter your email address to subscribe to this blog and receive notifications of new posts by email.

    Join 1,616 other followers

Windows Vista/7/8/… hangs for Windows Common dialogs when your COM initialisation is wrong

Posted by jpluimers on 2016/02/03

A while ago I bumped into this: As of Windows 7 (and probably Vista), the standard Windows Common Item (Open and Save) Dialogs expect the main thread to be initialised with STA because it is easier to support apartment threading in single-threaded apartments because COM provides synchronization on a per-call basis and the Windows GUI APIs are not guaranteed to be thread safe.

Windows XP and Server 2003 didn’t enforce this for the classic Windows Open and Save Dialogs, so it only appeared when the software below got run on Windows 7 in a way too late time frame (but the market share of XP is still high).

The reason is that when using Delphi, the TOpenDialog and TSaveDialog will use the classic Open and Save Dialogs on Windows < Vista and fall-forward to the new Common Item Dialogs handled by TFileOpenDialog and TFileSaveDialog (both will not fall backward).

When you have your COM initialisation done wrong, your application appears to hang. Amidst the plethora of threads started by the COM subsystem, these two dead-lock:

main thread COM dialog thread

:755a1148 kernel32.WaitForSingleObject + 0x12
:751f7690 ; C:\Windows\syswow64\ole32.dll

:7531ce06 ; C:\Windows\syswow64\ole32.dll
:7544421b ; C:\Windows\syswow64\RPCRT4.dll
:643a94b0 ; C:\Windows\SysWOW64\actxprxy.dll
:643a9587 ; C:\Windows\SysWOW64\actxprxy.dll
Dialogs.TCustomFileDialog.Execute(7145446)
Dialogs.TFileDialogWrapper.Execute(7145446)
Dialogs.TOpenDialog.DoExecute($42EA10,7145446)
Dialogs.TOpenDialog.Execute(7145446)
ExtDlgs.TOpenPictureDialog.Execute(???)
Dialogs.TCommonDialog.Execute
OpenDialogMainFormUnit.TOpenDialogMainForm.OpenPictureDialogButtonClick(???)
Controls.TControl.Click
StdCtrls.TButton.Click

Forms.TCustomForm.WndProc(???)
Controls.TWinControl.MainWndProc(???)
Classes.StdWndProc(7145446,273,2502,1182150)
:770262fa ; C:\Windows\syswow64\USER32.dll

:7703794a USER32.CallWindowProcA + 0x1b
Controls.TWinControl.DefaultHandler(???)

:0041f686 StdWndProc + $16
:77026d3a USER32.GetThreadDesktop + 0xd7
:77027bca USER32.DispatchMessageA + 0xf
:77032d99 USER32.SetParent + 0x35
:7704ce8a USER32.DialogBoxIndirectParamAorW + 0x36
:7704cc0e USER32.DialogBoxIndirectParamW + 0x1b
:75d6597b ; C:\Windows\syswow64\comdlg32.dll
:643a95b3 ; C:\Windows\SysWOW64\actxprxy.dll
:64385630 ; C:\Windows\SysWOW64\actxprxy.dll
:754c0966 RPCRT4.NdrUnmarshallBasetypeInline + 0x74
:7531d7e6 ; C:\Windows\syswow64\ole32.dll

:75239326 ; C:\Windows\syswow64\ole32.dll
:770262fa ; C:\Windows\syswow64\USER32.dll
:77026d3a USER32.GetThreadDesktop + 0xd7
:770277c4 ; C:\Windows\syswow64\USER32.dll
:7702788a USER32.DispatchMessageW + 0xf
:751fa48b ; C:\Windows\syswow64\ole32.dll

:7520d87a ; C:\Windows\syswow64\ole32.dll
:755a338a kernel32.BaseThreadInitThunk + 0x12
:777c9f72 ntdll.RtlInitializeExceptionChain + 0x63
:777c9f45 ntdll.RtlInitializeExceptionChain + 0x36

Example code to reproduce this is in my BeSharp.net repository.

The hang in the main thread is on this line of code in the below fragement:

Result := Succeeded(Show(ParentWnd)); //// <—- hangs here

Thanks to the help on G+ it was easy to track down it was indeed a COM issue.

The DLLs ole32.dll and actxprxy.dll sort of give it away: OLE and COM ActiveX proxy.

And indeed: the code contained this (for someone in the community familiar because he copy-pasted a Google search result) piece of code running before the Application.Initialize:

procedure TDMConfigMgr.DataModuleCreate(Sender: TObject);
begin
  CoInitializeEx(nil, 0);
  if not xmldoc.Active then
    xmldoc.Active := True;
end;

Solving the symptom was easy, just replace the COM initialisation with this:

CoInitializeEx(nil, COINIT_APARTMENTTHREADED);

But that is not the proper solution. The are is more wrong with the above code:

  1. There is no CoUninitialize (see below why this important)
  2. The usage of COM and initialization of COM are bound together (so what if someone else needs a different kind of initialisation?)

The proper solution is to deferring your dependency on the COM initialisation and letting Delphi do the COM initialisation by specifying it in your main project like this:

uses
  Forms,
...
  ActiveX;

{$R *.res}

begin
  CoInitFlags := COINIT_APARTMENTTHREADED;
  Application.Initialize;
...

Replace the COINIT_APARTMENTTHREADED to CoInitFlags with the required threading taking the below information into account so that Application.Initialize will pass it to the COM subsystem in the (undocumented) InitComObj method of the ComObj unit.

If you really need to use MTA, then read the COM FAQs (RemObjects SDK) – RemObjects Wiki about some more details: it’s tricky and complicated.

Background information

On the Delphi side, these by Delphi COM expert Chris Bensen are the canonical posts to get CoInitialize/CoInitializeEx and CoUninitialize right:

Read the underlying links for more information; here a quick summary:

CoInitialize/CoInitializeEx/OleInitialize initialize COM for a thread. CoUninitialize unitializes COM for a thread. OleInitialize and the “deprecated” CoInitialize set COM up for STA (Single-Threaded Apartment) mode by calling CoInitializeEx with the COINIT enumeration value COINIT_APARTMENTTHREADED.

A confusion for CoInitializeEx is that passing zero for the COINIT value is COINIT_MULTITHREADED which is not the default that OleInitialize and CoInitialize use.

The COINIT enumeration is a flag, so you can add these values together:

value indentifier meaning remark
2 COINIT_APARTMENTTHREADED Initializes the thread for apartment-threaded object concurrency. STA: Default that OleInitialize and CoInitialize use for calling CoInitializeEx.
0 COINIT_MULTITHREADED Initializes the thread for multithreaded object concurrency. MTA: Requires all your code to be thread-safe; Windows UI code isn’t thread safe.
4 COINIT_DISABLE_OLE1DDE Disables DDE for OLE1 support.
8 COINIT_SPEED_OVER_MEMORY Increase memory usage in an attempt to increase performance.

MSDN links:

SO links:

–jeroen

2 Responses to “Windows Vista/7/8/… hangs for Windows Common dialogs when your COM initialisation is wrong”

  1. Bott said

    Great stuff, thank you.

    • jpluimers said

      You’re most welcome.

      COM is like threading: they are both hard, especially when combined. People assume these technologies are a solution and can implement them without giving it much thought thereby introducing more problems than they had in the first place.

      What nerved me most researching the cause is that the code was apparently google-copy-pasted by one of the Professional Services Organisation members without consulting with the R&D team first.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

 
%d bloggers like this: