21

David's answer to another question shows a Delphi DLL function returning a WideString. I never thought that was possible without the use of ShareMem.

My test DLL:

function SomeFunction1: Widestring; stdcall;
begin
  Result := 'Hello';
end;

function SomeFunction2(var OutVar: Widestring): BOOL; stdcall;
begin
  OutVar := 'Hello';
  Result := True;
end;

My caller program:

function SomeFunction1: WideString; stdcall; external 'Test.dll';
function SomeFunction2(var OutVar: Widestring): BOOL; stdcall; external 'Test.dll';

procedure TForm1.Button1Click(Sender: TObject);
var
  W: WideString;
begin
  ShowMessage(SomeFunction1);
  SomeFunction2(W);
  ShowMessage(W);
end;

It works, and I don't understand how. The convention I know of is the one used by the Windows API, for example Windows GetClassNameW:

function GetClassNameW(hWnd: HWND; lpClassName: PWideChar; nMaxCount: Integer): Integer; stdcall;

Meaning the caller provides the buffer, and the maximum length. The Windows DLL writes to that buffer with the length limitation. The caller is allocates and deallocates the memory.

Another option is that the DLL allocate the memory for example by using LocalAlloc, and the Caller deallocates the memory by calling LocalFree.

How does the memory allocation and deallocation work with my DLL example? Does the "magic" happen because the result is WideString(BSTR)? And why aren't Windows APIs declared with such convenient convention? (Are there any known Win32 APIs that uses such convention?)

EDIT:

I Tested the DLL with C#.
Calling SomeFunction1 causes an AV (Attempted to read or write protected memory).
SomeFunction2 works fine.

[DllImport(@"Test.dll")]
[return: MarshalAs(UnmanagedType.BStr)]
static extern string SomeFunction1();

[DllImport(@"Test.dll")]
[return: MarshalAs(UnmanagedType.Bool)]
static extern bool SomeFunction2([MarshalAs(UnmanagedType.BStr)] out string res);

...

string s;
SomeFunction2(out s);
MessageBox.Show(s); // works ok
MessageBox.Show(SomeFunction1()); // fails with AV!

Here is a followup.

Community
  • 1
  • 1
kobik
  • 20,439
  • 4
  • 54
  • 115

1 Answers1

28

A WideString is the same as a BSTR, it's just the Delphi name for it. The memory allocation is handled by the shared COM allocator, CoTaskMemAlloc. Because all parties use the same allocator you can safely allocate in one module and deallocate in another.

So, the reason you don't need to use Sharemem is that the Delphi heap is not being used. Instead the COM heap is used. And that is shared between all modules in a process.

If you look at the Delphi implementation of WideString you will see calls to the following APIs: SysAllocStringLen, SysFreeString and SysReAllocStringLen. These are the system provided BSTR API functions.

Many of the Windows APIs you refer to pre-date the invention of COM. What's more, there are performance benefits to using a fixed length buffer, allocated by the caller. Namely that it can be allocated on the stack rather than a heap. I also can imagine that the Windows designers don't want to force every process to have to link to OleAut32.dll and pay the price of maintaining the COM heap. Remember that when most of the Windows API was designed, the performance characteristics of the typical hardware was very different from now.

Another possible reason for not using BSTR more widely is that the Windows API is targeted at C. And managing the lifetime of BSTR from C is very much more tricky than from higher level languages like C++, C#, Delphi etc.

There is an extra complication however. The Delphi ABI for WideString return values is not compatible with Microsoft tools. You should not use WideString as a return type, instead return it via an out parameter. For more details see Why can a WideString not be used as a function return value for interop?

David Heffernan
  • 572,264
  • 40
  • 974
  • 1,389
  • Thanks for answering. Look [here](http://stackoverflow.com/a/5007216/937125). my understanding is that the DLL should is call `CoTaskMemAlloc`. does Delphi call it when allocation `WideString`? also, when does the `CoTaskMemFree` called? – kobik Feb 17 '12 at 16:22
  • 1
    The `SysXXX` functions make the necessary calls to the COM allocator. – David Heffernan Feb 17 '12 at 16:23
  • The link in your comment merely states the fact that the p/invoke marshaller assumes that a return value of type `string` was allocated with `CoTaskMemAlloc` and so it calls `CoTaskMemFree` when it has finished marshalling. – David Heffernan Feb 17 '12 at 16:25
  • Not sure what is `COM allocator`. where is the COM interface? surely my Dll is not a COM server or ActiveX. – kobik Feb 17 '12 at 16:29
  • The COM allocator is a shared heap that is accessed through `CoTaskMemXXX`, `IMalloc` etc. It is implemented by Ole32.dll – David Heffernan Feb 17 '12 at 16:31
  • Understood :) finally, do you know of any known API that uses such convention? – kobik Feb 17 '12 at 16:38
  • Well, COM uses this convention! That's the very obvious example. – David Heffernan Feb 17 '12 at 16:39
  • @kobik I don't know of any such examples. – David Heffernan Feb 17 '12 at 16:47
  • 8
    @kobik, David, that's an interesting question/discussion... Thanks! – Francesca Feb 17 '12 at 19:01
  • 1
    @kobik: most Win32 API functions DO NOT use `CoTaskMem...()` to manage memory. They use `GlobalAlloc()` or `LocalAlloc()` instead, both of which are local to the calling process and do not need sharing across process boundaries. Documentation for functions usually states how memory is alloated and how it must be freed. – Remy Lebeau Feb 17 '12 at 22:38
  • @Remy Most Win32 functions do not use GlobalAlloc or LocalAlloc. Most Win32 function ask the caller to provide a buffer. – David Heffernan Feb 17 '12 at 22:54
  • @DavidHeffernan: Yes, and many functions allocate their own memory and return it to the caller instead. Usually those functions have counterpart functions to free the memory so the allocation details are hidden from the caller. But there are some functions that use `GlobalAlloc()` or `LocalAlloc()` and require the caller to manually call `GlobalFree()` or `LocalFree()`, respectively. – Remy Lebeau Feb 18 '12 at 00:16
  • @remy Clipboard functions are the prime example. Those type of apis will not want to impose ole dlls on their clients. But if you are creating your own api especially if you are using p/invoke, then BSTR makes coding very simple. – David Heffernan Feb 18 '12 at 09:05
  • @DavidHeffernan: the clipboard uses memory that is based on `GlobalAlloc()`. Look at the documentation for the `SetClipboardData()` function: "If the hMem parameter identifies a memory object, the object must have been allocated using the GlobalAlloc() function with the GMEM_MOVEABLE flag." – Remy Lebeau Feb 18 '12 at 23:04
  • @Remy You are repeating what I just said! I was continuing your thread. I'm not sure what point you are trying to make. – David Heffernan Feb 18 '12 at 23:05
  • @DavidHeffernan: sorry, I had misread your comment about the clipboard as trying to state it did the opposite of what I said about how many APIs do use `Global/Local()`. – Remy Lebeau Feb 19 '12 at 04:32
  • @DavidHeffernan, [your conclusions here](http://stackoverflow.com/a/9328272/937125) made me very confused. – kobik Feb 19 '12 at 12:56
  • @kobik I'm confused too. I think Delphi handles WideString return values incorrectly. I am about to ask a question on that very subject. What I stated above about memory management and the COM heap is quite correct. – David Heffernan Feb 19 '12 at 13:01
  • @DavidHeffernan, but it works with my D5. Have you tested it with XE2? – kobik Feb 19 '12 at 13:09
  • @kobik I'm running with XE2 yes. Perhaps this is a regression. – David Heffernan Feb 19 '12 at 13:12
  • @DavidHeffernan, tested with C#. Same results for me. when using `var` or `out` it works. returning WideString as function result fails. – kobik Feb 19 '12 at 14:30
  • @kobik I think return value should fail in D5 too. I guess the return value will fail for all callers apart from delphi callers. See my question: http://stackoverflow.com/questions/9349530/why-can-a-widestring-not-be-used-as-a-function-return-value-for-interop. – David Heffernan Feb 19 '12 at 14:36