15

I'm trying to build an custom comparer which allows the assignment of the comparison function to an internal field. In order to ease the creation of the comparer, I tried to add a constructor-like class function Construct which initializes the comparer.

Now if I try to compile the following example, the compiler displays

[dcc32 Fehler] ConsoleDemo1.dpr(37): E2555 Symbol 'Result' cannot be tracked

I have the following example-code:

program ConsoleDemo1;

{$APPTYPE CONSOLE}
{$R *.res}

uses
  Generics.Collections, Generics.Defaults,
  System.SysUtils;

type

  TConstFunc<T1, T2, TResult> = reference to function(const Arg1: T1; const Arg2: T2): TResult;

  TDemo = class(TComparer<string>)
  private
    FVar: TConstFunc<string, string, Integer>;
    function CompareInternal(const L, R: string): Integer;
  public
    class function Construct(): TDemo;
    function Compare(const L, R: string): Integer; override;
  end;

function TDemo.Compare(const L, R: string): Integer;
begin
  Result := FVar(L, R);
end;

function TDemo.CompareInternal(const L, R: string): Integer;
begin
  Result := AnsiCompareStr(L, R);
end;

class function TDemo.Construct: TDemo;
begin
  Result := TDemo.Create();
  Result.FVar := Result.CompareInternal;
end;

end.
David Heffernan
  • 572,264
  • 40
  • 974
  • 1,389
ventiseis
  • 2,760
  • 10
  • 31
  • 42
  • I suppose this is because of some compiler magic. If I use `TConstFunc = function(const Arg1: T1; const Arg2: T2): TResult of object;`, the code compiles and works. – ventiseis Jan 08 '16 at 12:53

3 Answers3

12

I don't think that this is a bug. Critically, you've defined TConstFunc as an anonymous method type. These are managed, reference counted, very special types that are quite different from regular object methods. By compiler magic they are usually assignment compatible, but with several important caveats. Consider the more concise :

program Project1;

{$APPTYPE CONSOLE}

type
  TFoo = reference to procedure;

  TDemo = class
  private
    FFoo : TFoo;
    procedure Foo;
  public
    class function Construct(): TDemo;
  end;

procedure TDemo.Foo;
begin
  WriteLn('foo');
end;

class function TDemo.Construct: TDemo;
begin
  result := TDemo.Create();
  result.FFoo := result.foo;
end;

end.

This also produces the same compiler error (E2555). Because the member method is a procedure of object (object method) type, and you are assigning it to a reference to procedure (anonymous method) type, this is equivalent to (and I suspect that the compiler is expanding this as) :

class function TDemo.Construct: TDemo;
begin
  result := TDemo.Create();
  result.FFoo := procedure
                 begin
                   result.foo;
                 end;
end;

The compiler cannot assign the method reference directly (since they are of different types), and therefore (I guess) has to wrap it in an anonymous method which is implicitly requiring capture of the result variable. Function return values cannot be captured by anonymous methods, however - only local variables can.

In your case (or, indeed, for any function type), the equivalent cannot even be expressed due to the anonymous wrapper hiding the result variable, but we can imagine the same in theory as:

class function TDemo.Construct: TDemo;
begin
  Result := TDemo.Create();
  Result.FVar := function(const L, R : string) : integer
                 begin
                   result := result.CompareInternal(L,R);  // ** can't do this
                 end;
end;

As David has shown, introducing a local variable (which can be captured) is one correct solution. Alternatively, if you don't need the TConstFunc type to be anonymous, you can simply declare it as a regular object method :

TConstFunc<T1, T2, TResult> = function(const Arg1: T1; const Arg2: T2): TResult of object;


Another example where attempting to capture result fails :

program Project1;

{$APPTYPE CONSOLE}

type
  TBar = reference to procedure;
  TDemo = class
  private
    FFoo : Integer;
    FBar : TBar;
  public
    class function Construct(): TDemo;
  end;

class function TDemo.Construct: TDemo;
begin
  result := TDemo.Create();
  result.FFoo := 1;
  result.FBar := procedure
                 begin
                   WriteLn(result.FFoo);
                 end;
end;

end.

The fundamental reason why this does not work is because a method's return value is effectively a var parameter and the anonymous closure captures variables, not values. This is a critical point. Similarly, this is also not allowed :

program Project1;

{$APPTYPE CONSOLE}

type
  TFoo = reference to procedure;

  TDemo = class
  private
    FFoo : TFoo;
    procedure Bar(var x : integer);
  end;

procedure TDemo.Bar(var x: Integer);
begin
  FFoo := procedure
          begin
            WriteLn(x);
          end;
end;

begin
end.

[dcc32 Error] Project1.dpr(18): E2555 Cannot capture symbol 'x'

In the case of a reference type, as in the original example, you really are only interested in capturing the value of the reference and not the variable that contains it. This does not make it syntactically equivalent and it would not be proper for the compiler to create a new variable for you for this purpose.

We could rewrite the above as this, introducing a variable :

procedure TDemo.Bar(var x: Integer);
var
  y : integer;
begin
  y := x;
  FFoo := procedure
          begin
            WriteLn(y);
          end;
end;

And this is allowed, but the expected behaviour would be very different. In the case of capturing x (not allowed), we would expect that FFoo would always write the current value of whatever variable was passed in as argument x to Bar, regardless of where or when it may have been changed in the interim. We would also expect that the closure would keep the variable alive even after it fell out of whatever scope created it.

In the latter case, however, we expect FFoo to output the value of y, which is the value of the variable x as it was the last time Bar was called.


Returning to the first example, consider this :

program Project1;    
{$APPTYPE CONSOLE}    
type
  TFoo = reference to procedure;    
  TDemo = class
  private
    FFoo : TFoo;
    FBar : string;
    procedure Foo;
  public
    class function Construct(): TDemo;
  end;

procedure TDemo.Foo;
begin
  WriteLn('foo' + FBar);
end;

class function TDemo.Construct: TDemo;
var
  LDemo : TDemo;
begin
  result := TDemo.Create();
  LDemo := result;
  LDemo.FBar := 'bar';
  result.FFoo := LDemo.foo;
  LDemo := nil;
  result.FFoo();  // **access violation
end;

var
 LDemo:TDemo;
begin
  LDemo := TDemo.Construct;
end.

Here it is clear with :

result.FFoo := LDemo.foo;

that we have not assigned a normal reference to the method foo beloning to the instance of TDemo stored in LDemo, but have actually captured the variable LDemo itself, not the value it contained at the time. Setting LDemo to nil afterwards naturally produces an access violation, even thought the object instance it referred to when the assignment was made is still alive.

This is radically different behaviour than if we simply defined TFoo as a procedure of object instead of a reference to procedure. Had we done that instead, the above code works as one might naively expect (output foobar to the console).

J...
  • 28,957
  • 5
  • 61
  • 128
  • I think your analysis of how the compiler wraps the method call as an anon method is spot on. However, if the compiler can work out that it needs to do that, then it should be able to work out that it needs to introduce a variable which can be captured, in my opinion. – David Heffernan Jan 08 '16 at 16:42
  • @DavidHeffernan One would think so, I agree. There may be something subtle going on, though. I have to assume there is a good reason why function results cannot be captured (generally). Introducing a variable in this way might not always be an approach without side effects... I'm not sure. – J... Jan 08 '16 at 17:06
  • The reason return values cannot be captured is that they are not declared in the function. They are implemented as var parameters, passed after all other parameters. – David Heffernan Jan 08 '16 at 17:20
  • @DavidHeffernan Yes, introducing the variable works for reference types because you only really care about capturing the value of the reference, not the variable that holds it. For value types it becomes nonsensical. – J... Jan 08 '16 at 17:39
  • @DavidHeffernan I've expanded my answer. Do you still think this is a compiler defect? – J... Jan 08 '16 at 18:20
  • 2
    I do. I think you've explained the implementation details behind the limitation, which you did very well. You deserve more than my sole up vote. However, this is an implementation detail. It's a method of object. There's no ambiguity. There should be no variable capture. The compiler should just do this. – David Heffernan Jan 08 '16 at 18:31
  • @DavidHeffernan There has to be variable capture - an anonymous method has no `self`. In the first solution you are capturing a local variable, in the second solution you are capturing the hidden `self` in the constructor. The object reference has to be stored somewhere, and in a way that is consistent with our expectation of how closures operate. I'm still not convinced this is as simple a problem to solve as it may seem. – J... Jan 08 '16 at 18:54
  • There doesn't need to be variable capture. We want to call a specific method on a specific instance. It's only an implementation choice of the compiler designers that Lead to the need to capture a variable. – David Heffernan Jan 08 '16 at 19:50
  • Fundamentally I don't see any need for this to be an anonymous method with variable capture. I can't see why you cannot assign a procedural type or a method of object to a reference procedure. The designers could have made this possible. To me variable capture should be something that happens in an explicit anonymous method only. I think it is a little disingenuous for the compiler to make an anonymous method behind the scenes here. It's a very interesting issue though! – David Heffernan Jan 08 '16 at 20:00
  • @J... This very detailed is an explanation for the error message and the proposed solution. Upvote! – ventiseis Jan 08 '16 at 20:10
  • @J... But: I don't expect this behaviour as an end user. If method references and function/procedure references are different things, I expect an error message which describes the type incompatibility. What I do not expect is an automagically inserted anonymous function stub. – ventiseis Jan 08 '16 at 20:13
  • @DavidHeffernan But `reference to procedure` is precisely the type of an anonymous method. They are one and the same - by declaring it as such you are explicitly telling the compiler that you want anonymous method semantics. If you want `procedure of object` semantics then that type exists explicitly for that purpose. – J... Jan 08 '16 at 20:17
  • @J... Well, that's one way to implement it. But not a very helpful one as we have found out. There's a huge benefit of the flexibility of allowing anything "function like" to be assigned to a ref function. But I really don't see why a plain function, or a function of object should be *promoted* to an anon function with capture. I think you are getting tied down in the implementation details. When designing functionality you should design it from the users' perspective, I'm sure you would agree. – David Heffernan Jan 08 '16 at 20:20
  • @DavidHeffernan I can't see it working in any sensible way without some additional explicit qualification - perhaps a new type `reference to procedure of object`, perhaps? A method with anonymous/closure semantics and behaviour but *with* a definite `self` parameter. This still doesn't solve the "no capturing var parameters" problem (which I think this is a halting problem? - unsolvable, generally), but it would allow the type of assignment you feel should be possible. – J... Jan 08 '16 at 20:27
  • @J... It would work perfectly well. An anonymous method would be the only thing that would ever invoke variable capture. No new types are needed. In my eyes a reference procedure is something that can hold an anonymous method, a plain procedure, or a method. One type that can hold three different things. Only the anonymous method would ever require capture. The implementation would have to change. – David Heffernan Jan 08 '16 at 20:49
  • @DavidHeffernan In response to your comment "There doesn't need to be a variable capture...". There has to be a variable capture. You are calling a method on an instance. What if that method was accessing class variables in that instance? It's not just the method itself that's captured but the specific instance. – Graymatter Jan 08 '16 at 21:56
  • 1
    @Graymatter No. There's no need for **variable** capture. When a method is assigned to a ref proc, the assignment could be the **value** instance & code. Just as for an `of object` method type. The designers chose a different way, but it is not the only way. In the first paragraph of this answer, mention is made of *magic*. I am suggesting that different magic should be used. Clearly it's too late for this. Any bug report will be closed as *as designed*. I just think the design choice was poor. – David Heffernan Jan 08 '16 at 21:58
  • @DavidHeffernan I agree that they could definitely have implemented it differently but in the present implementation as described in http://docwiki.embarcadero.com/RADStudio/XE6/en/Anonymous_Methods_in_Delphi I can't see it being a bug. I was interpreting what you were saying here as saying it was a bug where I would call it more of a design limitation. I can't see how `Result` could be anything other than a variable. I think it's working as designed, it's just that that design could be better. You updated your comment while I was typing mine. It looks like we agree. – Graymatter Jan 08 '16 at 22:05
  • @DavidHeffernan I agree, it *could* work perfectly well and I would love to have such a chameleon method type. As it stands, the only state an anonymous method, in its current form, can capture is in the form of variables. Given that, I think we can call this a *deficiency* rather than a *defect*. It is working exactly as the rules outline that it should, it just doesn't quite work the way we expect it should be able to (or would like it to). – J... Jan 08 '16 at 22:06
  • After reading all that helpful and interessting comments and answers, I think I can now understand why my original example does not work. As a note: I won't create a bug report for that because this is a rare case and there are surely clearer ways to construct such comparers / objects. An obvious way would be to implement the different behaviours in different classes or let the caller configure the comparer with some kind of flag / switch. – ventiseis Jan 09 '16 at 13:59
11

The compiler error on my English Delphi reads:

[dcc32 Error] E2555 Cannot capture symbol 'Result'

This is due to a defective design. There's no reason for any variable capture to be taking place here at all. The right hand side of the assignment is an instance method rather than an anonymous method. But the compiler handles that by wrapping the method in an anonymous method. The compiler translates

Result.FVar := Result.CompareInternal;

to

Result.FVar := 
  function(const Arg1, Arg2: string): Integer
  begin
    InnerResult := OuterResult.CompareInternal(Arg1, Arg2);
  end;

Leaving aside the confusion over the two separate result variables, the compiler rejects this because the outer result variable is not a local, it's a var parameter. And so cannot be captured.

But the whole design is wrong in my view. There's no need for any variable capture. When you write Result.CompareInternal you intend to refer to a normal of object method. With a better design, the compiler would allow this assignment without creating an anonymous method.

You can work around the problem like this:

class function TDemo.Construct: TDemo;
var
  Demo: TDemo;
begin
  Demo := TDemo.Create();
  Demo.FVar := Demo.CompareInternal;
  Result := Demo;
end;

Here the local variable Demo can be captured.

Or as I would suggest, like this:

program ConsoleDemo1;

{$APPTYPE CONSOLE}

uses
  Generics.Defaults,
  System.SysUtils;

type
  TConstFunc<T1, T2, TResult> = reference to function(const Arg1: T1; 
    const Arg2: T2): TResult;

  TDemo = class(TComparer<string>)
  private
    FVar: TConstFunc<string, string, Integer>;
    function CompareInternal(const L, R: string): Integer;
  public
    constructor Create;
    function Compare(const L, R: string): Integer; override;
  end;

constructor TDemo.Create;
begin
  inherited;
  FVar := CompareInternal;
end;

function TDemo.Compare(const L, R: string): Integer;
begin
  Result := FVar(L, R);
end;

function TDemo.CompareInternal(const L, R: string): Integer;
begin
  Result := AnsiCompareStr(L, R);
end;

end.
David Heffernan
  • 572,264
  • 40
  • 974
  • 1,389
  • 4
    It's fine. By the way, awesome MCVE. Automatic up vote from me for an MCVE like this. If only all questions were posed like this!! – David Heffernan Jan 08 '16 at 12:56
  • I wrote that to understand the behaviour myself. If i compile my original project, I wait two minutes and have 2GB of ram filled... – ventiseis Jan 08 '16 at 13:00
  • Oh, music to my ears!! If only everyone thought this way then so many people would be able to solve far more problems by themselves! Which is better for everyone. – David Heffernan Jan 08 '16 at 13:01
  • 2
    FWIW I've solved like this on XE4: `class function TDemo.Construct: TDemo; begin Result := TDemo.Create; Result.AssignInternalComparer; end;` and `procedure TDemo.AssignInternalComparer; begin FVar := CompareInternal; end;` – fantaghirocco Jan 08 '16 at 13:10
  • @fantaghirocco yes, another way to workaround compiler limitation, and if you would make that new function `inline` ... But `Construct` function steel is error-prone :-D – Arioch 'The Jan 09 '16 at 13:14
  • @Arioch'The I agree: the whole construction is strange and can provoke errors. But it was the first idea I had when I replaced _two parameterless constructors with different names_ (not `Constructor`) which initialized `FVar` differently. – ventiseis Jan 09 '16 at 23:05
2

This is not a full fledged answer, rather notes to David's answer and to the topicstarter's question.

Using answer mode for posting source snippets.

class function TDemo.Construct: TDemo;
begin
  Result := TDemo.Create();
  Result.FVar := Result.CompareInternal;
end;

class function TDemo.Construct: TDemo;
var
  Demo: TDemo;
begin
  Demo := TDemo.Create();
  Demo.FVar := Demo.CompareInternal;
  Result := Demo;
end;

Those both snippets use the same template:

  1. Create an object ( and memory management responsibilities attached to it )
  2. Tune and adjust the object
  3. Pass the object to outer world ( and m/m responsibilities with it )

Sure, the p.2 here is just one single line, still

  1. It has a function call, which might be error prone. Twice so if function would be virtual overrode by inheriting subclasses.
  2. Patterns are to work not in the easiest situations, but rather in hardest ones.

So I think we should assume that p.2 has risk of runtime error, risk of exception thrown. Then it is a textbook memory leak. The local function still holds the memory management responsibilities, since it did not passed the result outside. But it also does not fulfill the required cleanup.

From my perspective the correct pattern - and the one giving one more incentive to using a dedicated local variable than mere Result/Result compiler confusion - should be

class function TDemo.Construct: TDemo;
var
  Demo: TDemo;
begin

  Demo := TDemo.Create();  // stage 1: creating an object
  try                      // stage 1: accepting M/M responsibilities

     Demo.FVar := Demo.CompareInternal; // stage 2: tuning and facing
     // Demo.xxx := yyy;                //   ...potential risks of exceptions
     // Demo.Connect(zzz);  etc

     Result := Demo;   // stage 3: passing the object outside
     Demo := nil;      // stage 3: abandoning M/M responsibilities
     //  function exit should follow this line immediately, without other fault-risky statements
  finally
    Demo.Free;         // proceeding with M/M in case of faults in stage 2
  end;
end;                   // stage 3: passing the object outside - immediately after the assignments!

UPD: ventiseis: And as a side node: I would try to instantiate the configurated comparer TDemo only once. The comparison function should be a stateless function

  TDemo = class(TComparer<string>)
  private
    class var FVar: TConstFunc<string, string, Integer>;
   // function CompareInternal(const L, R: string): Integer; STATIC; // also possible
    class constructor InitComp;
  ...
  end;

  // would only be called once, if the class is actually used somewhere in the project
  class constructor TDemo.InitComp; 
  begin
    FVar := function(const L, R: string): Integer
    begin
      Result := StrToInt(R) - StrToInt(L)
    end 
  end;
Arioch 'The
  • 15,005
  • 31
  • 59
  • I agree that if any exceptions are raised, a memory leak would be created. I never came up with this kind try/finally pattern myself - basically that's a nice idea, but I think there is some danger to forget `Demo := nil; `.Why not use `try .. except on oE: FreeAndNil(Demo); raise oE; end;`? – ventiseis Jan 09 '16 at 13:40
  • And as a side node: I would try to instantiate the configurated comparer `TDemo` only once. The comparison function should be a stateless function, working with any list or array, so it seems of no benefit to create and destroy comparers. – ventiseis Jan 09 '16 at 13:46
  • 1
    @ventiseis that may be written simpler, `...except Demo.Free; raise; end`. Well, the answer would be literally the same: "there is some danger" of forgetting to re-raise the exception. *Also you forgot begin-end :)* So - choose your poison, they are similar. Also my snippet provide function to one extra option (for example in Find-Or-Nil kind of functions, like `TDataSet.FindField`) to delete temporary object and return `nil` without raising exceptions. – Arioch 'The Jan 09 '16 at 17:11
  • @ventiseis then `TDemo.FVar` maybe should be a `class variable` assigned in the `class constructor` – Arioch 'The Jan 09 '16 at 17:12
  • You're right - your solution is elegant and works nicely in case of exceptions. – ventiseis Jan 09 '16 at 22:56
  • @ventiseis Personally I think the main weakness for this pattern is not in "forget to nil Demo" - templates are for copy-pasting - but in the risk later to add extra lines of (potentially faulty) code between nilling the local var and finally, or between finally and the very end of function. So this pattern only suits for rather short strictly single-purpose functions, where once your tuning of the object is complete - there would be no any extra activity ever. Would extra code start silently creeping in, it would perhaps one day break the basic assumptions of this template. – Arioch 'The Jan 11 '16 at 15:17
  • `I would try to instantiate the configurated comparer TDemo only once` - Then you would not have to "instantiate" it perhaps? Just make a virtual function, why bother with extra variables? – Arioch 'The Jan 11 '16 at 15:21
  • I got that idea from C#:` [StringComparer](https://msdn.microsoft.com/en-us/library/system.stringcomparer.ordinalignorecase%28v=vs.110%29.aspx) provides ready-to-use-comparers for common use cases. – ventiseis Jan 12 '16 at 07:03
  • 1
    @ventiseis it return DIFFERENT CLASSES, just like Delphi's `TEncoding` which was designed after .Net as well ( and that was not correct IMHO as Delphi native is not GC-based! `TEncoding` should had been an interface, not a class!) So just like with `TEncoding` you are to make `function TDemo.Compare(const L, R: string): Integer; VIRTUAL; ABSTRACT;`, then you have to make a bunch of classes `TDemo1, TDemo2, TDemo3` with different `function TDemoNNN.Compare(const L, R: string): Integer; override;` implementations. And then return different CLASSES like your DotNet's example does. – Arioch 'The Jan 12 '16 at 14:38
  • it seems to me you fallen for "Describe the goal, not the step" from http://www.catb.org/esr/faqs/smart-questions.html#goal – Arioch 'The Jan 12 '16 at 14:41
  • You are right! It seems that I've always overseen that `StringComparer` is an `abstract class` defining the compare method `virtual`. But anyway, the original example wasn't a single instance anyway. After doing some more research, the comparison was used _only in two places_ for sorting a list. I'm wondering why the original developer didn't use `TComparer.Construct(..)`. – ventiseis Jan 12 '16 at 20:09