3

Doing code review at work and came across a use of template types I've not seen before. Upon first glance it looked like the code shouldn't compile as the definition seemed recursive. I've boiled it down to the most simple verifiable example:

interface Bar<T>
interface Foo<T: Bar<T>> // Surely this is recursive?

My understanding of how template types work is:

interface Foo<T> - a Foo of T, no constraints

interface Foo<T : Bar> - a Foo of T, where T is constrained to a Bar

Assuming what I said above is true, then this doesn't make sense to me:

interface Bar<T> - a Bar of T, no constraint on T

interface Foo<T: Bar<T>> - a Foo of T, where T is constrained to a Bar<T>

Uh oh, how can T be defined in terms of Bar<T>?.

We know T is a Bar<T>, so if we substitute the T in Bar<T>, well it's a Bar<Bar<T>>.

we still haven't resolved T for Bar... For sake of argument, let's substitute T again. Now we have T being a Bar<Bar<Bar<T>>>. Surely this goes into infinity no?

Thomas Cook
  • 2,861
  • 1
  • 13
  • 33

2 Answers2

5

CRTP (recursively bounded quantification) is a well-known design idiom which is often used (among other things) to provide generic code with some sort of a "self" type.

Here's a practical example of recursive generics.

Say you have a function that operates on a set of comparable values.

fun <T> findMax(collection: Collection<T>): T?

Ideally, we would like to constrain this function to only operate on collections of Comparable values:

fun <T> findMax(collection: Collection<Comparable<T>>): Comparable<T>?

And that's all. Right?

While this will work, you'd need a cast on the return value for it to do anything useful, since it returns a Comparable<T> rather than a T.

Now let's say we try:

fun <T : Comparable<T>> findMax(collection: Collection<T>): T?

Much better. This ensures:

  • T is Comparable
  • and more importantly, T is comparable to itself

The same applies to classes and inheritance.

interface SelfReturner<T : SelfReturner<T>> {
    fun getSelf(): T
}

class A : SelfReturner<A> {
    override fun getSelf(): A // <--
}

This works fine thanks to return type covariance, because A is a SelfReturner<A>.

This is commonly used to allow a class to "know" its own type, though it's important to keep in mind that it is not foolproof:

class Impostor : SelfReturner<A> {
    override fun getSelf(): A // <-- oops!
}

While you're right about the apparent recursiveness of these generics, because one could indeed instead write

fun <T : Comparable<Comparable<Comparable<...>>>> findMax(collection: Collection<T>): T?

this doesn't go on forever because the condition is generally satisfied after a single level of recursion (say we use String, for example. It is a Comparable<String>, and that's all the compiler needs to check.)

Note that unlike e.g. C++, Kotlin does not use templates. Generic type information is only used by the compiler to ensure code correctness, and is not preserved* (see type erasure) in the compiled code.

Whereas template instantiation will result in the creation of a new and completely separate type, generic types are all erased to the same (non-generic) class at runtime.

* This isn't completely true; some generic type information is available to reflection, which is why type tokens work, but it's only available in limited circumstances.


Fun fact: Wikipedia claims that this was discovered by accident,

by Jan Falkin, who accidentally derived a base class from a derived class

so it appears to have been just as confusing even to those who came up with the concept.

Yes, there's no citation, but let's not ruin the magic. :)

Moira
  • 9,571
  • 3
  • 40
  • 61
  • Hmm it's a good answer and after reading a bit about CRTP I believe a correct one. But I'm not gonna lie, still a bit confused and haven't fully wrapped my head around how it actually compiles. – Thomas Cook May 12 '20 at 10:15
  • Oh, so the compiler does recurse to satisfy the constraints. Does this means it's possible to break compilation using this method if you nest this technique too deeply? – Thomas Cook May 12 '20 at 10:16
  • @ThomasCook I'm not aware of any practical case where this would happen, but I recall that in C# one can easily [create a compiler bomb](https://codegolf.stackexchange.com/a/69200/64109) with deeply-nested generics. There has also been [at least one javac bug](https://bugs.openjdk.java.net/browse/JDK-8218030) related to this. – Moira May 12 '20 at 10:18
  • Thanks Moira, appreciate the time. Off to go and wrap my head around this more now :-) – Thomas Cook May 12 '20 at 10:18
  • 1
    @ThomasCook I've added a small toy example with inheritance, hope it helps a bit more :) – Moira May 12 '20 at 10:39
  • @ThomasCook: The crux is that `Foo` can be compiled without knowing anything about `T` other than it inherits from `Bar`. The fact that `Bar` is generic doesn't affect `Foo` at all. Similarly, `Bar` can be compiled without knowing anything about `T` at all. – Mooing Duck Nov 20 '20 at 21:33
1

Recursive Type Bound

The pattern you are referring to is called a recursive type bound in the JVM world. In generics, when a reference type has a type parameter that is bounded by the reference type itself, then that type parameter is said to have a recursive type bound.

For example, in the generic type Fruit<T extends Fruit<T>>, Fruit is the reference type, its type parameter T is bounded by the Fruit itself, so, the type parameter T has a recursive type bound Fruit<T>.

Let's solve a simple problem to understand this concept step by step.


Problem

Assume that we have to sort the fruits by their sizes. And we are told that we can only compare fruits of the same types. For example, we can't compare apples with oranges (pun intended).

So, we create a simple type hierarchy like following,

Fruit.kt

interface Fruit {
    val size: Int
}

Apple.kt

class Apple(override val size: Int) : Fruit, Comparable<Apple> {

    override operator fun compareTo(other: Apple): Int {
        return size.compareTo(other.size)
    }
}

Orange.kt

class Orange(override val size: Int) : Fruit, Comparable<Orange> {

    override operator fun compareTo(other: Orange): Int {
        return size.compareTo(other.size)
    }
}

Test

fun main() {
    val apple1 = Apple(1)
    val apple2 = Apple(2)
    println(apple1 > apple2)   // No error

    val orange1 = Orange(1)
    val orange2 = Orange(2)
    println(orange1 < orange2) // No error

    println(apple1 < orange1)  // Error: different types
}

Solution

In this code, we are able to achieve our objective of being able to compare the same types, that is, apples with apples and oranges with oranges. When we compare an apple with an orange we get an error which is what we want.

Problem

The problem here is that the code for implementing the compareTo() method is duplicated for Apple and Orange class. And will be duplicated more in all the classes that we extend from the Fruit when we create new fruits in the future. The amount of repeated code in our example is less but in the real world, the repeated code can be of hundreds of lines in each class.


Moving the Repeated Code to Interface

Fruit.kt

interface Fruit : Comparable<Fruit> {
    val size: Int
    override operator fun compareTo(other: Fruit): Int {
        return size.compareTo(other.size)
    }
}

Apple.kt

class Apple(override val size: Int) : Fruit

Orange.kt

class Orange(override val size: Int) : Fruit

Solution

In this step, we get rid of the repeated code of compareTo() method by moving it to the interface. Our extended classes Apple and Orange are no longer polluted with common code.

Problem

Now the problem is that we are now able to compare different types, comparing apples to oranges no longer gives us an error:

println(apple1 < orange1)    // No error

Introducing a Type Parameter

Fruit.kt

interface Fruit<T> : Comparable<T> {
    val size: Int
    override operator fun compareTo(other: T): Int {
        return size.compareTo(other.size) // Error: size not available.
    }
}

Apple.kt

class Apple(override val size: Int) : Fruit<Apple>

Orange.kt

class Orange(override val size: Int) : Fruit<Orange>

Solution

To restrict the comparison of different types, we introduce a type parameter T. So that the comparable Fruit<Apple> cannot be compared to comparable Fruit<Orange>. Note our Apple and Orange classes; they now inherit from the types Fruit<Apple> and Fruit<Orange> respectively. Now if we try to compare different types, the IDE shows an error, our desired behaviour:

println(apple1 < orange1)  // Error: different types

Problem

But in this step, our Fruit class doesn't compile. The size property of T is unknown to the compiler. This is because the type parameter T of our Fruit class doesn't have any bound. So, the T could be any class, it is not possible that every class in the world would have a size property. So the compiler is right in not recognizing the size property of T.


Introducing a Recursive Type Bound

Fruit.kt

interface Fruit<T : Fruit<T>> : Comparable<T> {
    val size: Int
    override operator fun compareTo(other: T): Int {
        return size.compareTo(other.size)
    }
}

Apple.kt

class Apple(override val size: Int) : Fruit<Apple>

Orange.kt

class Orange(override val size: Int) : Fruit<Orange>

Final Solution

So, we tell the compiler that our T is a subtype of Fruit. In other words, we specify the upper bound T extends Fruit<T>. This makes sure that only subtypes of Fruit are allowed as type arguments. Now the compiler knows that the size property can be found in the subtype of Fruit class (Apple, Orange etc.) because the Comparable<T> also receives our type(Fruit<T>) that contains the size property.

This allows us to get rid of the repeated code of compareTo() method and also allows us to compare the fruits of the same types, apples with apples and oranges with oranges.


More About Recursive Type Bounds

A recursive type is one that includes a function that uses that type itself as a type for some argument or its return value. In our example, compareTo(other: T) is the function of the recursive type that takes the same recursive type as an argument.

Caveat

The caveat in this pattern is that the compiler doesn’t prevent us from creating a class with a type argument of other subtype:

class Orange(override val size: Int) : Fruit<Orange>

class Apple(override val size: Int) : Fruit<Orange>    // No error

Note in the Apple class above, by mistake we passed Orange instead of Apple itself as a type argument. This results in the compareTo(other: T) method to take Orange instead of Apple.

Now we no longer get error while comparing different types and suddenly can't compare apples with apples:

println(apple1 < orange1)    // No error
println(apple1 > apple2)     // Error

So, the developer needs to be careful while extending the classes.


Infinite Recursion Not Possible

The declaration Fruit<T extends Fruit<T>> makes sure that only the subtypes of type Fruit<T> are allowed by the compiler. Fruit<Fruit<T>> or Fruit<Fruit<Fruit<T>>> and so on are not the subtypes of Fruit<T>, in other words, they are not within bound.

For example, if we use the declaration in the following manner:

class Orange(override val size: Int) : Fruit<Fruit<Orange>>

The compiler will give error: Type argument is not within its bound

There is no imaginable use case for Fruit<Fruit>, so the compiler doesn't allow that either. Only the first level is allowed, that is, Fruit<Apple>, Fruit<Orange> etc.

These two things together prevent the infinite recursion.


That's it! Hope that helps.

Yogesh Umesh Vaity
  • 16,749
  • 10
  • 82
  • 69