Spring JPA/Hibernate Repository findAll is doing N+1 requests instead of a JOIN by default in Kotlin

Question

I am working in a Spring JPA/Hibernate application with Kotlin and I want to find all elements in an entity.

That entity has a foreign key with a @ManyToOne relationship. I want to get all elements with their associated values with a JOIN query avoiding the N+1 problem.

One thing is that the foreign keys are not related to the primary keys, but to another unique field in the entities (UUID).

I was able to make that query with a JOIN creating a custom Query with a JOIN FETCH, but my point is to avoid creating those queries and make those JOINS in all findAlls by default.

Is that possible or do I have to make a query in JPQL manually to force the JOIN FETCH?

Here is the example code:

    @Entity
    data class A {
      @Id
      val id: Long,
    
      @Column
      val uuid: UUID,
    
      @Column
      val name: String
    }

    @Entity
    data class B {
      @Id
      val id: Long,
    
      ...
    
      @Fetch(FetchMode.JOIN)
      @ManyToOne
      @JoinColumn(name = "a_uuid", referencedColumnName = "uuid", insertable = false, updatable = false)
      val a: A
    }

    @Repository
    interface Repo<B> : CrudRepository<B, Long>

   ...
   repo.findAll() // <-- This triggers N+1 queries instead of making a JOIN
   ...

What happens if you annotate your entity A with @BatchSize(size = 1000) or whatever size you feel to be appropriate? Does that change the behavior at all? I would also suggest changing the fetch to be lazy @ManyToOne(fetch = javax.persistence.FetchType.LAZY). This paired together with the BatchSize on the referenced entity would result in 2 queries (or more precisely -> 1 + N/1000 queries) — Nathan, Jan 20 '21 at 11:46

Josh · Accepted Answer · 2021-01-21T10:07:17.740

Another option for you is using EntityGraph. It allows defining a template by grouping the related persistence fields which we want to retrieve and lets us choose the graph type at runtime.

This is an example code that is made by modifying your code.


@Entity
data class A (
    @Id
    val id: Long,

    @Column
    val uuid: UUID,

    @Column
    val name: String
) : Serializable

@NamedEntityGraph(
    name = "b_with_all_associations",
    includeAllAttributes = true
)
@Entity
data class B (
    @Id
    val id: Long,

    @ManyToOne
    @JoinColumn(name = "a_uuid", referencedColumnName = "uuid")
    val a: A
)

@Repository
interface ARepo: CrudRepository<A, Long>

@Repository
interface BRepo: CrudRepository<B, Long> {
    @EntityGraph(value = "b_with_all_associations", type = EntityGraph.EntityGraphType.FETCH)
    override fun findAll(): List<B>
}

@Service
class Main(
    private val aRepo: ARepo,
    private val bRepo: BRepo
) : CommandLineRunner {
    override fun run(vararg args: String?) {
        (1..3L).forEach {
            val a = aRepo.save(A(id = it, uuid = UUID.randomUUID(), name = "Name-$it"))
            bRepo.save(B(id = it + 100, a = a))
        }

        println("===============================================")
        println("===============================================")
        println("===============================================")
        println("===============================================")

        bRepo.findAll()
    }
}

On B entity, an entity graph named "b_with_all_associations" is defined, and it is applied to the findAll method of the repository of B entity with LOAD type.

These things will prevent your N+1 problem by fetching with join.

Here is the SQL log for the bRepo.findAll().

    select
        b0_.id as id1_1_0_,
        a1_.id as id1_0_1_,
        b0_.a_uuid as a_uuid2_1_0_,
        a1_.name as name2_0_1_,
        a1_.uuid as uuid3_0_1_ 
    from
        b b0_ 
    left outer join
        a a1_ 
            on b0_.a_uuid=a1_.uuid

ps1. due to this issue, I don't recommend using many to one relationship with non-pk. It forces us to use java.io.Serializable to 'One' entity.

ps2. EntityGraph can be a good answer to your question when you want to solve the N+1 problem with Join. But I would recommend the better solution: try to solve it with Lazy loading.

ps3. It's not a good idea that using non-pk associations for Hibernate. I truly agree on this comment. I think it's a bug that is not solved yet. It breaks the lazy loading mechanism of hibernate.

score 0 · Answer 2 · answered Jan 14 '21 at 07:41

As far as I know, the fetch mode only applies to EntityManager.find related queries or when doing lazy loading but never when executing HQL queries, which is what is happening behind the scenes. If you want this to be join fetched, you will have to use an entity graph, which is IMO also better as you can define it per use-site, rather than globally.

score 0 · Answer 3 · answered Jan 20 '21 at 11:52

I don't know how to configure exactly what you are asking, but the following suggestion might be worth considering...

Change

  @Fetch(FetchMode.JOIN)
  @ManyToOne
  @JoinColumn(name = "a_uuid", referencedColumnName = "uuid", insertable = false, updatable = false)
  val a: A

to

  @ManyToOne(fetch = javax.persistence.FetchType.LAZY)
  @JoinColumn(name = "a_uuid", referencedColumnName = "uuid", insertable = false, updatable = false)
  val a: A

And then on your entity A, add the annotation to the class

  @BatchSize(size = 1000)

Or whatever batch-size you feel to be appropriate.

This will generally give you the results in 2 queries if you have less than 1000 results. It will load a proxy for A rather than joining to A, but then the first time that A is accessed, it will populate the proxies for BATCH_SIZE number of entities.

It reduces the number of queries from

N + 1

to

1 + round_up(N / BATCH_SIZE)

score 0 · Answer 4 · answered Jan 20 '21 at 17:36

The findAll implementation will always load b first and then resolve it's dependencies checking the annotations. If you want to avoid the N+1 problem you can add the @Query annotation with JPQL query:

   ...
   @Query("select b from TableB b left join fetch b.a")
   repo.findAll() 
   ...

Spring JPA/Hibernate Repository findAll is doing N+1 requests instead of a JOIN by default in Kotlin

4 Answers4