3

I would like to know what are the different kinds of IPIs available for x86_64 in Linux. In particular, I want to find out the different interrupts handlers for IPI interrupts.

In Understanding the Linux Kernel, 3rd Edition by Daniel P. Bovet, Marco Cesati https://www.oreilly.com/library/view/understanding-the-linux/0596005652/ch04s06.html lists three kinds of IPIs:

CALL_FUNCTION_VECTOR
RESCHEDULE_VECTOR
INVALIDATE_TLB_VECTOR

However in the latest kernels, I find the below comment in arch/x86/include/asm/entry_arch.h.

 * This file is designed to contain the BUILD_INTERRUPT specifications for
 * all of the extra named interrupt vectors used by the architecture.
 * Usually this is the Inter Process Interrupts (IPIs)
 */

/*
 * The following vectors are part of the Linux architecture, there
 * is no hardware IRQ pin equivalent for them, they are triggered
 * through the ICC by us (IPIs)

https://git.kernel.org/pub/scm/linux/kernel/git/stable/linux.git/tree/arch/x86/include/asm/entry_arch.h?h=v5.6.15

Could someone confirm whether all those Vectors listed in the file are different kinds of IPI for x86_64. For ARM I could find a unified handler - handle_IPI() for all the IPIs. A switch case is used to find out which IPI.

Peter Cordes
  • 245,674
  • 35
  • 423
  • 606

1 Answers1

3

On x86 any interrupt vector can be triggered by an IPI, so there isn't (or aren't) a designated interrupt vector.

The IPI delivery mode and vector field

The image above depicts the format of the register used to send IPIs, the Fixed mode uses the Vector field to make the target CPUs execute the interrupt service routine associated with that vector. It's like an int vector instruction was executed in the targets.

So Linux can, theoretically, directly invoke any interrupt on any other CPU.
However, kernel modules often need to run a function on specific CPUs; so Linux has a set of utility functions like smp_call_function_single that will make the life of the programmer easy.
These functions are implemented with a mechanism that's worth a chapter on its own, now I don't know the detail but it's not hard to image the basic idea behind: have a global queue of functions to execute and an interrupt vector that, once invoked, dequeues an item and executes it.
By calling that interrupt vector with an IPI, Linux can make the target CPUs execute the given function.

The interrupt vectors you found are used for this. You probably want to look at their 64 bits counterpart in entry_64.S and under the guard #ifdef CONFIG_SMP.
The acpiinterrupt and acpiinterrupt3 are just macros that define a label with the second argument, call interrupt_entry with the first argument (the vector number) NOTted and call the function named in the third argument.
Be careful that the 32 bits analog does some nasty prefix-concatenation with the target function name.

apicinterrupt CALL_FUNCTION_SINGLE_VECTOR call_function_single_interrupt smp_call_function_single_interrupt is roughly equivalent to defining the function:

;Metadata stuff (e.g. section placement)

call_function_single_interrupt:               ;<-- first arg
  push ~CALL_FUNCTION_SINGLE_VECTOR           ;<-- second arg
  call interrupt_entry

  ;other stuff (tracing, flags, etc)
  call smp_call_function_single_interrupt     ;<-- third arg

  ;other stuff (like above, plus returning)

The vector numbers are defined in irq_vectors.h and are, of course, also used in idt.c for the IDT.

The target functions (the interrupt handlers) are mostly (all? I didn't check) defined in smp.c and they probably are the closest thing to the ARM's handle_IPI handler.

Those seem to be the only vectors invoked through an IPI.

Margaret Bloom
  • 33,863
  • 5
  • 53
  • 91
  • I know I am quite late to this, but thanks a lot. Your explanation helped me understand better how IPIs are implemented and solve the issue I was facing. Cheers! – Sreena_th_read Jun 19 '20 at 16:46