I find that many authors write the YangMills action as follows: $$\mathcal{J}= \int \operatorname{Tr}(F \wedge \star F).$$ I have yet to find a formal description of the symbol $\operatorname{Tr}$ and am especially interested in its relation to differential geometry. How should I understand it?
3 Answers
We choose the physics convention $F_{\mu\nu} = \partial_\mu A_\nu  \partial_\nu A_\mu  i g [A_\mu,A_\nu]$, where the generators of the Lie algebra are Hermitian. In this case the integral has a minus sign, $\mathcal{J} = \int \operatorname{Tr}(F \wedge * F)$.
The first thing to note is that $F$ has vector and Lie algebra indices. The trace is over the Lie algebra, not over the vector indices. The vector indices are just like those of the field strength in QED. In YangMills the field strength (also called the curvature form) is Lie algebra valued. In this case $F_{\mu\nu} = F_{\mu\nu}^a T^a$, where the summation convention is used, and where the $T^a$ are the generators of $\mathfrak{su}(n)$. To be explicit, $F$ has not only tensor components but matrix components $$(F_{\mu\nu})_{ij} = F_{\mu\nu}^a T^a_{ij}.$$
The inner product of $F$ with itself is by definition $(F,F) = \int F\wedge * F$ where $*$ is the Hodge $*$ operator. Thus, we are calculating $\mathcal{J} = \operatorname{Tr}(F,F)$. It is a standard exercise to find the exterior product of two $r$forms. We find $$F\wedge * F = \frac{1}{2!} F_{\mu\nu} F^{\mu\nu} dx^1\wedge\cdots\wedge dx^4.$$ Note that the differential forms don't "know" the Lie algebra. The algebra hasn't come into the calculation yet.
From here we must calculate the trace of $F_{\mu\nu}F^{\mu\nu}$, $$\begin{eqnarray} \operatorname{Tr} \left(F_{\mu\nu}F^{\mu\nu}\right) &=& \operatorname{Tr} \left( F_{\mu\nu}^a T^a F^{\mu\nu b}T^b\right) \\ &=& \operatorname{Tr}\left(T^a T^b\right) F_{\mu\nu}^a F^{\mu\nu b} \\ &=& \frac{1}{2} \delta^{a b} F_{\mu\nu}^a F^{\mu\nu b} \\ &=& \frac{1}{2} F_{\mu\nu}^a F^{\mu\nu a}, \end{eqnarray}$$ where we have used the standard normalization convention for the $T^a$, $\operatorname{Tr} T^a T^b = \frac{1}{2}\delta^{ab}$. (This comes from the fact that we want $\mathfrak{su}(2)$ to live naturally in $\mathfrak{su}(n)$, and the generators of $\mathfrak{su}(2)$ are taken to be $T^a = \sigma^a/2$, where $\sigma^a$ are the Pauli matrices.) Thus, we find $$\begin{eqnarray} \mathcal{J} &=& \int \operatorname{Tr}(F \wedge * F) \\ &=& \frac{1}{4} \int d^4x \, F_{\mu\nu}^a F^{\mu\nu a}. \end{eqnarray}$$
 18,473
 2
 39
 92
It's the trace and it's being taken over the vector space of the fields $F$.
 2,936
 1
 14
 16

1Thank you! I see, then $F{\wedge}{\star}F$ should be viewed as a linear operator...This way it makes sense ... – brail Apr 03 '12 at 16:15

@Autolatry: It is not clear what you mean by this. – user26872 Apr 08 '12 at 21:52

1@oenamen Sir, to provide extra explanation would require repetition of your excellent post. – Autolatry Apr 09 '12 at 14:43

@oen: What do you mean? Seems clear to me $ \mbox{ } :\int$. – Abhimanyu Pallavi Sudhir Aug 04 '13 at 13:54
As others have mentioned this is just the trace on matrices. But the following viewpoint may help to see where this is coming from. $F$ lives in the tensor product space $su(2) \otimes \Omega^2(M)$ of the Lie algebra $su(2)$ with $2$forms. $\Omega^2(M)$ has an inner product given by the $L^2$ inner product and $su(2)$ has a unique (up to scale) Adinvariant inner product given by the Killing form, which is really just $(A,B) \mapsto tr(AB)$ viewing $A, B \in su(2)$ as $2 \times 2$ matrices. Taking the induced inner product on the tensor product, the expression you wrote is the norm squared of $F$.
 18,051
 3
 52
 82