The matrix inner product $\,A:B={\rm Tr}(A^TB)$ can be used to write the Frobenius norm as
$$\eqalign{
f^2 &= \|A\|_F^2 = A:A \cr
}$$
Find the differential and gradient of this expression.
$$\eqalign{
2f\,df &= 2A:dA \cr
df &= \frac{A}{f}:dA \cr
G=\frac{\partial f}{\partial A} &= \frac{A}{f} \cr
}$$
This matrix gradient can be vectorized.
$$\eqalign{
g &= {\rm vec}(G) = \frac{{\rm vec}(A)}{f} = \frac{a}{f} \cr
}$$
Note the alternate expressions for the differential of the norm.
$$\eqalign{
df &= G:dA \cr
df &= g^Tda \cr
}$$
Now find the Hessian.
$$\eqalign{
dg
&= \frac{da}{f} - \frac{a}{f^2}\,df \cr
&= \frac{da}{f} - \frac{a}{f^2}\,(g^Tda) \cr
&= \frac{1}{f}\Big(I-gg^T\Big)\,da \cr
H=\frac{\partial g}{\partial a} &= \frac{1}{f}\Big(I-gg^T\Big)
\,\,= \frac{\partial^2f}{\partial a\,\partial a^T} \cr
}$$
The Hessian can also be written entirely in terms of $a$
$$\eqalign{
H &= \frac{1}{f^3}\Big(f^2I-aa^T\Big)
}$$
Without vectorization, the Hessian is a 4th order tensor, which can be written in component form as
$$\eqalign{
H_{ijkl} &= \frac{\partial^2f}{\partial A_{ij}\,\partial A_{kl}} \cr
&= \frac{1}{f^3}\Big(f^2\delta_{ik}\delta_{jl}-A_{ij}A_{kl}\Big)
}$$
Note the permuted order of the indices on the $\delta$ symbols.