$\def\QQ{\mathbb{Q}}\def\ZZ{\mathbb{Z}}\def\Gal{\mathrm{Gal}}\def\char{\mathrm{char}}$I want to record here what can be done reasonably easily following the approach of my 2011 answer. I have a plan for a much larger blog post, but it is an ambitious one so it may be several months until I get to write it. The aim of this answer is to provide a concrete proof of the following:

Suppose that $K/\QQ$ is Galois with Galois group $\ZZ/p$, where $p$ is a regular prime. Then $K$ is contained in a cyclotomic field.

Making $p$ irregular seems to force more advanced methods, either involving ramification groups or the Stickelberger theorem. Working with prime powers rather than primes seems to only require more care (especially $2^t$), not more high power tools, but it is a lot of work.

The case $p=2$ was done in the other answer, so I assume $p$ odd.

Let $\zeta_p$ be a primitive $p$-th root of unity. We have $\mathrm{Gal}(\QQ(\zeta_p)/\QQ) \cong (\ZZ/p)^{\ast}$. Explicitly, for $a \in (\ZZ/p)^{\ast}$, let $\sigma_a$ be the element of $\mathrm{Gal}(\QQ(\zeta_p)/\QQ)$ with $\sigma_a(\zeta_p) = \zeta_p^a$.

**Basic computations**

Let $\beta \in \QQ(\zeta_p)$. We would like to know whether or not $\QQ(\zeta_p,\beta^{1/p})$ is Galois and abelian over $\QQ$.

**Claim 1** $\QQ(\zeta_p,\beta^{1/p})$ is Galois and abelian over $\QQ$ if and only if, for all $a \in (\ZZ/p)^{\ast}$, we have
$$\sigma_a(\beta) = \beta^a f_a^p \quad (\ast)$$
for some $f_a \in \QQ(\zeta_p)$.

**Proof:** Suppose that $\QQ(\zeta_p,\beta^{1/p})$ is Galois and abelian over $\QQ$. Fix $a$ in $(\ZZ/p)^{\ast}$. Lift $\sigma_a$ to some automorphism $\sigma \in \Gal(\QQ(\zeta_p, \zeta^{1/p})/\QQ)$ and let $\tau \in \Gal(\QQ(\zeta_p, \zeta^{1/p})/\QQ(\zeta_p))$ be the automorphism $\tau(\beta^{1/p}) = \zeta_p \beta^{1/p}$.

Since $\sigma$ and $\tau$ commute, we compute
$$\tau(\sigma(\beta^{1/p}))= \sigma(\tau(\beta^{1/p})) = \sigma(\zeta_p \beta^{1/p}) = \zeta_p^a \sigma(\beta^{1/p}).$$
We also have
$$\tau(\beta^{a/p}) = \zeta_p^a \beta^{a/p}.$$
Therefore, $\sigma(\beta^{1/p})/\beta^{a/p}$ is fixed by $\tau$. Since $\tau$ generates $\Gal(\QQ(\zeta_p, \beta^{1/p})/\QQ(\zeta_p))$, we deduce that $\sigma(\beta^{1/p})/\beta^{a/p} = f_a$ for some $f_a \in \QQ(\zeta_p)$, and thus $\sigma_a(\beta) = \beta^a f_a^p$.

Conversely, suppose that $(\ast)$ holds. We can reverse the argument to show that $\QQ(\zeta_p, \beta^{1/p})/\QQ(\zeta_p)$ is Galois and
$$1 \to \Gal(\QQ(\zeta_p, \beta^{1/p})/\QQ(\zeta_p)) \to \Gal(\QQ(\zeta_p, \beta^{1/p})/\QQ) \to \Gal(\QQ(\zeta_p)/\QQ) \to 1$$
is a central extension. But $\Gal(\QQ(\zeta_p)/\QQ)$ is cyclic, so every central extension of it is abelian. $\square$

Let $\beta$ as above and let the ideal $(\beta)$ factor into primes as $(\beta) = \prod \pi^{v_{\pi}(\beta)}$. Note that, if $q$ is a prime of $\ZZ$ which is $1 \bmod p$, then $q$ splits completely in $\QQ(\zeta_p)$. Fix for convenience a particular $\pi_q$ lying over $q$ for each such $q$. For any prime $\pi$ of $\QQ(\zeta_p)$, let $\mathrm{char}(\pi)$ be the characteristic of the residue field.

**Claim 2** With the above notation, there are constants $c_q$, one for each $q \equiv 1 \bmod p$ such that
$$v_{\pi}(\beta) \equiv \begin{cases} 0 \bmod p & \char(\pi) \not \equiv 1 \bmod p \\ c_q/a & \pi = \sigma_a(\pi_q),\ q \equiv 1 \bmod p \\ \end{cases}.$$

**Proof** If $\char(p) \not \equiv 1 \bmod p$, then there is some non-identity $a$ with $\sigma_a(\pi) = \pi$. (Here we use $p \neq 2$. Otherwise, $p=2$ and $\pi = (2)$ is a counterexample.) Then $(\ast)$ implies that $a v_{\pi}(\beta) \equiv v_{\pi}(\beta) \bmod p$, so $v_{\pi}(\beta) \equiv 0 \bmod p$.

If $\char(\pi) = q \equiv 1 \bmod p$, then $(\ast)$ implies that $v_{\pi_q}(\sigma_a^{-1}(\beta)) \equiv a^{-1} v_{\pi}(\beta) \bmod p$, so $v_{\sigma_a(\pi_q)}(\beta) \equiv a^{-1} v_{\pi}(\beta) \bmod p$ and the result follows. $\square$

Before proceeding to the proof, we need a supply of $\gamma$'s for which $\QQ(\gamma^{1/p},\zeta_p)$ is cyclotomic.
If $q$ is a prime which is $1 \bmod p$, then $\Gal(\QQ(\zeta_q, \zeta_p)/\QQ) = (\ZZ/q)^{\ast} \times (\ZZ/p)^{\times}$ surjects onto $(\ZZ/p) \times (\ZZ/p)^{\times}$, so there is some field tower $K \supset \QQ(\zeta_p) \supset \QQ$ with $\Gal(K/\QQ) \cong (\ZZ/p) \times (\ZZ/p)^{\times}$. By Kummer theory, we must have $K = \QQ(\zeta_p, \gamma_q^{1/p})$ for some $\gamma_q$.
From the previous computations, we know that $v_{\pi}(\gamma_q) \equiv 0 \bmod p$ if $\char(\pi) \not \equiv 1 \bmod p$. Considering ramification, or computing with Gauss sums, we also have $v_{\pi}(\gamma_q) \equiv 0 \bmod p$ if $\char(\pi) = q' \neq q$ with $q' \equiv 1 \bmod p$, and there is some $b \in (\ZZ/p)^{\ast}$ such that $v_{\sigma_a(\pi_q)}(\gamma_q) \equiv b/a \bmod p$. We may (and do) replace $\gamma_q$ by $\gamma_q^{1/b \bmod p}$ so that $v_{\sigma_a(\pi_q)}(\gamma_q) \equiv 1/a \bmod p$. (Stickelberger's relation gives an explicit choice of $\gamma_q$ with this normalization.)

**The proof**
Now, suppose that $K/\QQ$ is Galois with Galois group $\ZZ/p$. Then, by Kummer theory, $K(\zeta_p) = \QQ(\beta^{1/p}, \zeta_p)$ for some $\beta$, and $\beta$ must obey condition $(\ast)$.

We will show that $\beta$ is in the group generated by $(\QQ(\zeta_p)^{\ast})^p$, by the $\gamma_q$ and by $\zeta_p$.

We first consider the factorization of $(\beta)$ into prime ideals: $(\beta) = \prod \pi^{v_{\pi}}$, which is described by Claim 2 above. We replace $\beta$ by $\beta \prod \gamma_q^{-c_q}$.
After making this replacement, we have $v_{\pi}(\beta) \equiv 0 \bmod p$ for all $\pi$.

Thus, we can assume that $(\beta)=I^p$ for some ideal $I$. Since $p$ is regular, we deduce that $(\beta) = (f^p)$ for some $f \in \QQ(\zeta_p)$. Replacing $\beta$ by $\beta f^{-p}$ does not change the extension, and we may now assume that $\beta$ is a unit.

We abbreviate $\sigma_{-1}(x)$ by $\bar{x}$, since in any complex embedding of $\QQ(\zeta_p)$, the element $\sigma_{-1}$ acts by complex conjugation. Taking $a=-1$ in $(\ast)$, we have
$$\beta \bar{\beta} = \theta^p$$
for some $\theta$ and, since $\beta$ is a unit, so is $\theta$. Hitting both sides of the equation with complex conjugation, we have $\theta^p = \bar{\theta}^p$.

Now, replace $\beta$ by
$$\beta' := \beta\cdot (\theta^{(p-1)/2}/\beta)^p.$$
(This is the second time we have used that $p \neq 2$, to make $(p-1)/2$ an integer.) Note that $\beta'$ is likewise a unit. We have
$$\beta' \bar{\beta'} = \frac{\beta \bar{\beta} \theta^{p(p-1)/2} \bar{\theta}^{p (p-1)/2}}{\beta^p \bar{\beta}^p} = \frac{\theta^p \theta^{p(p-1)/2} \theta^{p(p-1)/2}}{\theta^{p^2}} = 1.$$

We see that $\beta'$ is a unit of $\QQ(\zeta_p)$ such that $|\beta'|=1$ in every complex embedding. So, by a result of Kronecker, $\beta'$ is a root of unity. We see that $\QQ(\beta'^{1/p}, \zeta_p)$ is cyclotomic, as promised.