$\newcommand{\QQ}{\mathbb{Q}}$
Derivatives don't really go wrong, it's antiderivatives.
(EDIT: Actually, the more I think about it, this is just a symptom. The underlying cause is that continuity on the rationals is a much weaker notion than continuity on the reals.)

Consider the function $f : \QQ \to \QQ$ given by
$$f(x) = \begin{cases} 0 & x < \pi \\ 1 & x > \pi \end{cases}$$

This function is continuous and differentiable everywhere in its domain. If $x < \pi$, then there's a neighborhood of $x$ in which $f$ is a constant $0$, and so it's continuous there, and $f'(x) = 0$. But if $x > \pi$, there's a neighborhood of $x$ in which $f$ is a constant $1$, so it's continuous there too, and $f'(x) = 0$ again.

So the antiderivatives of $0$ can look rather messy. By adding functions like this, you can construct arbitrarily "jagged" functions with zero derivative. As you can imagine, this completely destroys the Fundamental Theorem of Calculus, and any results that follow from it.

This can happen in the real line to some extent, but it's not nearly as bad. The traditional antiderivative of $1/x$ is $\ln|x| + C$. But so is the following function:
$$ g(x) = \begin{cases} \ln x + C_1 & x > 0 \\ \ln(-x) + C_2 & x < 0 \end{cases} $$

By changing $C_1$ and $C_2$, we can push the two halves of the real line around completely independently. This is only possible because $1/x$ isn't defined at $0$, and so we've "broken" the real line at that point.

If you like dumb physical metaphors, here's one:

The real line is kind of like an infinite stick. If you wiggle a section of it, the whole thing must move.

With the $1/x$ example, you've made a cut at $x = 0$, and now you have two half-sticks. They can be wiggled independently, but each half must still move as a unit.

The rational numbers are more like a line of sawdust. You can't really move one grain by itself, but you can certainly take an interval and move it around independent of its neighbors.

By completing the rationals, you're adding all the glue between the grains to form a stick again. (I hope no one from diy.stackexchange is reading this...)