I don't believe you understand the notion of efficiency in terms of encoding. Informally speaking, you have to keep it mind there are two factors involved: (i) cost of having different symbols (in case of base 10 there as 10 different symbols, in case of base 16 there are 16 different symbols etc) and the length of the resulting string to encode a particular number.

When you consider both factors and apply some basic information theory to it, the answer may look a bit surprising: the most efficient encoding has a base $e$ (yes, that very $e = 2.718\dots$). Since we'd rather have some natural number as a base, the best we can get is base 3, and the next is base 2.

So, why, then, computers use base 2 (0 and 1) rather than base 3 (say, -1, 0, and 1)? The answer is that it is simple to design the circuits that distinguish between two (rather than three) states. (I do remember reading some of the earlier computers did use base 3, but I can't recall all the details.)

Now, with respect to octals and hexes, those are simply convenient ways to record the binary strings. If you did some machine-level debugging, you probably had a chance to read what's known as "hexadecimal dump" (contents of a memory). Surely it's easy to read than if it were written as binary dump. But what's lurking underneath of that is base 2.

(The answer on "why do we use base 10" has been answered elsewhere.)