What's typically meant by 32-bit or 64-bit machine is the size of the externally visible ("architected") general-purpose integer registers.
This has very little to do with how the hardware is built though. For example, let's consider the (long obsolete) Intel Pentium Pro. It's normally considered a "32-bit" processor, even though it supports up to 36-bit physical addresses, has a 64-bit wide data bus, and internally computations on all supported operand types are carried out in a single set of registers (which are therefore 80 bits wide, to support the largest floating point type).
At least in the case of Intel processors, even though larger physical addressing has been available for a long time, the largest amount of memory directly visible within the address space of any one process on a 32-bit processor is also limited to 4 gigabytes (32-bit addressing). The 36-bit physical addressing allows addressing up to 64 gigabytes of RAM, but only 4 gigabytes of that can be directly visible at any given time.
The change to 64-bit machines mostly involved changing what was made visible to the user (or to code at the assembly language level). Again, what you see is rarely identical to what's real. For example, most 64-bit code sees pointers/addresses as being 64 bits, but actual processors don't support that large of addresses. Current CPUs support 48-bit virtual addresses, and (at least as far as I've noticed) a maximum of 40 bits of physical addressing. On the other hand, they're designed so in the future, when larger memory becomes practical, they can extend the physical addressing out to 48 bits without affecting software at all. Even when they increase the 48-bit virtual addressing, in a typical case it'll only affect a small amount of the operating system kernel (normal code is unaffected, because it already assumed addresses are 64 bits).
So, no: a 64-bit machine does not really support up to 64 bits of physical addressing, but most typical 64-bit software should remain compatible with a future processor that did support directly addressing that much RAM.