According to the Stanford Encyclopedia of Philosophy,
Reasoning is the ability to make inferences, and automated reasoning is concerned with the building of computing systems that automate this process
Most of the problems in Artificial Intelligence (AI) are reasoning problems, and indeed both planning and CSP are among them.
Planning is a subfield of AI devoted to finding action sequences that achieve an agent's goal. For example, for going from Times Square NY to London Piccadilly Circus, you need a plan, namely a sequence of actions that will allow you to reach your goal, namely "being in Piccadilly Circus." A plan might be: "go to the nearest subway entrance", "take the first train to Jamaica station", "take the transit to the JFK airport", "take the airplane towards London Heathrow", "take the underground until Piccadilly Circus".
A constraint satisfaction problem (CSP) is a search problem where the goal is to satisfy some constraints. For instance, if there are N countries on a map, given a set of colours, the goal could be to colour each country such that no two adjacent countries are the same colour.
Having said that, there are approaches that transform a planning problem in a CSP problem, e.g. SATPLAN.
For your reference, a good starting point is "Artificial Intelligence: A Modern Approach" by S. Russel and P. Norving; most universities and colleges have it in their library.