You should handle that on the server-side, If you really want to prevent multi-vote or prevent the same people from voting several time on the same subject.
This is why real votes always use authenticated users and never anonymous votes.
By checking the request is really a XmlHttpRequest (with @Shaun Hare response code or with the linked stackoverflow question in your questions comments) you will eventually block some of the CSRF but you won't prevent a repost from the user, using tools like LiveHttpHeaders 'replay' and such. Everything coming from the client side can be forged, everything.
edit* if it's not a voting system as you commented, the problem is teh same, you nedd 'something' to know if the user is doing this action for the first time, or if he can still do this action. There's a lot of different things available.
You can set a token on your page, use that token in the ajax requests, and invalidate this token for later usage server side. This is one way. the problem is where to store these tokens server-side (sessions, caches, etc)
Another way is to check on the server side the situation is still a valid situation (for example a request asking to update 'something' should maybe handle a hash/marker/timestamp that you can verify with current server side state.
This is a very generic question, solutions depends on the reality of the 'performed action'.