Commits in git are inviolate - you can't change a commit. A commit is defined recursively - it includes its previous commit. So you can not just change a single point in history.
You can checkout the last good commit - before the mistake - then make a good commit. Now you will need to rebase all the changes from the tag following the mistake to the last tag you made.
Then you will need to move the tags created after the mistake on to the re-based commits (use git tag with the -f option).
In pseudo code (you would use the SHA's of the commits instead of the numbers here):
3
|
2
|
1 - mistake
|
0
git checkout 0
* make changes *
git commit -m
3
|
2
|
1 1' (good)
| /
0
git rebase --onto 1' 1..3
* fix merge conflicts etc.*
3 3'
| |
2 2'
| |
1 1'
|/
0
Now you will have to delete the tags on 1,2,3 and recreate them on 1', 2' and 3' (or move them with the force option on git tag). With the old tags gone (assuming any branch names on the old commit are also deleted), a git gc will delete the old commits.