Okay here's my $.02. You need to decide once and for all whether you want to be with this guy. You haven't yet, and that is why he can still "get to you" and why you were bothered enough to make the post. You still care for him, at least a little bit. You may have closed the door, but you left the windows cracked a little. You know it, and he knows it.
You have to ask yourself why. Why don't you really want to end it. Are you afraid you won't find anyone better? Are you unsure whether or not you deserve better? Etc. Don't answer those for us here, just answer them for yourself. Once you are sure you are done with him you will do all the things necessary for this guy to get the message and leave you alone. Until then, you are just still playing the game with him.
Good luck sweetie.