Is this rude??? please help?

Ok, well I have this Ex-best friend and he is my Ex-best friend for a reason. So we were really good friends, even tho he always insulted me. He is in love with me, he has told me so many times, but ughh. He is also kinda messed up in da head, like emotionally, he has tried to commit suicide 4 times. I kinda just think he wants attention. He doesn't have many friends, anymore. Ok but one day he asked me out but i had liked this other guy for 2 yrs so i said no. Then that day i found out that that guy liked me back, so we started going out (still going strong) But my friend got jealous, and started saying all kinds of things that were completly uncalled for, he called me a b*tch, a c*nt, a wh*re, sl*t, said my IQ was only 2, fat, also other stuff. He was also insulting things about me that i cant change, like my name (he was saying Crystal is a stupid name) even while all this was happening i said NOT ONE mean thing to him, not one. but he continued on. Then i made him cry, untentionally. Then about 3 days later he starts saying sorry and how he wants 2 b friends again, i keep saying no, then he was like stalking me so i get a lil rude and tell him to F off. He wont ever leave me alone, and blah blah blah, ill tell u the whole story if u ask me. OK so the ques is......The other day he asked for my new cell # and i told him no, he asked y and i said cause i didnt want to give it to him, then he has the nerve to call me rude.......so is it rude of me to not give him my # even if that stuff started mnths ago, he still insults me and trys to be my friend again ....