I think your relationship is a bit too young to tell. You didn't say how long you have known him before you decide to go out with him.
However, there were some stuff I find awkward from your story, especially about the phone number thing. It seems that you're a nice and caring lady. I just hope that he realizes and appreciates that. Relationship will only work if two parties are working together..
My suggestion is to take everything slow and let him make another move. It's okay to be cautious because you have a heart to protect. See how it goes... if he really cares and feels the same way for you, it'll show.
Another thing you can do is to talk to him, tell him how you feel and see how he would respond.
I hope things go well... take good care
*~Vie~*