Don't expect things will get better.

If it was just that he needed more time to get affectionate, he would have said it. Instead, he plainly told you he is like that.
Since the relationship is new, you had better decide now if you can be with a boy who is not affectionate and be happy.
Breaking up later in the relationship is usually way worse.
People can be very in love and show very few signs of affection though. So it's really not you, like he said.