Perhaps his friend and mom are a bit jealous of the time you spend together? Really, what should matter is how he feels about you. If you feel they are negatively influencing his opinion of you, then the relationship was not that strong to begin with. If he really likes you, he wouldn't care what others think.
If you really care for him and vice versa, try to warm up to the mom and friend if it would make you feel better, but remember you're dating him, not them.
Good luck!