My mother was in the same position as you when she married my father (probably worse - he beat her and cheated on her practically every day/night of their marriage) but anyway, what I want to say is that being alone in a foreign country without friends and family (and financial freedom of being able to work) puts you in a VERY vulnerable position! You either need to stay where you are and quuckly establish a circle of friends to keep you happy, or insist on going with him if that is what you want. Who knows - maybe give him an ultimatum - you go with him or you go home to your family. See how he responds, but you do need to develop a base of support for yourself including friends and finances.
Good luck
Cheers,
A.G.