Whether he means it or not I wouldn't give him the power of deciding whether or not the relationship remains. He can't force you off the mortgage, any judge will rule to sell this property where there is no consensus as to what to do. Don't leave the home as legally it's not a wise move. Sounds like you may need to start thinking about ways to protect yourself ie what you would do in the event of you splitting up, or an exit strategy, if nothing else it's empowering to be prepared whether you need to act upon it or not.