He couldn't stop you, I think as long as reasonable access was agreed? Maybe get some legal advice. I live 11 miles away from my ex (not sixty, but some distance) - and frankly if I moved further I wouldn't ask him for permission. As long as access is possible, it should be fine x