Wilson, I'm so sorry to hear what you're going through and I know there's nothing I can say that will make this better. I'm assuming that the doctor who is telling you that you won't conceive is the GP? Personally, I would feel happier discussing my options with a fertility specialist as in my experience, GPs generally don't know much about fertility. Although IVF would be the treatment offered, it is important to remember that things can be done to improve sperm. As you may have read in some of my other posts, we have dealt with sperm issues. Although it sounds like they weren't as bad as your OH's, I thought I'd pass on our experiences all the same.
My OH's first sample showed low motility so he was asked to do the same and it was practically identical. We then learned that sperm takes 90 days to be made so samples should be done at least that far apart. He also started taking vitamins and improving his diet and lifestyle in general. He was really committed to it and both his recent samples were much improved with all parameters well within normal range. My point is that there are things you can do without resorting to IVF although there are of course no guarantees.
Leading on from what GG said though, I think you may be dealing with greater issues and I think you need to work out where you both are with this. Having faced male factor issues, I know that accepting a childless future with a husband who was on board and did his best would be entirely different to one who wasn't. I think it's really important to consider that this really could build resentment just like GG said. I also don't think that he should be defining a timeline - that's a decision you both should be making together. No one ever expects to be faced with infertility but when you're unfortunate enough to have it forced upon you then of course you need to open your mind to alternatives about your future but only you know which of these are genuinely an option for you personally.
I hope you manage to talk things though and get a better understanding of both sides. As GG said, try not to make snap decisions and give yourself time to process and make the right decision for you whatever that may be. Emotions cloud judgement and I would hate for you to make a decision that you go on to regret later.
Wishing you all the best.