My midwife is a man, and I feel very comfortable with him. In fact I feel more comfortable than I did with the female midwife I had before (I didn't have a problem with her, they just reshuffled the staff).
His manner is great and he's very knowledgable. Also I think it's great for my partner because a male midwife can relate more to the father than a female midwife can, and the appointments are more a discussion with the three of us than they were with a female midwife.
As for the birth itself I can't see I'll have a problem. As someone else said I think I'll be rather too distracted to be worried what sex my midwife is - and if I need a consultant there they are all male in my area in any case.