well, it is the guy that determines gender insofar as he's the one who supplies the X or Y chromosome - the egg has just an X chromosome. And it is true to a certain extent that the number of X sperm and the number of Y sperm may be influenced by genetic factors, which may also have affected the number of brothers/sisters he has.
However, his lifestyle will play a very big part. For example, most chefs have daughters rather than sons - this is believed to be due to the high temperatures of their working environment. Similarly a lot of professional cyclists (or very keen amateurs) have daughters.
Timing also plays a big part. X sperm tend to be heavier and slower, but stronger than Y sperm (hence coping better with higher temperatures which kill off the Y sperm earlier). Chances are that if you have sex three days before ovulation, but not after that, you are more likely to have a girl. The Y (boy) sperm tend not to survive long enough to meet the egg. If, on the other hand, you have sex for the first time that cycle very close to ovulation, the Y sperm usually win the race...
Then there's the female body - the PH level of the reproductive system favours some sperm over others - can't remember which way round it is.
None of this is definite though. Men produce millions of sperm each ejaculation, so there'll always be some of each that get through if any do, no matter what the conditions, it's just a question of how the odds are stacked. If 10 sperm make it to the egg at the same time, and 9 are Y and 1 is X, it's still a 10% chance of a girl!