Time to throw some maths into the mix. Just because something increases risk it doesn't mean it is something to worry over. If I buy 1 lottery ticket and you buy 2, you are twice as likely as me to win. I still don't think either of us will be millionaires come the weekend.
In this case the stats I can find are based on a study in the USA from 2013:
http://www.environmentalhealthnews.org/ehs/newscience/2013/06/nitrate-in-moms-drinking-water/
They conclude that a mother with a baby born with certain birth defects was almost twice as likely to have consumed 5.42 milligrams of nitrate daily. But what does that actually mean?
In the USA approx 1 in 1500 babies are born with a cleft palate. If you double your risk by consuming more nitrates than they recommend (and the study, by the way, does not necessarily mean that you do) then that would mean 2 in 1500 babies. Still great odds on having a baby with a normal palate. Most birth defects are less likely than cleft palate, so the difference this study reveals is even smaller in those more worrying cases.
Also - cause and effect is very difficult to prove when it comes to these things. The researchers interviewed the mothers of babies after they were born - asking them to estimate how much water they drank per day throughout pregnancy, and where that water came from (bottled, tap, filtered etc). Not the most precise way of gathering the information. There is no mention of taking other factors into account here. Do women who consume bottled (still) water instead of tap water (i.e. significantly less nitrates in their water) have more money and therefore better nutrition in general for instance?
In short - don't panic. I'm sure your baby is fine. Newspapers love headlines like 'this thing that everyone eats every day increases your risk of cancer by 50%!!!!', but they are never all that good at explaining that an increase of 50% might actually only mean that your risk has gone from 1 in 1000 to 1.5 in 1000.
