I know what you mean about the panic, last night was the first time I have slept properly since finding out, panic about every little twinge and I've had spotting almost everyday since before I could even do a test!
How many goes at IVF did you have? Did they know why it wasn't happening? I have pcos but didn't respond at all to Clomid and took nearly a month on injections with IUI before any follicles did anything.