I'm not sure if different State have their own procedure. If you are in California, you can fill out application online thru the county web site, but make sure you print out the filled form, get notary, then send it back to the county. You can find details from this web site if you are in CA: http://www.cdph.ca.gov/certlic/b ... thDeathRecords.aspx
For passport, you can apply that in post office (not all post office, so you need to check which one can process the passport application first). Also, need to call in to make appointment (as I know they don't take wake in application now). See this site la: http://travel.state.gov/passport/get/minors/minors_834.html
I'm not sure if it is the same for every state and county. For my city, I filled out information of my kids in hospital after they bornt. Then, after around 3 months, I went to the Department of Public Health and applied for birth certificate. It's very simple and fast, just filled out the form, paid the fee and waited no more than 15 minutes for me.
For passport, yes, you can order it via post office. It takes a few weeks to have the passport mailed to you.