I always keep our window close at night. Since they always come out near pipe/water area, pay more attention to kitchen and bathroom....see if there is any hole and cover it. Then use insect spray oftenly around the house and put 'cockroach house', which is very effective, as the cockroach will eat the poison food and pass the poison to the rest of their family.
If where you live is surrounded by food/cuisine then you better hire pest control regularly. I am really scare of cockroaches too
I also keep our windows closed at night! We have a cover of the kitchen drain but my husband isn't willing to do that in the bathroom (I bought it already!!)