I think you need to have a good talk with your husband. After all, the basic unit that God has created, before church, is family, more so, Adam and Eve. So, if you two do not have a common understanding, your family will face a spiritual challenge.
Get your husband to understand you more and ask him to spend more time at home, since he cannot ignore his responsibility of being a FATHTER, he is the image of Jesus in front of the kids.