I think you can consider the following questions before making the decision?
Are you really affordable if you quit your job?
Are you patient enough to get along with your child all the days?
From my own view, if you give up your job, there is no any financial problem. You can try to bring up your child by yourself. Developing the intimate relationship between parents and children should be as young as your child is.When they grow up, mostly they are willing to obey you much more.