I am a female and married for almost 20 years with 2 boys. To be honest, I would think you may have more issues. Your husband may not give you all the caring you need but you ignore him (esp no sex) also a big No No. Divorce is not an easy way out, can you support the 2kids yourself financially.
I have issues in my marriage before as well. Esp I have a pre mature born so focus on baby too much.
You put some efforts well to see any improvement.