It really depends whether you have someone to help you to take care of your kids. If so, I will ask the maid to go definitely. You will always feel insecured if you keep someone at home you don't trust.
I understand how you feel. You are pissed off but you have hesitation to ask her to go. I have the same situation too.