I guess you have to look at it from your baby's perspective. Is your baby happy about this? Stimulation is very important for kid's growth. All babies are learnt thru' playing. Baby will know who is the mother. I would rather like a harmony environment.