when my new maid arrived one year ago, my agent gave me her passpart and an authorization letter signed by my maid that her passpart is kept by her employer through the whole contract.
So far, her passport is kept by me and I would talk to the agent after hearing what you said and considering giving it back to her. However, I will terminate her contract soon because of unsatisfactory performance.