My hubby had the same experience of yours. But now he has a new family with his second wife. He was very sad like the present you, but he moved on and focused on his future life with his lovely son. Now he believe what he did is absolutely right.
Agree with your decision. When my hubby was pending their divorce, he and ex-wife had to live in the same house, but he slept with his son. He felt happy to keep closer to his son every night as the kid gave him much comfort and hope for future.