After you finished the distilled water brought from HK, you can buy bottles of distilled water in the convenience stores. If you need hot water, you can boil the distilled water in hotel first.
About the food, if you go there by yourself, you can choose the food that suitable for young children. But you join tour, maybe it's better for you to bring along with some baby food that can mix with rice ( you can buy at sogo, jusco, or UNY...)