I bought one in Seiyu. The shop assistant said that the fridge functions best when it's full. So maybe you can try to put in more bottles and see if it's better.
I've got the same problem some time ago and after I've ready the instruction carefully, I found out why.
When the milk is freshly bumped, it is warm and if you put it into fridge-to-go, it will have to cool down the milk first and then keep it cool, so it can't last 8 hours, but around 3-4 hours. It can only last 8 hours when the milk is already cooled down, like if you put if into fridge to cool down before put it into the fridge-to-go, then you can keep it up to 8 hours.