No, I think your milk supply can be boosted up again if you pump more, say twice at office. Also, mama milk tea and fish soup may be effective!
Actually, I know my milk supply will be difficult to maintain if I decrease pumping frequency or let Hughes suck less! Thus, I keep regular pumping and let him suck quite clearly!!