Air carrier Captain reported slight un-commanded aircraft movement due to low pressure in brake accumulator after push back was completed. Issue was resolved and flight continued.

Narrative

On Day 0 we were operating flight from ZZZ-ZZZ1. We just arrived on ZZZ2-ZZZ flight to turn the same tail number (Aircraft X) to ZZZ1. We noticed no malfunctions; abnormalities or ECAM messages during flight ZZZ2-ZZZ. During cockpit preparation I conducted a break check per SOP and it checked normal. After boarding; we pushed back normally to spot XX per ramp control instruction. We were utilizing smart start" and did not start an engine on push back. Once the push back was complete; ground ops requested that we set the breaks. I switched the parking break to on; noticed no abnormalities or ECAMs and then requested the tug to be disconnected. Seconds later; ground ops came back on the mic and asked us if the breaks were set. We replied yes and they said the aircraft rolled slightly. We did not have any ECAM faults or failures. We asked if the parking break light was on and he replied it was on. We recycled the breaks and noticed that our triple indicator was showing all zeros on the break pressure and accumulator pressure. We requested that ground ops immediately chock the aircraft and check for any fluid leaking on the ground around the wheels. (there were no leaks). We then coordinated with ramp control of the issue and that we needed to hold location. With the APU on; Engines off and Aircraft chocked I called maintenance control for guidance. After explaining the situation and events he asked us to verify N/W (Nosewheel) Steering Disc was shown on the ECAM and asked if we had any ECAM messages. We replied no ECAM messages and he requested that we push the yellow electric pump to pressurize the system. System pressure returned and maintenance control advised us that we did not need to come back to the gate and we could continue the flight. Now with the parking break on and showing pressure we requested chalk removal and we started engines. We taxied out; tested the breaks several times and had no other breaking issues during the flight and initial taxi in. When we arrived at the gate in ZZZ1 we set the parking break; shut the engines down and completed the parking checklist. With the aircraft chalked at the gate we called maintenance and recycled the parking break to test the yellow system again. Once again we lost all accumulator pressure. A logbook entry was made for the yellow accumulator's loss of pressure. *This report is being filed now instead of immediately after the event because new information of the aircraft bumping into the tow bar after the break was set has been recently relayed to us. At the time of the event ground ops only notified us that the "aircraft rolled a little." It was so slight that the First Officer and I didn't feel or notice anything. When we reset the break in ZZZ1 after blocking in; maintenance told us that he believes that it was low pressure or a slow leak in the accumulator. He believed this is why we never got an ECAM fault or failure."