A321 flight crew reported uncommanded movement during pushback procedure.

Narrative

During pushback; the push crew communication was intermittent. One moment; insufficient communication would occur then the headset communication would restore to a sufficient level. I even told the push crew to be ready to use hand signals in the event of lost communications.At the completion of the pushback process; the push crew requested that I set the parking brake. We received an ECAM pertaining the low brake pressure. I attempted to communicate to the push crew that the parking brake is not set and to not disconnect. Unfortunately; communications were lost for approximately 20 seconds. I attempted to tell the tug crew not to disconnect the aircraft and that the brakes were not set as I needed to troubleshoot the ECAM and wanted them to remain connected before I could trust our brake systems. Because of the loss of communications; they proceeded to go through their routine and the aircraft rolled back. I was able to set the parking brake to stop the uncommanded movement; thankfully; the ECAM cleared itself out. The main concern here is the spotty headset communications and the potential consequences of lost comms.