Air Carrier Captain reported a late instruction from ramp control; a miss communication with the push back crew and a momentary loss of control of the aircraft during push back from the gate. The ground crew was informed by the Captain that brakes were set; ramp tower asked for the aircraft to moved further back and the Captain relayed the instructions to the push back crew. When the brakes were released the aircraft began to roll backwards. The Captain aggressively set the parking brake. The Captain reports the ground crew needs more training in proper procedures and communication.

Narrative

I was in command of Aircraft X from ZZZ to ZZZ1. The flight was scheduled to depart out of Gate XX. We boarded; loaded cargo; and received our closeout and paperwork without incident. For the initial pushback clearance; we were instructed to push back tail east; and to drop abeam [Gate] XY. The ground crew was instructed to follow the clearance. In the middle of the pushback; our clearance was amended to push tail south. We noticed that Aircraft Y was on the ABC line facing east in preparation to park at [the gate next to XY]. There was also a company Aircraft Z on the ABD line waiting on further instructions. I amended the instructions to the ground crew in turn. I opted to keep engines off until after the tug was disconnected; it became apparent that the Ramp Controller on frequency was struggling to figure out where to move traffic.As the ground crew signaled for the parking brake to be set; I set the brake and cleared them to disconnect. Seconds later; the Ramp Controller asked us to be pushed further back abeam [Gate] XZ. I rang the ramp horn and flashed the landing light three times to reestablish communications with the ground crew. Note that the parking brake was still set. Once communication was established; I told the crew that Ramp called and wanted us to push further south abeam XZ. I then asked if they had control of the aircraft and were ready for brake release. The Ramper said she had control and was ready for the brake to be released. I released the brake and then verbalized my action. I noticed uncontrolled aircraft movement backwards and the Ramper quickly started that they did not have control. I set the parking brake immediately and stopped the aircraft. I had no reason to believe that contact with any obstacle was made; nor did I have reason to suspect aircraft damage. I made a radio call to Ramp Control that we were trying to sort out communications with the tug crew and we would be pushing back when it was safe to do so. Furthermore; due to the sudden jolt; I made a PA reassuring the passengers that there was a miscommunication during the pushback; everything was fine; and that the jolt was the parking brake being set. I had the FO (First Officer) clarify what the Controllers intentions were; which was to turn into the ABE line; pass [Taxiway]; then exit through Spot XXX. Eventually; proper ground communications were established; we safely released the brake; disconnected; taxied; and completed the flight with no further incident.In my opinion; contributing factors to this incident were a confused ramp controller and either an inexperienced ground crew or one which lacked proper vigilance. Multiple clearance amendments from the Controller had to be relayed to the ground crew; and this only caused confusion for everyone. Needing to reestablish communications after disconnecting the interphone also increased risk. Fortunately I was vigilant in scanning and monitoring the aircraft during the pushback so I could intervene in a timely fashion to avoid damage or injury. At this company; on time performance is heavily emphasized; so I cannot overlook this pressure as a possible factor for everyone involved.The continued emphasis of standard pushback phraseology can never be understated; not just for pilots; but for ground crew as well. On time performance must be tertiary to safety and comfort; respectively. The state of who is in control of the aircraft and the state of the towbar should never be ambiguous; and with that said; no punitive pressure should be on anyone's mind. The consequences are too high for rampers; in addition to airline property; to underemphasize pushback safety. The company has correctly emphasized constant vigilance during pushback and other flight phases via messages and training; and it should continue to do so. Proper training of both ground crew and ramp controllers in high workload situations should made. Emphasis should be made on preventing ramp congestion and conflicts; not managing it. This way indecisive instructions should not have to be made to an aircraft under tow.