A319 pilot reported an uncommanded aircraft movement during push back when the tow bar was disconnected without the parking brake being set. The pilot applied the brakes and stopped aircraft movement.

Narrative

We were cleared to push pull to taxiway 1 off of the gate. All communication with ground crew was normal through engine start. The ground crew somehow disconnected the tow bar and began driving the tug away from the aircraft without ever confirming parking brakes were set with the cockpit. During engine 2 start I noticed the crew chief indicating to stop the aircraft. I tried to contact the tug via intercom to tell them to stop the pull with no response from the crew chief. The urgency was then noted from the ground crew to stop the aircraft at which point I applied brakes unsure if the tug was still attached as we could not see them from the cockpit. Simultaneously the FO indicated that the tug was off the right side of the aircraft and we came to a stop on the ramp. I then contacted ramp let them know we would be filing a report; to confirm visually if the area was clear around the aircraft; and to have a crew chief come back out to the aircraft to confirm no damage to the nose gear; the area was clear; and that the pin was removed. The flight deck never received communication from ground crew that the pull was complete; to set brakes; or any of the standard clearance from the Captain to disconnect inter phone.Cause: No communication from ground crew to flight deck after tug complete.