A321 pilot reported the aircraft began to roll during pushback when the tug driver disconnected without confirming with flight crew that brakes were set. Captain applied brakes to stop roll.

Narrative

During pushback from Gate X in ZZZ; the ramp controller cleared us to push back past Gate and tail South so that an arriving aircraft could taxi to its arrival Gate Y. I conveyed these instructions to the tug driver. The tug driver stopped the push back; tail East just a bit farther out than the normal Gate pushback. I reminded him that the instructions were tail south and he refused to continue and stated 'There is plenty of room for him to get by'. At the same time; the ramp controller and the parking aircraft were querying why we had not pushed as directed. During this time of confusion; the tug driver cleared us to start engines but did not use standard tug driver to flight deck communication to disconnect. In the flight deck as the First Officer was communicating with Ramp Control; I saw the aircraft begin to roll. I immediately applied foot brakes and set the parking brake. I asked the tug driver if there were any injuries or any aircraft; tug; or tow bar damage and the tug driver replied 'no there is not'. We completed the disconnect and continued the taxi out for departure. This was a nonstandard pushback for ZZZ Gate X. However; the tug driver refused to push tail South as cleared. Despite the confusion; the ramp personnel that disconnected the aircraft did not confirm the nose gear brake set light; nor did the tug driver query with me that the brake was set despite the lack of a light indication. This was a very dangerous situation and I am thankful that no one was injures and that there was no damage incurred. Lack of ground crew training has become very evident in the recent past. Ground crew have become defiant to Pilot in Command instruction and despite recent improvement; continuously attempt to communicate to the flight deck with nonstandard communication.