Air carrier flight crew reported an aircraft loss of control during push back procedure.

Narrative

Crew Chief at our gate push back; reported 'ready for push back; cleared to release brakes.' I queried him; asking if all his checks were completed. He acknowledged with a; yes. I replied; 'standby'. After receiving clearance to push back tail 'west'; the push back commenced. We were cleared to start engines. At a point we were told the push back was complete; to set brakes. I next commented we needed to be pulled forward to the yellow dashed line as I assumed this was the release line and also precluding the left wing from passing over pallet/ cargo bin carts. I did not reply and I did not set the brakes...thinking we would be pulled forward. Without further instructions from this Crew Chief the airplane started to move toward the push back tug. Out of the corner of my eye I could see the aircraft left wing man signal for an emergency stop. I did and set the brakes. I then signaled to the same wing man to connect onto the headset. The original crew chief eventually came on and said it was mix up in hand signals. I emphasized the brakes were now set and we needed to still be pulled forward. The tow bar was reconnected and we pulled forward to complete this procedure.Non compliance of SOP's by this Crew Chief performing the push back. Lack of understanding of how important confirmation from the Captain that the brakes are (in fact) set when asked. I never confirmed they were...we needed to be pulled further forward as I instructed him.This Crew Chief should be removed from tug driving to conduct push backs until he and associated ramp personnel successfully complete a course to fully re-train him. Possibly send them back through a full training program.