ERJ-175 Captain reported a loss of control event after releasing the parking brake for push back.

Narrative

While preparing to push; after getting on headsets; the ramp crew asked if we could release the brakes. We were 10 minutes prior to scheduled departure time; didn't have passenger numbers; and hadn't yet done the Engine Start" flow or checklist. I dropped the brake; and told the tug driver that we didn't yet have numbers; he responded that he already sent his numbers; to which I replied that we had his numbers in ACARS but that we were waiting for the passenger numbers. I don't remember what we were doing for the next 90 seconds; but we soon after got the passenger numbers and takeoff performance. We were about to start plugging in the performance numbers when I looked up and noticed we were moving backwards.It felt very smooth and nothing had been said by the ramp crew on headset since the mention earlier regarding the fact that we were waiting for numbers. I said out loud; "we're rolling (maybe moving); what is going on?" The tug driver responded with a hesitant "I don't know". In my startle and surprise; combined with how smooth the transition from stopped to rolling was; and that the ramp is slightly sloped backwards. My brain processed all that data into the idea that we were just rolling back; maybe not hooked to the tug. So I started applying foot brake pressure. The next thought through my mind; given that there was virtually no response from the brakes was that I had no hydraulic pressure; so I applied more brake pressure. All of the above happened in roughly 3-4 seconds.After applying the increase in brake pedal pressure is when we heard the thump from the tow bar banging against the tug. We came to a stop pretty quickly after that. I honestly can say exactly how abrupt the stop was. Afterward in talking with the FA's and FO; we all agreed that it didn't feel like an unusually hard bump. Not more than a usually bad push driver might give with a large and lurchy tug. The forward FA was in the isle at the time and said she felt two distinct bumps that caused her to need to hop a bit to keep herself standing. After we came to a stop; one of the rampers came to my window; which I opened; and asked if I had applied the brakes. I responded that we had applied brakes; since we had not been cleared to push; and I didn't know they had begun the push. We set the parking brake; they disconnected the tow bar; I made a maintenance write up; and we waited for contract maintenance to come.In their assessment; he discovered that 2 of the 3 shear pins on the tow bar were broken and the front connector part of the tow bar itself had been bent. Visual inspection of the nose wheel looked normal; however Mx (Maintenance) control determined that a gear swing would be necessary to determine that the landing gear was safe to fly with. Cause: In talking with the ramp crew afterwards; I learned that the guy who was on the headset and at the controls of the tug; was new and not checked out for pushes. He had a trainer in the cab of the tug; but the trainer was not on headset. The tug driver had only 2 previous pushes; but the trainer didn't realize just how inexperienced he was at the time. When he began the push; the trainer assumed that he had verbal clearance from us to begin. The tug driver said later that when he saw the green light he thought that meant we were cleared to push. We hadn't even gotten to that part of the communication yet; see the narrative above. Recommendations: 1. Better coordination and communication prior should be required. At a lot of stations when they are training a new tug driver; either them or their trainer would come up to the flight deck and mention that they are doing training. There was no indication to us that he was inexperienced; partly because the trainer himself didn't realize just how new he was.2. Once queried on why we were moving; had the tug driver responded with something more descriptive than "I don't know"; perhaps "aren't we cleared to push"; or "we're pushing back"; or"I have control of the plane". Any of those would have helped me create a better mental model of what was going on. Part of that communication is also on me. I could have been clearer in my question; like "why are we rolling backwards?" or "are you pushing us back already?"3. Despite the poor communication mention in #2 above; I also should not have applied the brakes. I arrived at the wrong conclusion in my brain and the consequences were worse than had we just kept rolling back for another 5 or 6 seconds. Which is obvious now in hind sight; and not the most obvious conclusion in the moment.4. Dropping the brake to get passenger numbers might be a legitimate procedure; but leaving the brake off in the following few minutes also contributed to this incident. Resetting the brake; which would give a red light on the nose gear; would have absolutely prevented the tug drivers confusion in the first place."