Air carrier flight crew reported loss of aircraft control following pushback and tug disconnect.

Narrative

Before push back we briefed that we would single engine taxi to the departure runway. After push back we did our normal flows and called for taxi. After releasing brakes to taxi; the aircraft rolled backwards approximately 10 ft. This was the first time I have ever been on an aircraft when the aircraft rolled backwards without being attached to the tug; so it took me a few seconds to realize what was occurring. I then applied the brakes to stop the aircraft and asked my First Officer (FO) if we really rolled backwards. They said yes and out their window they could see there was a drainage grate that; due to the grade of concrete we were disconnected from the tug on; we had rolled back. This report is being filed just as a general safety concern due to the possibility of an aircraft that is tail heavy could possibly tip backwards if someone applied brake pressure too aggressively. Recommendations I have are to have the push back crew disconnect in an area where the aircraft won't roll back.