AA-5 pilot reported experiencing engine issues shortly after takeoff; including failure and re-start; which resulted in the pilot returning to the departure airport where they landed safely. Post-flight inspection revealed water in the fuel system.

Narrative

City X; normally very dry; had a significant amount of rainfall the week before the incident flight. A thorough preflight was performed; in which water was found in both the left and right fuel tanks. Repeated samples drew decreasing amount of water until clean samples were obtained. Approximately 4-5 12oz jar samples were required from the right tank. Only 2 samples were obtained from the left tank. Notably; no sample was entirely water; all were mostly fuel.Total time from the beginning of runup to incident was 15 minutes. We were cleared for take off with a turn south on course almost immediately; and with tower aware we intended to transit the ZZZ1 Bravo we were given an early frequency change. We continued to climb at ~700fpm; and contacted approach. We received a transponder code from approach; and during the read back experienced a complete engine failure.I experienced a brief startle reaction; during which I interrupted my own read back with the word 'standby'. Within approximately seven seconds; I executed the emergency checklist and turned left back toward ZZZ. The aux fuel pump was already on; and mixture was already full rich. I applied carb heat with no change noted; but left carb heat on. I switched to the right tank (which I now suspect may have prolonged the situation). Fuel pressure never deviated from normal (approximately 5 PSI). For reasons I don't entirely understand; I closed and reopened the throttle-- an action I've done in the past during engine start. This operated the carburetor's accelerator pump; which caused the engine to briefly surge.Having informed the approach controller that we were having engine trouble and turning back; I was told he would 'get you back to tower right now'. I struggled to communicate while I was scanning for landing spots; managing airspeed; and trying to coax as much energy from the engine as I could by pumping the throttle. Approach also did not respond further and I told him I was going back to tower as that seemed to be the most expedient way of deconflicting any traffic.The tower controllers at ZZZ were phenomenal at coordinating our immediate arrival. With power bursts; we achieved a glide ratio of approximately 12.5:1 (10.1:1 published) and landed without incident. On the ground; the engine idled (roughly) but would not run at partial throttle. Pumping the throttle did indeed allow it to cough and surge to higher RPMs; but it would very quickly run out of gas and quit. Only additional throttle pumps or returning the throttle to idle allowed it to run at all. The fuel flow rate never deviated from normal throughout the incident. We kept the engine running for approximately 45 seconds after exiting the runway and stopping on the ramp.The carburetor bowl was drained; and a small amount of water was present. The right tank was sampled; and significant additional water was present; mostly in the post-tank sump (inboard drain). The left tank (initially used in flight) did not have detectable water present.Red food coloring was added to 12oz of water and added to the right tank. A borescope camera was utilized to visually track water moving toward the sump. We were surprised to see that while much of the water easily passed through the wing rib; some blobs remained on the bottom of the tank skin. Some caught on rivets; or sealant; or simply stuck to the clean aluminum. A shake of the wings caused the water to immediately break free and flow through the rib to the sump drain.I was not trained to rock the wings to remove water; nor on the importance of removing all traces of water (as opposed to simply obtaining a clean sample). The POH only specifies to check fuel 'for contamination by water or sediment' which can also be taken to require only an eventually clean sample. While I did obtain a clean fuel sample; I did not suspect any further contamination was a risk; and did not pursue fuel quality issues any further. I believe training should specifically emphasize the importance of actively removing water and procedures for doing so.I was also not trained to pump the throttle; which I believe made the difference between a landing without incident and an off-airport incident. While not all airplanes are carbureted; and not all carburetors have accelerator pumps; this should be communicated as part of engine systems understanding in private and sport pilot training.Finally; modifications to certified aircraft that improve climb rates should be seen as directly safety enhancing; and encouraged. Had I been in a less performant aircraft (or the same aircraft in hotter weather); the outcome would have been different. Performance margin matters; and the field approval and STC processes should better reflect that reality.