A320 Captain reported a loss of directional control resulted in a rejected takeoff. The flight crew elected to return to the gate for maintenance action and to deplane several passengers who did not want to continue on the flight.

Narrative

We were in ZZZ to ZZZ1 taxing out it was raining and overcast 1;700 ft. and rain. After three reroutes we then was cleared for position and hold. We aligned on Runway X be ready for immediate departure traffic 2 miles out. Aligned; I held the brakes powered up the engines to approximately 40% N1 FF approximately 1400 pounds; feeling the aircraft rumble/ shake engines stabilized are ready to go. Once cleared for Takeoff I then stood up engines throttles to the 12 o'clock position stable released brakes with aircraft moving then smoothly advanced to FLEX detent. Once in detent with engines powered up I could not keep aircraft on center line and was veering quickly to the right. Giving the engines No 2 sec more to catch up; veering uncontrollably to the right off center line. I rejected the takeoff; full reverse manual brakes and placing my hand on the tiller to align back to center line. This exacerbated the loss of nose control; nose wheel skidding on the wet runway left then right then to a slight left. I then once again hit the brakes to a quick total stop. All then cooled down on runway with reversers stowed. We then taxied off on the second left taxiway. This event on the runway was approximately 850 ft. total airspeed was not even alive for the First Officer (FO) 'call' We both examined all engine gauges all was normal and sat for a moment to review what just went on called the flight staff on all call. They were all OK and passengers too then several passengers wanted to get off jet as we were think if it would it be prudent to go to gate for maintenance review. This was then with no hesitation of us to go back to the gate and deplane jet and allow maintenance to figure out what was the cause of engine issue.The use of the tiller exacerbated the loss of control of aircraft steering below VMC ground on a wet runway. The use of the brakes to a stop is when we truly regained total control of the Jet. Once we returned to gate is when we then realized the event and how my adrenaline was running. The station manager even noticed how our demeanor was and asked how we felt. I was reluctant to say then it hit me I need to wind the clock and slow down I then called myself off of the flight; fatigued not safe. I as Captain felt not safe to continue flying this day. Take the event in to adjust there was no hull loss nor any soul lost and the next day I flew as Captain ZZZ to ZZZ2 with no further issues.