Air Carrier Captain reported he failed to amend the flight release for added Hazmat prior to departure.

Narrative

I arrived at the aircraft and met my third brand new First Officer (FO) in as many legs. The inbound aircraft was approximately :30 late and we were hoping to make up some time as I was just pushing my first flight of the day at (time); with anticipated weather; aircraft and Crew Member swaps downline; meals and fatigue to be factors later in the night. During the preflight; we realized we were over Allowable Takeoff Gross Weight (ATOG) and had to pull two revenue Passengers and plan a engine bleeds off takeoff; which caused some anxiety with the probationary First Officer. We had Company HAZMAT added to the flight and I signed the required paperwork; but in the hectic environment; forgot to amend the Release for the Operations Agent. On climbout; I noticed the call light was illuminated and I reviewed the ACARS messages. The light was for a Part 117 notification; but there was also a previously reviewed message from Dispatch reminding us to amend the release for the Hazmat with times and initials. The message was sent while we were at the gate. Neither the First Officer nor I remembered seeing the message prior to pushback.It was busy; but since I had a brand new FO again; I was trying to be methodical; slow down and cover everything. The Dispatcher; Ops Agent; freight and Ramp Agents all did their jobs and personally briefed me of the HAZMAT location. The mistake was totally on me.