Air Carrier flight crew reported a communications breakdown between Captain and First officer regarding Hazmat documentation during busy closeout & pushback.

Narrative

We were waiting for our close out so the FO (First Officer) made about 4 calls to Load Control to see what the hold up was but we got no response. I called Dispatch and asked them to call Load Control. Because we were departing off XXR I decided to start the taxi and get the numbers on the taxi out so we weren't blocking [Aircraft Y and Aircraft Z] from taxing out. Load Control called and said we had dry ice in the forward compartment and birds in the aft compartment and they needed a Dangerous Goods Form. I sent the acknowledgment and we received the close out. By the time we had uploaded the numbers and run the taxi checks we were at Runway XXR and cleared for takeoff. The FO (First Officer) noticed that the dry ice was not on the closeout. I told him I had received a verbal from Load. I should have followed up when he queried it.I think I was too concerned with holding up ramp traffic. I should have not taxied until all the paperwork had come through and we could see that it wasn't complete. I guess I was thinking that because I had received a verbal description of our hazmat which I recorded and acknowledged I was good.