B757 Captain reported failure of ground crew to follow proper communication and procedures resulted in a critical ground conflict event.

Narrative

We were delayed out of ZZZZ by about 2.5 hours. The boarding process moved along quickly. However; once the aircraft door closed we waited a while with no communication. I could only see one ground crew. Once I had his attention I signaled asking what's going on. She shrugged his shoulders. Eventually someone came on the intercom. We pushed back and set the brake. The ground crew on the intercom cleared us to start engines. During engine start the tug disconnect and all crew walked away. We were very close to the runway so we ran the before takeoff check list and waited for engine warm up. During that entire time there were no ground crews visible or anyone communicating on the intercom. I had no marshaller and we assumed all ground crew had walked away. We called for taxi and received a clearance. As I spooled the engines up I heard a call out on the intercom. I immediately brought the engines to idle and set the brake. The ground crew on the intercom was still there. I asked if he was OK then cleared him to disconnect head set. He disconnected; walked to the front side; gave me a wave and returned to the terminal. The rest of the flight was uneventful.