Air Carrier pilot reported the need to brake abruptly to avoid another taxiing aircraft which was supposed to give way.

Narrative

After engine start cleared to taxi to XYR. Aircraft Y had just landed XYR and was taxiing to their ramp near approach end of XYR. They were instructed to give way to us. After A/C began moving I looked away just for a moment to check a setting on my headset; F/O (First Officer) stated 'I don't think they're stopping'. I looked left and abruptly applied brakes as Aircraft Y went by. I immediately checked in with the F/As (Flight Attendants) to make sure they were ok. 'We're all good' was the reply. I let them know what happened; and we continued after confirming with ATC that yes; we were cleared; and also that Aircraft Y was supposed to give way; but had not. Headset issue. Older block one A/C. Early departure. No meal because everything in terminal was closed until after we had already boarded A/C. In the future if I have anything distracting me I'll just stop the aircraft. We frequently 'multi-task' for minor stuff but it depends on who else is operating. ZZZ is becoming a very busy place; yet the ramp remains 'uncontrolled'. It was dark and very early.