Flight instructor on training flight with student reported NMAC with another aircraft in climb.

Narrative

Coming into ZZZ; another Aircraft Y was in the hold at ZZZZZ and we coordinated so we decided to climb and turn away from the hold to give the Aircraft Y time to shoot the approach. I noticed that the aircraft had taken off from ZZZ1 but we were climbing up to 4500; so I decided to continue the climb in that direction opposite the FAF course. I saw that the airplane departing ZZZ1 was climbing so I decided to continue my climb. Soon; I noticed that they were rapidly climbing to our altitude right towards us on the left. I asked my student if he could see him. As soon as I said that; I saw the aircraft and noticed that if they continued their climb; we would be at the same altitude so I took controls and abruptly pitched us down to avoid. Our belonging flew upwards. We cleared the aircraft and continued with the flight once both the student and I agreed we felt good enough to fly. The aircraft was out climbing us and was flying straight to us as viewed from our left hand side of the airplane. They were not talking on the ZZZ frequency.