Flight Instructor reported taking evasive action to avoid an aircraft deviating from their course.

Narrative

My student and I were in Aircraft X; working on instrument approaches. We took off from ZZZ and flew north to ZZZ1 to fly the ILS XX approach. ZZZ1 tower instructed us to execute a left turn prior to the runway; to which we complied. ZZZ1 tower then instructed us to fly west bound at or below 5;500 feet MSL while in their airspace; and to remain at least one mile south of the ZZZ VORTAC for traffic inbound on the VOR-A approach. We complied with this and climbed to 5;400 feet MSL. Aircraft Y was the aircraft inbound on the approach when we were departing to the west. ZZZ1 tower issued an urgent traffic alert to Aircraft Y; telling them to turn north immediately. They did not respond; so ZZZ1 tower issued the same alert to us; telling us to turn South. I spotted the traffic at our 1 o'clock at the same altitude as this was happening and immediately executed a steep left turn to avoid them. Aircraft Y finally responded after a brief delay and apologized; turning north bound. ZZZ1 tower then told Aircraft Y that they were more than a mile off course to the south from the VOR-A approach; which put us on a head on collision course. I verified this by reviewing information after I landed. I estimate that if no action was taken by either of us; we would have come well within 500 feet of each other within the next 10-15 seconds.