Flight instructor reported a near miss with an aircraft while departing TOI airport and took evasive action to avoid a collision. Reportedly; the other aircraft was not communicating their position.

Narrative

On departure; we were cleared to climb VFR departing northeast from the airspace after exiting the downwind. While still inside the lateral and vertical confines of the Delta airspace at around 2;600 to 2;800 feet MSL; on about a 045 degree heading; and about 4 NM from the airport; we saw and avoided an Aircraft Y that was very close to us and was headed at about a 145 degree heading to the southeast. To my knowledge; this aircraft was not on frequency communicating their position with Tower. However; it was a particularly busy day in the airspace so it is possible they were cleared by Tower before we changed over to the active Tower frequency. The Tower also did not advise of any traffic in our vicinity. Due to constant altitude and heading of the traffic (heading to our right); we made a full power climbing turn away from the traffic. Once clear; we reported to the Tower that we were clear of traffic and exiting the Delta. Tower did not respond to traffic call specifically; but cleared our frequency change. I watched as the aircraft continued to clip the Delta airspace before exiting south and continuing on its way beyond the confines of the airspace. I tried to look up the tail number of this aircraft after we were able to avoid them; but [was unable].