The private pilot used differential power to help land his twin engine, tail wheel-equipped airplane on a paved runway with a left crosswind. During the landing roll out, when the tail wheel touched down on the ground, the airplane turned into the wind and departed the runway to the left. It crossed over a ditch and struck a metal fence. The airplane sustained structural damage to the left engine mount. The left wing tip, left landing gear drag link, right vertical stabilizer and the rudder were also damaged. The pilot stated that the accident could have been prevented if he had kept the tail in the air longer with more differential power carried into the landing roll out. He reported the wind was from 100 degrees at 15 knots.