THE AIRCRAFT COLLIDED WITH THE AIRPORT PERIMETER FENCE AFTER THE PILOT EXPERIENCED A LOSS OF ENGINE POWER IMMEDIATELY AFTER TAKEOFF. THE PILOT WAS FERRYING THE AIRCRAFT WITHOUT A PERMIT TO ANOTHER AIRFIELD TO FIX A KNOWN ENGINE EXHAUST SYSTEM DISCREPANCY. A MAINTENANCE PERSON ADVISED THE PILOT BEFOREHAND THAT THE AIRCRAFT SHOULD NOT BE FLOWN WITH THE PROBLEM. THE AIRCRAFT SERVICE MANUAL INDICATES LOW ENGINE POWER COULD BE CAUSED BY A RESTRICTION IN THE EXHAUST SYSTEM. THE MUFFLER WAS FOUND TO BE BROKEN IN THE AREA ADJACENT TO THE NUMBER 6 CYLINDER STACK ATTACH POINT.