During a winter flight, passengers reported that cold air was entering the airplane from the left side of the passenger cabin. Afterwards, the pilot examined the area and discovered that there was a small gap under the emergency exit window that was allowing air to enter the cabin from outside the airplane. He opened the window and examined the rubber seal which was intact. He could not tell though, if it was compressed or thinner than normal. He then closed and latched the window and inspected the latch with a flashlight to make sure it was latched. Since he was going to fly back to his home airport in similar winter conditions on the next flight, He took several rolled-up paper towels and placed them between the trim and the window to try and keep the cold air out, and placed a strip of blue painters tape on the outside of the lower portion of the window to further reduce the entry of cold air. Since it was a beautiful day, he decided to fly the airplane once around the traffic pattern before fueling up for his return flight. After takeoff while on the downwind leg of the traffic pattern at 800 feet, he suddenly heard a "whoosh" behind his seat. Instead of landing, and then checking to see what happened, he instead checked for other traffic, turned on the autopilot, in heading and altitude mode, then reached around behind him to shut and latch the window which had opened 2 to 3 inches. Moments later, after turning back around to his normal seated position, he then heard a loud "pop" and turned around to find that the window had now opened completely. Since he was afraid it would come off the airplane and strike the tail, he reached back once again and pulled the window back down. The pilot advised that he must have "bumped" the autopilot off while he was doing this, since when he looked forward to check for traffic, he noticed that the airplane was approaching the ground. He then banked left and right to determine his location and spot any obstacles, raised the nose, and added power to climb. He then noticed that there were powerlines slightly higher than his altitude directly in front of him, and rather than risk a possible stall close to the ground by pulling back suddenly, he lowered the nose and "put" the airplane on the ground. At this point the airplane was approaching the edge of a field bordered by trees, so he pointed the nose of the airplane between trees. The airplane then struck the trees, and a fire ensued, resulting in substantial damage to the airframe. Examination of the emergency exit window by a Federal Aviation Administration inspector revealed that the paper towels the pilot inserted in the gap between the window and the airframe were interfering with the window's latching mechanism.