During initial climb from the airfield, the airplane began to shake violently and the propeller broke apart. The non-certified pilot turned the airplane in an attempt to return to the airfield. The pilot was unable to land at the airfield and elected to perform a forced landing to an adjacent field. During the landing roll the airplane contacted a levee and nosed over, coming to rest in an inverted position. In an interview with Federal Aviation Administration inspectors, the pilot stated that he had recently installed an experimental propeller on the airplane and during a ground test run the propeller made an unusual sound and the pilot discovered a "nick" on the leading edge of one of the propeller blades. The pilot notified the manufacturer, who recommended that the pilot trim the propeller portions down. The pilot, who was neither a certified repairman nor the airplane's builder, performed maintenance on the airplane and propeller prior to flight.