GA pilot reported during takeoff retracted the gear a few feet off the ground and immediately the wind pushed the aircraft into the ground with the gear up and the aircraft slid about 40 feet.

Narrative

I am a dual rated fixed wing and rotorcraft commercial pilot with instrument ratings in both categories. I was planning on getting experience on grass fields. So I used another pilot; Person A's airplane; to do a simulated training flight last year. And then again this year to do a real grass flight. Person A met me at ZZZ1 and we did a briefing of a couple airports; the POH and landing and take off data for soft fields. I recommended various airports and he recommended ZZZ as a suitable runway. We took off from ZZZ1 when the winds were coming from the North at 12-16 knots and ZZZ1 was using Runway XX. I performed a simulated soft field takeoff. We then proceeded to bring the plane south and then turned around and headed towards ZZZ. We followed the approach path for Runway XX and although I was coming in a little fast the plane landed just fine. A plane landed right after me on XX as well.Person A then instructed me to come around back to Runway XX. We did not do the after take off and landing checks. Did not check the windsock. When we came around; I back taxied for a full length take off on Runway XX. The plane was not taking off properly so I aborted the take off. I then noticed that we had full flaps. So we did the full after landing and takeoff checks and went back around. We did not check the windsock. At almost no time did we stop the plane to brief or check. He insisted I have the power in on taxi with the yoke all the way back and focused on taxi instructions rather than issues with the take off.Then the accident came. I took off with a clean configuration in accordance with the POH. The plane was halfway down the runway and got off the ground. I leveled the nose and the plane was flying. I retracted the gear a few feet off the ground and then almost immediately the wind pushed my plane into the ground. With the right gear door first started digging; then the step; and then eventually the plane started sinking with the doors cutting in the ground. I chopped the power either right before or right after the prop hit the ground. It slid about 40 or so feet. There was minor damage to the runway. No injuries. No damage to other property.During that time people came out and told me about the wind shift and complications about that particular airfield. Person A said he didn't know what was happening and why I raised the gear. He actually didn't realize I raised the gear at all. He and I did not notice the wind had shifted downwind and now was over 10 knots tail. I believe that after the first aborted take off the instructor should have told me to taxi to a parking spot and get out and evaluate the conditions of the soft field. Instead he advised me to taxi back and did not even discuss why the plane failed to take off initially. These same people wanted to get the plane off the runway to avoid a shutdown of ZZZ. Since this was right after the incident I was confused. When they attempted to bring equipment out to move the aircraft and tried to take the cowling off to access the engine hoist; I advised them to stop and not move the aircraft. They desisted.