PA-60 pilot reported an NMAC during takeoff from a non-towered airport resulting in a C310 aircraft executing a go-around.

Narrative

I was the 3rd aircraft to land on Runway XX in a row. I dropped off my friend and taxied back out to Runway XX. There was a Citation jet departing Runway XX. The Cessna 310 and jet were talking saying they saw each other. The 310 announced downwind for Runway YY; but in my mind I was thinking XX since everyone else was using that runway. I looked at my traffic display and saw the 310 looking like it was on downwind for XX (when actually he was on base for YY). I looked at final and base for XX and it was clear; I announced departing XX and took off. Someone called 'go around' 3 times loud and fast. I looked left and saw the 310. He banked left and I banked right to miss each other. Had he just landed; I would have flown over him. This was my most close call in many years of flying. The 310 pilot didn't know the other planes had been using XX other than the jet. My main mistake was in expecting the 310 to use Runway XX like all the other planes. ZZZ has become so busy; a Tower would have prevented this near miss from happening.