Pilot of an experimental aircraft reported an engine anomaly during a test flight at the flight levels. During descent the anomaly dissipated and pilot continued to a safe landing. Pilot also reported confusion with ATC communications and that ATC informed them they had violated RVSM separation.

Narrative

The flight was a test flight of the engine performance at altitude on the experimental Aircraft X. The flight involved an envelope expansion of the operating ceiling of the aircraft. Previously; I had taken the aircraft to 43;000 feet. This flight called for a climb to 51;000; a trim shot; throttle frequency sweeps; (about 5 minutes total); and then a descent back to VFR altitudes and an RTB (Return To Base). Flight departed and landed at ZZZ airport.Prior to the flight; given that it was an envelope expansion flight; and the risk of compressor stalls/engine surges/engine instability; I pre-briefed with the ZZZ Center Ops Manager by phone. We had a 5-10 minute conversation; where I gave him my proposed route ZZZ - ZZZZZ- ZZZ1- ZZZZZ- ZZZ; and he agreed that it kept me out of the way of a lot of traffic (as I would be up against the western side of the mountains). We also discussed; that given the climb profile; should I have any engine instability; IAW the checklist on my test cards; I would bring the power back to ~25%; and request an immediate descent with a left 180 degree turn back to ZZZ1; or to ZZZ2 (whichever was closer at the time); to take advantage of the larger runways. If engine instability occurred; it would likely clear itself with the appropriate [abnormal/emergency plan] actions upon descending. The main reason we discussed this was so there would be no surprise to the controllers; and thought it would be an abnormal situation; unless the engine failed or I had to secure it; I would likely not need to declare an emergency. He agreed; and we also discussed the implications of my aircraft being negative RVSM. I then proceeded to spend the next 2 hours conducting final pre-flight checks; and a flight briefing.Prior to engine start; I phoned ZZZ Approach to pickup my flight plan. They gave me a squawk; but told me to expect the plan on calling them airborne. I started the engine; taxied out to Runway XX; and conducted a normal takeoff and left downwind departure (to avoid overflying the town). I contacted ZZZ Approach; who handed me off to ZZZ1; who then handed me off to Center as I flew direct ZZZZZ. Center cleared me first to FL230; and then eventually once on course to ZZZ; cleared me to FL330. Aircraft X is a military aircraft; and is aerobatic; as such; it is not the most stable IFR platform; and I did not have an effective autopilot available. During the level off; I exceeded the assigned altitude by 148 feet. ATC asked me to confirm I was level 33;000; and I informed them I was 33;100; and was correcting back to 33;000. After 5 miles; they cleared me to FL510.Upon climbing through FL410; I experienced an engine anomaly; I got a master caution indicating low engine fuel inlet pressure (possibly a failing boost pump); and the engine started surging near redline (ITT); then back near idle. As per my brief; my immediate action was a reduction in power; eventually bringing the throttle back to idle. I informed ATC that I 'needed an immediate descent' Center responded by asking me what altitude; I informed them I just needed a descent; and I wasn't sure what altitude. I informed (or at least thought I did; I may have not keyed the mic; or I may have been stepped on) ATC that I had a surging engine. This was an extremely high workload environment for me -- flight test; engine anomaly; single pilot; operating under an IFR flight plan. I was given a descent; to FL320 as I recall; and I immediately requested a left turn direct 'ZZZ1' the controller cleared me to turn 'upon reaching FL350'; I believe I began the turn above where I was supposed to. The engine was still surging; I was still in the process of executing the emergency procedure (the last step of which was; if the engine did not recover; to shut it down; so I was preparing for that possibility; and ensuring that I was within the ZZZ1 glide cone; which I was). ATC said that they were unfamiliar with that 'fix' and I returned with the phonetic spelling of'ZZZ1 airport'. I was then cleared for a further descent and direct ZZZ1. ATC then asked if 'everything was ok' I responded with 'I'm fine'. I believe this was misinterpreted. I meant that to indicate that I was executing my abnormal procedure; and was in no danger; and did not require assistance; I did not mean that the aircraft was operating completely normally; as I assumed he had heard my engine surge call.Upon reaching FL340; the engine had almost entirely stabilized; and I was able to increase my airspeed and rate of descent (above 35;000 the aircraft is Mach limited; and given the level of workload; I set my allowed speeds for specific altitudes low enough as to have significant margin to avoid an MMO overspeed). This is the moment that the controller called back and asked me 'to be ready to copy a phone number'. He read me the number; and I quickly copied it down; but did not read it back. To say that I was surprised; and slightly irritated that the controller decided to distract with such a non-critical thing during a critical phase of this flight; would have been an understatement.I was handed off to the next center controller; who asked 'if I needed any assistance' I thanked him; and re-iterated that I had an engine surge at altitude; and needed to descend; and wanted to turn towards the longest runway I could find that had services; just in case. He acknowledged this; very professionally; and it seemed like he could tell (perhaps from my tone of voice) that I was highly tasked; and he gave me a further descent. Once below 30;000; I started conducting engine checks (throttle response; and checked ITT against expected numbers); the engine showed no signs of an issue. Once I was stabilized in the 20's; I requested to turn back to my original destination; ZZZ. He cleared me on course; and to descend. Once with ZZZ approach below 18;000 ft.; I cancelled IFR; and proceeded to a happily uneventful landing and post flight inspection of the aircraft.Once the aircraft was secured; I called the number for ZZZ1 Center that I was told to call; and was told I would receive a call back in 20 minutes. I spoke to someone (I did not recall the name) there; I believe it [was] the manager. During our call; I recounted what had happened on the flight. They informed me that on that initial hold at FL330; I climbed 150 feet high; and that I violated separation. This confused me; as I was operating IAW the AIM; chapter 6; paragraph 4-6-11; and my understanding was I had +-200 feet. I was at +150 feet; and informed ATC. They said 2;000 feet of separation was required; as I was non-RVSM and there was opposite direction traffic at FL350. What seems strange is that means that in order to comply with that separation; I would have been allowed +0 feet on my assigned altitude; which I don't understand. I was told it was a violation of separation; and was not acceptable; and would be forwarded to the FSDO. Our conversation continued to the engine situation; and they informed me that the first controller never heard my call that I had an engine surge. I was informed that my 'communication was lacking'; and I informed the controller that my priority at the time was to execute the appropriate procedure; fly the aircraft to avoid slowing (by trying to hold altitude); and determining the suitable airport to fly towards. Communication with ATC is important; but I was triaging; and communications were lower on my list of priorities; as ATC would not have been able to help me with the engine problem. The controller also informed me that the briefing I had with the ZZZ Center Ops Manager never made it to ZZZ1 Center; so they didn't have the understanding of just how much of a 'test' the flight was; or the expected emergency/abnormal actions. To [the controller's] point about communications though; the right call might have been to have just [requested priority handling] once it was clear that ATC didn't understand what was going on. This was likely afailure on my part. Not to shirk responsibility; but I will admit; that I was hesitant to do so; because -- 1) This was a pre-briefed item; and while it required immediate action; I did not need assistance aside from a descent and a turn. 2) Multiple friends of mine have had minor things occur (rejected takeoffs; near misses with some small balloons; etc.) and then had a significant amount of FAA scrutiny as a result; the FSDO would ask for all their paperwork; and conduct an interview and review of the flight -- almost like a 'super ramp check'. This policy unquestionably results in pilots being more hesitant to alert ATC to issues they are having.To summarize the issues during this flight:1) I did not/do not understand altitude limits when operating a non RVSM aircraft through RVSM airspace? I thought it was +- 200 feet. ATC's assumption of a 0 foot tolerance resulted in a loss of separation.2) The briefing I conducted with the ZZZ Center never made it to the later Center controllers; so they did not know what to expect.3) I should have [requested priority handling] once I realized that I was saturated from a task loading perspective.4) ATC should ABSOLUTELY NOT have told a heavily task saturated pilot; dealing with an engine problem in a single engine aircraft on a flight test; to copy a phone number. This was the least critical thing to communicate; and adds to a pilot's stress and workload.Thanks for taking the time to read this; and for helping to keep us all safe!