A Center Controller reported they lost communications with a military UAS and it descended below its assigned altitude.

Narrative

I was in the middle of a fairly complex briefing with a fair amount happening during it. Aircraft X was holding in the ZZZ airspace at flight level 510B590 and asked to return to base. I told Aircraft X to standby and dealt with several other aircraft requests and continued the briefing. My relief controller said that he would handle Aircraft X request. Aircraft X called again stating they were 'leaving FL520' (still within the assigned block); again I told them to standby. After the briefing finished the new Rside noticed that Aircraft X was descending through FL480 without his clearance. The new Rside cleared Aircraft X back to ZZZ but never got a readback. I told the Supervisor and plugged back in as the D side to coordinate either a Lost Link or a NORDO with ZZZ approach. They said that the have been having issues with that aircraft and that they would block the airspace. We later established comms with Aircraft X; while the Supervisor attempted to make contact with the ops phone number and the pilot's phone number in the remarks.Military aircraft and the drones have notoriously bad radios; needs updates and revisit the RTB (Return to Base) clearance and procedure.