A TRACON Controller reported an aircraft descended below its assigned altitude and flew below the Minimum Vectoring Altitude.

Narrative

I received a briefing from the controller on ER and took the position. Seconds after sitting down; I observed Aircraft X at 051 in a 045 MVA (Minimum Vector Altitude); but about two miles from and pointing at a 056 MVA. I thought I was briefed that he was assigned 070 but I could have been mixed up; so I quickly asked the relieved controller 'Is Aircraft X on a visual?' to which he responded 'No I gave him 070.' I issued Aircraft X a climb to 056 for terrain; which he read back. He entered the 056 at 052 and climbing. In the moment I chose not to issue a safety alert. Review of the tape confirmed the prior controller did issue 070 and the pilot read it back.The 7110.65 on safety alerts says:2-1-6. Issue a safety alert to an aircraft if you are aware the aircraft is in a position/altitude that; in your judgment; places it in unsafe proximity to terrain; obstructions; or other aircraft. Once the pilot informs you action is being taken to resolve the situation; you may discontinue the issuance of further alerts.'a. Terrain/Obstruction alert. Immediately issue/initiate an alert to an aircraft if you were aware the aircraft is at an altitude that; in your judgment; places it in unsafe proximity to terrain and/or obstructions.'In my judgment; the moment I issued the climb Aircraft X was not in an unsafe position; nor was he in an unsafe position when he entered the 056 MVA as the highest obstacle was an antenna 4+ miles from his position and 600' (and increasing) below him. I believe issuing a low altitude alert in this situation would have potential to cause further panic in the cockpit over what I view as a relatively minor incident. Recommendation: Mistakes happen and I believe this was an honest mistake by the pilot. As I'm not a pilot I don't feel qualified to make suggestions on how this could be prevented.