A TRACON Controller reported an aircraft descended below their assigned altitude and flew below the Minimum Vectoring Altitude.

Narrative

Aircraft X was issued 1;700 ft. in a 1;700 ft. MVA. The aircraft was direct ZZZ requesting a visual approach. I stated that 1;700 ft. was the lowest altitude I was going to be able to give him and to have him a position report of the airport and instructed him to report the field in sight. The pilot said 'Roger'. I observed the aircraft descending and issued a low altitude alert with the appropriate phraseology; stating what the MVA was in the area and to ensure that he was at an altitude of 1;700 ft. The lowest I observed was 1;300 ft. about 10 northwest of the airport. He responded by climbing and saying roger; adding that he was VFR in the rain. He then came back shortly after and said he would like vectors for the GPS approach into ZZZ and that the Tower was advertising a ceiling of 5;000 ft. but he could see the field. I responded by issuing a climb to 2;000ft. and a vector to the final approach course.In my opinion; this was strictly a pilot error. I should have been clued in to watch him like a hawk even more so because on check in from ZZZ1 Approach; he checked in saying he was turning direct ZZZ (not the flight plan given by ZZZ1) to avoid weather; which a pilot can't just do without requesting a course deviation first. I just immediately responded 'cleared direct ZZZ' to alleviate any issues on both ends. The initial check in was also missing his assigned altitude which is required by the AIM. There were a couple of what I thought to be slight course deviations when he was nearing the airport (again without asking) but as the plane was at 020 at the time and no other airplanes around; and the deviations weren't that noticeable; I didn't take issue with it. When I issued 1;700 ft.; I thought me saying 'that's going to the LOWEST I can get you... report the field in sight' that that would have been an extra heads up to the pilot that he can't descend any further.