Controller reported failure to hear altitude readback of pilot resulted in aircraft descending below minimum vectoring altitude.

Narrative

I took position for the first time to start my week. I had a few military aircraft that had asked for approaches. We brought in a final Controller to work them and I continued to work the rest of the aircraft. Traffic picked up significantly and quickly. Aircraft X came in from the north for ZZZ. I descended aircraft to set them up for the visual approach. The ceilings were dropping but below 3200 ft. visuals had been working well. I sent them direct a fix (ZZZZZ) that was on the RNAV XXR so as to keep him away from an antenna that was northwest of ZZZ. I then descended aircraft to 2500. At this time I was working several arrivals into ZZZ1 and some overflights. I scanned to see aircraft at 2200 ft. about 7 miles north of ZZZ (our MVA is 2500). I doubled checked that I did not clear aircraft for the approach and then asked if they had the field in sight. Pilot said there was no visual and did not have the field. I climbed the aircraft back up to 2500 and turned him to avoid ZZZ airspace. Aircraft climbed but did not take the turn correctly and was flying somewhat erratically. I asked if they could accept the RNAV approach since they were unable to get the field in sight. Pilot said they could so I climbed the aircraft to 3500 and began to vector them for that approach. I cleared the pilot for the approach and then observed aircraft deviating significantly from the course. I asked if the pilot was correcting and pilot replied 'that they were having issues pulling the approach up'. I then realized that they were not familiar or comfortable flying the approach or possibly even IFR. I then told them that I would vector them to the field and had the supervisor call ZZZ to tell them that they were having issues so I was keeping them longer than usual and may go into ZZZ airspace. 5 miles north of ZZZ they requested to cancel IFR so I did that but kept the aircraft on a vector since they were still having issues seeing the field. I pointed out several landmarks for them and pilot finally saw the field on about a 2.5 mile final. I switched him to ZZZ Tower and they landed safely. I was later informed that when I initially vectored aircraft for the visual approach on the first try that he read back 2000 ft. instead of 2500. Pilot later told the supervisor that they were unsure what I said so guessed and I did not catch the incorrect read back.Recommendation: Increased importance on better active listening. Education to pilots that if there is confusion to an instruction that they should ask clarifying questions.