Fractional Captain reported a descent below the minimum altitude on the initial approach. The Captain states the Pilot Flying/First Officer descended when believing they had been cleared for a visual approach.

Narrative

We were operating a flight into ZZZ from ZZZ1. Upon approach; multiple cloud layers were reported; with the lowest being a scattered layer at 2;500 ft. The winds were favoring landing on Runway X so we chose to execute the RNAV to Runway X until we could pick up the airport visually. We were given direct to the ZZZZZ fix to intercept the approach. At this time we were at 3;000 ft. and still did not have the airport visually. We were then given a clearance to maintain 2;000 ft. or above until ZZZZZ and we were cleared for the RNAV X approach. 2;000 ft. was set and seen in the altitude selector and the Pilot Flying (PF) started a descent to 2;000 ft. so we could pick up the airport visually. I then made a position report on the CTAF on Comm2. While I was off giving the position report we had entered VMC conditions. I returned to COMM 1 and told the PF I was back and I was looking for the airport. I glanced down to see how far away the airport was and I noticed we were level at 1;600 ft. outside of ZZZZZ. At this time we were not cleared for the visual and still on an IFR approach. I informed the PF to get back up to 2;000ft. We then saw the airport and quickly cancelled IFR about 8 miles from the airport. I asked the PF/First Officer how did we get down to 1;600 ft. The PF seemed confused and said as we came out of the clouds he thought we were on a visual approach and not an IFR approach and went down to 1;600 ft. for traffic pattern altitude. I had missed this loss of 400 ft. of altitude during a very task saturated time.