EMB-135 flight crew reported a GPWS warning on approach.

Narrative

We were on a visual approach to Runway XX (the box above only gave me the option for Runway XY/XZ). We were following the PAPI guidance and at about 400 ft. AGL we began to get 'Too low; Terrain' warnings from the GPWS. We were confused by this because we were in day VFR conditions; on the glide path with the runway in sight. So we decided to continue the approach and landing because we were not in danger of hitting any terrain. After we landed and rolled clear of the runway we realized that that our altimeters were still set to 29.92. This caused the GPWS to think we were lower than we actually were and triggered the warnings.After debrief with the First Officer we realized that several factors contributed to this event. We were going into an unfamiliar airport at a high field elevation of 7;000 feet. This put the 10;000 feet AGL checklist point at 17;000 feet MSL; so we accidentally missed the second half of the 'descent and approach checklist' that contains the setting and cross checking of the altimeters to local altimeter setting. This is normally completed at 18;000 feet. We went straight to the '10;000 feet AGL checklist.' We were also looking visually for the field and other traffic at this point; especially because it is an non towered airport. We were planning on making the visual approach and were looking for landmarks so we could cancel IFR before losing communication with ATC in the mountainous terrain. These factors of a non standard arrival and approach into an unfamiliar airport contributed to us not realizing that we had not completed the second half of the descent and approach checklist. Proper performance of all required checklists per SOP would have prevented this event.