Air carrier Captain reported selecting an improper altimeter setting as received on the ATIS from ATC; resulting in a low altitude alert while on an instrument approach in mountainous terrain. The crew leveled the aircraft then continued the approach; and after landing; ATC updated the ATIS to the correct altimeter setting.

Narrative

Before the descent; I got the weather information for ZZZ. At the time it was ( I 280 08 kt 10 sm Clear below 12000 temp 7 dewpoint -2 altimeter 29.93). We were cleared to descend to 15000 then down to 10500 then down to 7800 for ZZZZZ. then we got cleared down to 7000 feet for the final approach fix. We leveled off about 7 miles prior to ZZZZZ1 and were fully configured 3 miles prior. we made sure the GPS approach was activated which i then calculated our decent to be about 800 feet per minute. After starting the decent .2 prior to ZZZZZ1 and making sure we were stable and clear to land we got a Radar Altimeter read out of 1000. Our altimeter was still showing about 6800 on the altimeter. I assumed it was due to the rising terrain and would be going back down. once it got to about 600 feet; I had the FO disconnect the auto pilot and level the airplane. While level and correcting the altitude Tower called us and issued a low altitude alert. I responded as we were aware and already fixed it. Upon landing on Runway XX and taxiing into the gate. We thought there might have been an issues with the Radar Altimeter. While debriefing at the gate we heard a Cessna 172 ask Tower what the altimeter setting which they responded to 29.93. My Self and the FO both looked at the current elevation for ZZZ and it should have been showing a different elevation. At this time I called Tower and asked them for a current altimeter setting and they came back with a 29.39 and said they would be updating the ATIS.Cause: The contributing factors of this even were the incorrect altimeter setting being recorded for the ATIS.Suggestions: The suggestion I would have is that who ever is working in the Tower be aware of the altimeter settings when they are recording them or typing them into the computer. Next time when a setting comes back that different from where we took off from I will continue to double check verbally with the Tower.