SCT Controller reported problems associated with the radar outages and attempted repairs.
Synopsis
SCT Controller reported problems associated with the radar outages and attempted repairs.
Narrative
SCT was the key site for STARS software build sct_s6r9d13_e4 on Date. After troubleshooting the initial problem at XA:00L with Interfacility Data Transfer (IFDT) requiring a mandatory keyboard entry; which we were not briefed on or informed would be necessary the decision was to proceed forward with the new software build. Throughout the day we and other key sites identified several issues with this software.While these issues are important; none stand up to the real safety issue of 15 remote Tower displays failing in various degrees at XL:26 on Date. This means the display went BLANK; which is everyone's nightmare. It has now been almost three (3) months since the failures and all we have received are verbal hypothetical (We THINK) scenarios as to what happened. This is unacceptable in the safety culture we work in day to day.At this point; it is almost impossible to know the amount of aircraft we LOST RADAR on due to these failures; let alone the emotional stress created on those controllers during the event.Summary of the 15 failed Air Traffic Control Tower processors on Date at XL:26z. This means a total loss of the TDW. The majority recovered automatically without intervention; technicians brought back some manually; and two (2) had to be completely restarted.RNM TDW failed and had to be completely restarted to return. Per the 7340-4 Daily Log; all the processors produced a save crash (snapshot of processor errors prior to crashing). The Tech had pulled Long Beach (LGB) and Los Alamitos (SLI) GCA so far.I am not a tech; but I would have thought with the safety concern that surrounds this issue someone would be at SCT troubleshooting; to determine what stressed the system to fail. Since they are unable to replicate SCT system at the tech center it only seems logical they need to come here to evaluate.
Source: NASA Aviation Safety Reporting System (public domain). Reports are voluntary submissions and are not verified by NASA.