A320 Captain reported loss of both ELACs after take off. The flight crew performed an air turn back and precautionary landing at departure airport.

Narrative

Aircraft X I was operating ZZZ-ZZZ1 in Aircraft X. We departed Runway XXL on the ZZZZZ departure with a climb via SID clearance to 17;000 ft. Push back and taxi out were normal; takeoff roll was normal. Just after liftoff as the gear were retracting I noticed the amber 'X's appear on the PFD as the aircraft reverted to alternate law. There were no immediate ECAMs. We cleaned up the aircraft and accelerated to 250 kts. and asked to level off at 5;000 ft. to fly present heading to trouble shoot. The First Officer began running the ECAM actions while I communicated with ATC and the Flight Attendants. There was only an ECAM for alternate law; there were no other ECAMs at this time and there were no fault lights on the overhead panel. When we got to the status page is showed ELAC 1+2 as INOP systems. We made the decision to return to ZZZ as it wasn't clear why the ELACs had both failed. We also [requested priority handling] to get back to ZZZ as well as in case we had any further flight control issues as we weren't sure why both ELACs had failed at the same time. The weather was VFR conditions. We set up for a visual approach backed up with ILS to Runway XXL as that was our return runway in the secondary flight plan as well as the longest runway. We sent an ACARS message to Dispatch advising them of our situation and communicated with the Flight Attendants and Passengers. Landing in direct law was uneventful and after landing CFR checked out the aircraft and we taxied back to the gate uneventfully. After we had arrived at the gate and ran the parking checklist the Flight Attendants in the back of the aircraft reported that they had heard a banging or thumping sound in the back of the aircraft while we were flying.