Air Carrier Captain reported a Flight Attendant discovered there was a passenger who had inadvertently left lithium-ion batteries in a checked bag and the aircraft needed to returned to the gate to remove the battery from the cargo compartment to inside the cabin.

Narrative

Before cabin door closing; the Gate Agent checked in with me with the proper brief and informed me that there were 2 lithium batteries belonging to a passenger wheelchair. During pushback; the #1 Flight Attendant (FA) went to check with the passenger the actual location of the lithium batteries. The passenger tells the #1; that she does not physically have the batteries. After pushback and engine start; I find out that the wheelchair batteries location is unknown. We then returned to the gate to resolve the location of the batteries. The batteries were in the cargo compartment and not in the cabin. They were then brought to the cabin and given to the passenger to keep during the flight.There was miscommunication between the passenger and the Gate Agent of who is responsible for the custody of the lithium batteries during the flight. I don't think passengers realize the importance of not having lithium batteries in the cargo compartment.Suggestions: More education for passengers that have wheelchairs with lithium batteries. More education on the dangers of lithium-ion thermal runaway and why we want to have access to the batteries in the event of a thermal runaway hence why they should be in possession of the passenger during the flight.