If a gateway is using NAT, then ESP packets won't get through because ESP packets have no ports for NAT to translate. Use NAT-T to encapsulate the ESP packet into a UDP packet using port 4500 as a source and destination port that NAT can now use to translate.