SIM swapping is more a vulnerability at the carrier level than the phone hardware or software level. An attacker convinces a cellular carrier they are a particular customer who got a new phone, and they are able to get them to move an existing number over to a new SIM. From there they can receive SMS and calls to that number and they often use that access to bypass 2FA that happens over SMS.
Many cellular carriers put additional authentication steps and other safeguards in place these days to try to defend against this, but all it takes is an attacker who is good at social engineering to convince the right customer support employee.