Hardware is a large part of it. Purism is putting a lot of work into making sure the phone would qualify for the Free Software Foundation “Respect Your Freedom” (RYF) certification. It’s not just better hardware, it’s hardware that is engineered to give you a secure device. The Pinephone makes no such effort.
Purism has also put a lot of work into software development. The Pinephone has put, as far as I’m aware, no effort into software development (according to their homepage, it runs mainline Linux and “anything else you’ll get it to run.”) All of their efforts in this area consist of taking existing OSes developed by other companies/organizations and making them work on the phone. That’s not to say that this approach is wrong, but it certainly doesn’t guarantee a functioning (or supported) phone once the customer gets the hardware.