After reading an article on the Arch wiki, I came up with a more permanent fix
This uses udev and the systemd hdwb hooks provided by most upstream distros
This will work fine on the latest Ubuntu, Fedora, Arch etc
Create /etc/udev/hwdb.d/90-purism-pipe-symbol-fix.hwdb with these contents:
evdev:atkbd:dmi:bvn*:bvr*:bd*:svnPurism:pnLibrem13v2*
KEYBOARD_KEY_56=backslash
Then execute
sudo systemd-hwdb update
sudo udevadm trigger
Now the | \ key below backspace (backslash) should work as expected permanently