Purism Smartcard: gpg: KEYTOCARD failed: Bad secret key

I’m trying to move my gpg keys to the purism smartcard.

I ran:
gpg --expert --edit-key email@address
typed 1 to select the account, y to move the primary key, 1 select to store the key in Signature Key. It asks for my passphase and then says bad secret key. Can someone help understand why this is not working?