Here’s a thought… (I don’t know enough about the internals of your software to know how easy it would be). Why not let a user register a PGP public key to encrypt everything not already encrypted? It doesn’t solve the potential problem of state actors seizing the server and reading the messages before they get encrypted, but it’s already the case that a compromised server could read the DMs before they get round-filed.
Using ascii-armored PGP would also let federated users successfully send DMs, if they go to the effort of PGP encrypting them first. Unencrypted DMs should generate a bounce message warning that they are insecure (and require an opt-in on the user’s part to accept them, since they have to provide the public key).
Just spit-balling here, but there’s a truism with web standards that “he who ships first sets the standard”. I think if your Mastodon server handled PGP encrypting DMs on behalf of the sender on the way in, that could be leveraged into getting other Mastodon servers to encrypt DMs on the way out. As well as support for one-click decrypting PGP messages within various Mastodon clients.