See Tips & Tricks · Wiki · Librem5 / Librem 5 Community Wiki · GitLab under Geary. I think I added that tip. It definitely works for me.
Check config file as mentioned at POP3 -for- email - #26 by irvinewade
I am using a non-default port number for both incoming and outgoing.
Also be aware of a current problem: Keyring unlock / Email crashes (hopefully fix coming soon).