Oh, sorry for not mentioning it. I already did a reflash with byzantium before starting to set it up.
I also saw on https://docs.puri.sm/Librem_5/Known_Issues/Birch.html this entry:
Call audio routing is not functioning completely.
Not sure how up-to-date this information is. According to Troubleshooting phonecall problem on Librem 5 Birch, it can work on Birch.