matrix-rust-sdk should help a lot with the backend performance and GTK 4 with hardware acceleration should speed up the interface a bit in desktop Linux. It sounds like it will be a while before we get hardware acceleration for GTK 4 in the Librem 5.