If the authenticator is using TOTP (or HOTP) then in principle it does not need WiFi to be enabled (or any other network connectivity) in order to complete the authentication.
I appreciate that an Android phone with blackbox firmware might not truly allow you to disable WiFi. However you can at least keep it off the WiFi by not giving it the (correct) passphrase - and hence keep it off the local network.
By definition, you wouldn’t put a SIM in the phone but, again, without knowing what evil the Android firmware might get up to, you can’t be certain as to what leakage might be occurring anyway.
Rather than locking it in a drawer, you might keep it powered down when not needed but, again, without knowing what evil the Android firmware might get up to, you can’t be certain as to what leakage might be occurring anyway.
Of course, if the authenticator only needs TOTP/HOTP then you can safely run those algorithms on the Librem 5.
For the moment I am doing what @StevenR suggests (except it is an old iPhone, rather than an old Android phone).