If Mozilla gave you a result (valid approximate location) then maybe your data had already gone long before the Librem 5 arrived.
So it took your wireless access point’s MAC address, or any visible MAC address from any one of your neighbours, and sent that to Mozilla and Mozilla gave a location back, which it must already have known i.e. the location associated with that MAC address - since by your own statements there was no SIM, the IP address was obscured by a VPN and GNSS is not working yet i.e. no other means of determining the location (and hence no way to report it to Mozilla either).
You can try to maintain your own privacy but you can’t force your neighbours to care about privacy.
So the only way to manage that is either a) enough shielding so that you can’t receive a signal from the wireless access point of any neighbour, or b) to be far enough away from your neighbours.
Even someone who visits your house and carries a spyphone (Google or Apple) and runs an appropriate app can compromise the location of your WAP. Do you check all your visitors and ask them to leave their mobile phone in a shielded box?
According to Wikipedia, Mozilla has 1450 million WiFi networks in its database. Yours or that of one of your neighbours may already have been in that database.