Hi Amstram,
I use a very broad approach here, really depending on what I need.
USB drives only used on one machine (e.g., backups):
I use LUKS to encrypt the whole drive. I have full system backups that way, but also three small 64GB keys (Amazon link here). For these keys my strategy is the following. They are used as backups for my most imported documents and my KeePassXC database. The drives are all LUKS encrypted and the documents are in addition inside a veracrypt encrypted storage file. All three sticks are redundant, are regularly backed up and stored in different locations (home, office, my bag). This way, even if my house, my backup, and my computer burns down, I have the most crucial things still available.
USB drives for various OSes:
As you already pointed out, this depends a lot on your budget. I use an Apricorn Aegis SecureKey for work. Justin Carroll wrote two in-depth review on the SecureKey (here) and on the Fortress secure SSD (here). I can highly recommend checking this out. I only have experience with the SecureKey. In summary, it’s an extremely handy device with some features that I especially like: the configurable brute-force protection and the self-destroy PIN. These devices are fairly pricey, but work very well on all operating systems and don’t require software.
If you would like to go the (much) cheaper, but less convenient way, consider encrypting your devices with veracrypt. You will need the software installed, but you will have excellent protection for your data.
Hope this helps and happy 2020 to you as well!