The intention is that that statement applies to all devices with old, abandoned, unpatched software, whether online or offline.
However you also need to consider that some devices must be used online in order to pick up a correct time at all. (That would not typically apply to a desktop, however, unless the button cell battery inside it has long gone flat.)
As this bug is inherent to old UNIX/C environment software, it may not apply to Windows at all. Old Windows has its own death dates. So on Windows you should really only experience the 2038 problem if you are running software that was written in C (and which is old enough not to have been adjusted to a newer API).
As Windows is blackbox, there is no way to be sure. I guess if you are keen, set the system time to the correct date in 2038 and watch it roll over. Since your computer is guaranteed offline, it has no way to pick up the correct time, so it should persevere until it passes the problematic moment, one way or another.
The problematic moment (the so-called epochalypse) is
date -u -d @2147483647
Tue 19 Jan 2038 03:14:07 UTC
which you would then need to adjust into local time for your timezone.