I think you are right in the above quoted statement for that case. Since all possible keys can be tried (by the quantum computer) but all possible keys will also yield all possible outputs (including mostly meaningless outputs but also all meaningful outputs and including the actual plaintext).
The quantum computer will have no way of deciding which is the actual plaintext.
However even small changes to the algorithm could alter the balance. For example, many crypto protocols like to include integrity protection and/or replay protection - and those issues would need to be thought about. (I think replay protection is somewhat covered.)
Hence: with no integrity protection at all, an attacker can alter your data (AND do so in a predictable way e.g. flip a bit, one of the weaknesses of such a simple encryption algorithm). As soon as you put in integrity protection, that may weaken your resistance to QC attack, although it will depend on whether the integrity protection is inside the encryption or some kind of supplementary data outside the encryption (like a HMAC on the encrypted stream but that requires work of its own).