So we don't perceive reality as it is, and therefore there's no consensus reality. If there's no consensus reality, what is this thing we call consensus reality? Just packets of energy vibrating at different frequencies? Slightly uncomfortable thought to entertain, but also impossible to deny!
We see a street scene one way and assume everyone sees it the same way. But if you have eyes on either side of your head, this* is what it looks like. Well, probably not, but we can rightly assume it's a very different construct to what we might each perceive.