Setting it up with a square gets you to within 0.002 over 6" on the fixed jaw.
Hard to say but even if the vise was perfectly square initially, its possible to introduce that amount of run out just tightening the bolts. By applying torque you are twisting on the washer which is displacing the vise. Even if you alternate tightening the heads progressively back & forth between 2 bolts, chances are good the displacement will be in the same collective rotation direction (vs opposing or cancelling one another) as one bolt kind of acts as a pivot for the other. What might help is 2 washers so they slip. My vise has a slot & uses Z shaped type hold downs which is more downward force vs torque. Even so its still possible to introduce a bit of junk.
Just a personal opinion but I just use the square to get it quite close, usually within the same range as you are getting. Clamp one bolt rather firm, say 60% of full tight & leave the other looser. Run the DTI across the jaw, bump the vise & dial it in, snugging up the looser bolt. If you dial it in as table is moving you can usually get it within one pass with a bit of practice. 2 or 3 for sure so it really doesn't take much time. I know a lot of people use milled keys in the T-slot & prefer that. But I find the key position sometimes don't land the vise where I want it, so I just clamp it where I want it & dial it in. Its maybe personal preference. Keys might have a bit more stability retaining the vise under heavier load/vibration, but I think only if it fits the T slots well on both front & rear edges. My current mill is pretty good but my RF-45 was pretty rough.