I just removed the motor pulley from my (very similar mill-drill). The pulley really did not want to come off. I heated the pulley with a propane torch to probably around 100C. It still needed a puller, but it slit off quite easily at that point. So my pulley had a very slight interference fit.
Why am I doing that? I'm looking at replacing the single phase 2Hp motor with a 3HP 3-phase motor and VFD that I have.
I am going to end up doing very much like what is being suggested that the TheLocalDunk do. The current motor and pulley have a 24mm shaft with a 6mm keyway. My "new" motor is a 7/8" shaft with a 3/16" keyway. I'm going to machine out the pulley a bit, then make an aluminum insert with a 3/16" keyway slot (that I will probably shrink fit with heat) into the now larger pulley hole, then re-install it. The set screws will the be drilled and threaded through the insert and provide the equivalent of locating pins.
It seems a 0.0010 to 0.0005" interference fit for the modified pulley on the motor shaft will be my target dimension. A little bit of heat will be required to install or remove it, but it won't wobble, and shouldn't come loose like that.