The best way to control tooling costs is to not buy sets of anything.
Buy the sizes you need as you need them. And as noted, AliExpress is a great source. You can mill to inferial dimensions with metric milling cutters
Esp easy with a DRO, touch off in metric, switch to inferial for actual milling.
Hold downs/clamps, buy indivdual ones Teco brand from e.g. KTS Tooling (reasonable shipping), buy a few different length studs and nuts, make your own t-nuts.
Drills, buy a metric .1mm increment 1-6mm set and a inferial-metric cross ref chart, you won't need the number/fractional/letter sets. For larger drills buy .5mm increment.
Stick with HSS milling cutters, more forgiving and less costly to buy. and less costly to replace.