I'm on a similar trajectory with a Teensy 4.1 and a custom controller board running grblHAL - the 32 bit version of GRBL driving my 5 DOF gear hobbing machine (more later).
With the T4.1, level shifters are required for most peripherals (LCD, OLEDS, some I2C devices, etc) I purchased the shifters from Amazoom for about $1 each and mounted them on a veroboard with spring terminals which is a great option when breadboarding.
I made my controller some time ago, with 4 pin GH1.25 connectors for a very neat wiring setup. This board exposes most of the ports and allows up to 4 steppers. I also purchased a T4.1 carrier from Phill Barrett (via Tindie.com) which uses screw terminals for connections. Here are pics of both boards: