Alright. Went back to the system. Found a 720K 3.5" floppy so I could do some screen saves from the old TEK scope. Let's start here: This is the version I'm running with screen shots from the actual hardware.
Now let's take a look at the screens. Notice I have Pin 1 set as STEP and Pin 17 set as DIR. On the BoB Pin 1 goes into the PWM circuit to create the 0-10V and pin 17 runs a relay that can be used to switch motor direction. Much like what you have.
Next let's go to the Outputs screen and make sure nothing is set.
Finally in the Spindle setup screen we have disabled Spindle Relays, Use Spindle Motor Output with PWM control. PWM base frequency is set to 25kHz.
Apply and save and then from the Config menu choose Pulley Selection. I want Pulley # 1 to be the current pulley with a max spindle speed of 3000 RPM and that's the motor RPM since the Ratio for the pulley is 1:1
Now let's run it at 100 RPM in the CW direction. The relay on the little board is OFF. The top yellow trace is the PWM signal (active low from 5V to 0V) showing a very small pulse for 100 RPM. The direction signal is the blue one and is low or 0V.
If I type in an M4 on the MDI screen I want to reverse the direction. The relay comes on.
And the direction (blue trace) changes to logic 1 (5V).
Now let's go back to clockwise which is blue trace low and this time 2800 RPM which is almost continuously ON for the PWM (logic low since it's inverted) with just a short time OFF (logic high)
At this point as long as there is a module connected to the PIN1 with that 5V PWM I can create 0V to 10V which this little BoB does. But I could use an external module too. Or in the case of the Gecko it routes Pin1 through an optical isolator over to the PWM to DC circuit and puts it out on VFD_PWM with the supply for the isolator coming from VFD_GND and VFD_10V.
The scope leads in the above samples are on PCGND and the buffered PC Parallel Port signals.
Maybe this will clear things up?