Agree with Susquatch, looks like something may be loose or not rigid. Reduce the stickout of fly cutter as much as possible. Tighten up the ways a tad bit more.
When you say fly cutter, are we talking single point HSS tool type? Tool ground properly?
Too large of tip radius can cause chatter.
"
Point Radius
Professional machinists maintain the point radius of the fly cutter tool to less than 1.5mm. This is because the smaller the point radius, the slower the feed rate, thus giving a fine finish. Thus, make you avoid large tip radiuses as they increase the tool cutting pressure, causing deflections. This eventually results in squealing, chatters, and poor finishing outcomes.
"
When I use a single point flycutter I prefer to have the tram slightly out. (I'm not sure if this is right). The leading edge of the cut is very clean. The trailing edge cut is just brushing the surface. Maybe try working in the direction of the low side of your tram.
Just a few amatuer thoughts. cheers, P.