havent used handbrake in years, but there must be a way to see what commands it is sending to FFmpeg, just copy those parameters and use that if you want it to be exact. Seems a lot of work though and kinda reduces what you can do with FileFlows, or makes FileFlows a lot more complicated.
Also very likely to be using two different versions of FFmpeg, FileFlows docker ships with FFmpeg 5.0.x.