After way too long trying to find a solution, VelOCRaptor now deals with multi-hundred page PDF documents, if you have the patience!
The issue turned out to be 2 leaks, one in the code that writes extracts images from PDF files to feed to the engine, and one in the code that writes PDFs from those images once the reading is done. Both were easy to diagnose, but hard to fix, as they were symptoms of bugs in Mac OS rather than my code. But nobody cares - you just want software that works - and now it does work a lot better.
[Edit] You can download the update (it is build 195) by using 'Check For Updates' on the VelOCRaptor menu