It's not an uncommon problem in classics, stock ****ty starter with headers...starter gets hot and the rotor rubs and it loses torque trying to turn itself over.
FWIW I run a Powermaster starter and mid length headers, for ground clearance. The more power the starter has, the longer it lasts turning any given engine over.
Who cares how much horsepower it has, all that matters is how fast it goes!
Untested 331, lots of suspension, chewing up corners.