quite frankly you should be seeing about 13-14.5 volts at idle . If you are seeing less than 12v at idle then your not keeping your battery charged. This leads me to believe that your voltage regulator is shot or you got a bad alt.
Starters pull more amps when they get hot due to resistance. A hot starter has more resistance than a cold starter. This would strain a weak charging system.
A bad starter on an other wise good charging system would have the same effect, as it heats up resistance builds to the point where it pulls more amps than the battery can handle.