Whenever I have my Mustang running, and I turn the key off, the car will keep running. Then If I take the battery out of it, which I'm kinda sure cant be completely safe to do while the vehicles running, it will still keep running. So I'm pretty sure my alternators good because of this. But, usually you need to turn a car off, so what is it? FYI, I had to brake the ignition over into the start and run position after buying the car to be able to change the ignition because the car didn't come with a key. So could the Ignition switch be jacked because I did that?

 

You need to buy a new ignition switch.

A fast test is with a meter or test light, clip on the positive post of the coil and with the key "off" or even with the key out of the ignition switch. You should not have any voltage. Now turn the key to the "run" position", not "start", you should have 12VDC at the coil. Then put the key back to "off" and power should be gone, "0 VDC". If the car was "running OK" before you broke the switch then I would start by replacing it.

 

there's plenty of opinions about if you should disconnect a battery on running car. i do it on old ones to see if the alternator is working. (some will disagree). anyway, if the ignition is in 'on' position it will keep running with battery out. no surprise.
do what LCC suggested

 

I had turned the key to off before I took the battery off though.

 

you can check at coil+ for voltage like LCC said. check with voltmeter at coil+. if you have 12V+ at coil with ignition switched off, then it will explain your problem and
a) your switch or wiring is bad
b) the coil won't live for much longer