The starter relay (not technically called a solenoid when mounted remotely) is stuck closed. Give it a sharp strike with a screwdriver handle and it should unstick it. You'll find it on the fender apron right behind the battery. The short positive cable runs to it and then a long one down to the starter. Get it replaced asap.
Here is an
one so its top dollar, should be $15 - $20 at your local AP store or RockAuto.com