The solenoid has a control circuit and a high current set of contacts. If you installed the solenoid and nothing happened till you hit the key and you either heard a clicking noise come from it, or your starter started to spin, the control circuit will also de-energize when you switch to the OFF position. So don't be chasing that fox.
The high current loop is a pair of ⅜" studs with large flanges on the interior ends. A large copper disc is connected to a spring loaded shaft that goes down through the control coil. Switching the control to the START position energizes the control coil which sucks the rod down inside it causing the big copper disc to connect between the studs, making a high current connection capable of a couple hundred amps or so. When the coil voltage is removed....you release the key and it goes back to the ON position from START, the coil de-energizes and the spring forces (is supposed to force) the plate to separate from the two studs and stop the high current. That's how the solenoid works.
If you have a new solenoid of a different type, any differences will be in the control circuitry, not in the high current portion. The changes will consist of how the coil is wired. Usually you have 2 options:
There are 2 isolated small terminals on the side of the thing and 12v and ground are applied across these terminals when starting. There is NO electrical connection between these terminals and the metal/plastic housing.
Some installations in all sorts of other things, have only one small terminal and the case is in electrical contact with the battery (usually - terminal) through a mechanical means. This type only requires one wire with 12 v when desiring to start the engine.
On well used solenoids, the high current contacts develop pits over time from the arc that occurs when you open the contacts when you are through operating the starter. These pits cause a reduction in the surface area of the circuit and cause excessive heat which in time will be adequate to weld the contacts together meaning that the spring will not be stout enough to open the contacts when you switch out of the START position and will continue to power the starter. If you have this problem with a new solenoid, regardless of the type, brand, what have you, you have a malfunction and need to get your money back for a defective unit.
In short, if you can turn the new solenoid on, you can turn it off unless there is a malfunction in the high current part of it and that is a defective unit, not a properly designed solenoid.
For your NEW solenoid powered starter to continue spinning when you let off the key says the solenoid is still getting drive power which means you shorted something out in the replacement....mis-wired something. Unless the starter is dead shorted and your system has the capability to supply something like 500 amps or so to the shorted starter which could weld the solenoid contacts shut, the starter can't hurt the new solenoid that quickly.
HTH,
Mark