I just had an issue w. my starter. I thought it was going bad, it would try to turn over then you'd just hear a loud "Click...click..click...". I was going to swap it out for a backup starter I had. When I pulled the starter off the engine, I decided to re-check it. Worked fine when removed and with jumper cables hooked up to it to a battery. Turns out that the face where the bolts mate with the engine block was rusty. Water was dripping down the engine block (either rain water or manifold.. I'm still investigating..) The water seeped right into that gap and caused it to rust. Apparently this is what the starter was using to ground itself.
Long story short: Make sure the starter has a good ground. I'd use one side of a jumper cable to ground it when trying.