jkust, I'd like to share a brief (true) story with you that your situation reminds me of.
Once upon a time there was this 22-year-old kid that purchased a '94 Larson Cabrio 280. It was dirty and clearly had lacked care and attention for a while, but seemed to function fine, and with the marine surveyor's thumbs-up, the kid went ahead with the purchase. The first year he had the boat, much like yours, it had some issues with sloppy shifting. Sometimes it'd take some coaxing to get it into gear, sometimes it'd take some coaxing to get it out of gear, but with some caressing it seemed to get along just fine. The kid had the shift cable, shifter, etc looked at by a few professions, and the issue would seem to go away and come back with every mechanic that looked at it.
Fast forward to the 2nd year of ownership, the kid takes a nice ~45 mile trip north on his beloved boat for a long weekend in Door County. Upon arriving at the destination marina, he's passing nice, new, large and expensive yachts as he approaches his transient slip. When he goes to throttle down, however, the boat continues forward. He pops it into neutral, and the boat continues forward. He jiggles to forward and back to neutral a few times, but the boat continues in forward with a bit of haste as the dock (and stone / cement breaker wall, and yachts that cost more than he'll make in a lifetime) approach quickly. With trepidation, he throws the shifter between reverse and neutral and forward and reverse before abandoning the attempt to get it out of gear and killing the motor all together.
Luckily, all of his marina neighbors had made the trip North a few hours earlier and, despite being rather intoxicated, were able to line up along the dock and catch the 28 foot boat without any harm being done. After a tow to the boat yard, haul-out, and complete disassembly of the out-drive, they found massive corrosion of internal components, missing parts (some prior idiot had apparently tried to work on it), loose bolts, etc...
Moral of the story, if it's a new (to you) boat and has been doing this from day one, it might be worth tearing apart for a proper inspection.