For filling the old holes on that boat:
I would cut some hardwood dowels to length just less than the core wood thickness of the transom, coat them with an epoxy putty (like PC-11, MarineTex, or homemade), then tap into place so that they more or less match up on the wood. Then fill the holes in the fiberglass "skin" (inside and outside) with the same epoxy putty. Stick some blue painter's tape over the epoxy putty so it doesn't "sag out." Let cure, remove tape, then sand.
This of course assumes the core of the transom is in good shape. And you don't mind visible "epoxy spots" showing on your gel coat.
For the repower, a few hints/suggestions:
If you're drilling the new holes yourself, I'd strongly suggest making, buying or borrowing a transom drilling jig. These holes have to be nearly perfectly located and straight-through, or you'll have a devil of a time sealing them. If you're hiring someone to do it, make sure THEY have a transom jig!
Unless you are 100% certain what your new motor's height needs to be (i.e., which holes on the mount bracket to use) beforehand, seal the new hardware with something less permanent than 3M 5200 on your initial mount-up (something like 3M 4000 UV, or Loctite PL Fast Cure Marine Sealant---same stuff). This way, if you find you do need to adjust the motor up (or down) after a test run or two, it'll be a far easier job for you to remove the old sealant. Once you're dialed in on motor height, you can swap to 5200 if you want to...but IME, the 4000 UV / Loctite PL is fine for this application, it's what I use.
