I see very little carbon on the Pistons Crowns. If the Rings are have Tension and spring back when pushed, the Grooves are likely free of Carbon.
How long have you had this Motor?
Was there any water coming out of the Prop, when you were running it on the Muffs?
Water coming out from around the Spark Plug, means a bad Jacket Gasket, or more likely, in My Opinion, a warped Water Jacket Cover, which happens when the Motor gets too Hot
They really haven't been used much since the early 80s, as most shops acquired Dynos, which can vary the Load place on the Motor. Have you tried Evil Bay?
It wouldn't be revealed in a Cylinder Compression Test, as Air can enter the Cylinder via the Exhaust Ports, instead of via the Crankcase. Crankcase Compression Ratios are even lower than Cylinder Compression Ratios, and Maybe a Reading can be obtained off a Fuel Pump Port for one of the Crankcases.
In 76, I think OMC was using the Ford 5.8. Switching from the OMC Stringer will require extensive Transom Work(at least 10 layers of properly layed Fiberglass) as the Transom will be bearing the strain of the Thrust. New Holes for the Mounts will likely be required.
You should get ahold of the...
50mph from a 5.7 in a 25ft boat is asking a lot, maybe attainable with a Big Block.
That engine has a WOT Range in the 4800-5200, so if 4200 is what you attain, then it appears to be over propped, or the engine is not up to snuff.
Do what racer suggested, and/or some basic checks, like...
I find Black Sabbath, Motley Crue, Iron Maiden, and AC/DC to be the Music that relaxes me, and relieves Stress. Rap, gets me to wanting to destroy Things, Violently
