Yep, without the chopped strands, that thick of a layup will tend to crack as it cures due to shrinkage...
Also make sure you are not over catalyzing...
If your chopped strands are much bigger than say 1/4-1/2" in length, then take a little time with some scissors and chop them up a bit more...
You should not need very much...figure to do 4 stern eye holes at approximately 3/4" diameter by 2" transom thickness, you should only need about 6-8 ounces of resin, approximately the same in volume of the cabosil, and a small hand full, maybe 4-6 tablespoons of chopped strands/milled fibers...mix it all up really well, then add your catalyst.
Even though it is a bit hairy, a quick hit with a D/A and some 24-40 grit, will knock them off...
Don't try to bang them out, you might cause more damage than you want...
It would be best to just drill them out again using the same hole saw as before...just make sure you get it good and centered...