Lots of mildew would indicate a lack of use and/or a lack of maintenance. The closed compartments of boats generally acquire mildew due to trapped moisture.
So, with that in mind I would want to check things out more carefully than you might with a well maintained boat. It does not mean wet foam, but it does not guarantee dry foam either. You will have to check everything out to your satisfaction.
Also, 2005 is right around the time when Caravelle transitioned from wood to synthetic materials in their construction. So, you want to be sure which side of that line this boat is on.