Are you talking about trying to reconstruct walls of the fiberglass stringers you're cleaning out? If so, I'd rebuilt the sidewall of the stringer by putting in something temporary (e.g., a section of plastic-wrapped plywood clamped in place) against which you can glass a new fiberglass wall, then remove and foam the whole thing as normal.
Using some sort of foam board, and leaving it in, would give me some concerns about adhesion of poured-in foam to the stringers. I think the more the below-deck structure functions as one piece, the stronger it will be.