If you want a clean deck to start, replace it all. I wouldn't put another layer of plywood on, just another space to hold water, even if small and more weight.
But all the adhesive does not have to come off. Sand it to smooth it, of course anywhere there is looser stuff or carpet still...
