Re: How old
You have your basic safety considerations, as discussed, and shade issues, just like if you went to the park, or camping.
The real issue is more about the time and trouble invested in the outing and the exit strategy. You will learn about taking an infant and toddler to a restaurant or movie, or not, and it is the same thing. But you can walk out of a restaurant without eating (you still pay of course) and get home, but not so with a boat unless you live on the water. When there is a melt-down, you have to evacuate. If you aren't swift, you have paniced mother and child to deal with and it gets worse, exponentially, by the minute.
Try this strategy if you can--shorten the time the mother and child are in the process, since they will both have short tolerance. You prep the boat and take the boat to the lake, launch and be squared away. Mother and child arrive independently, timed to arrive (or called) when the boat is launched and running; yo pick them up at the dock and off you go. Stay within a short run back. Any problems, or before problems, return to the dock, send mother and child home, you haul the boat, etc. Trust me you would rather push knitting needles into your eyes than to do the ramp thing with a howling baby.
Once they get a little older, as someone suggested, you will have to change to what they like and don't like. if they don't like going fast, you will not be tubing. If they don't like sitting still, you will not be fishing. Don't even try.
PLan trips where you can get out of the boat.
Mine have been in small boats from the start, and sometimes in difficult conditions because we have to use a boat to get to a vacation place. Taking them in the boat is a lot of work, cuts into your fun, but is well worth it--a front-loaded investment as it were.