Yeah, never of rust travelling, but if you did get a spot of it close to the sealing surface of the yoke, eventually it would grow into and compromise the seal. The OP should inspect this area and treat any rust spots that are too close, but otherwise rust on the u-joints is no big deal and...