I don't know the makeup of the tank(s) on the 135, but on the 3 and 4 cylinder engine mounted oil tanks, the beeper is controlled by a floating sensor and a sensing switch. It seems the Magnet part of the mechanism has a habit of falling off (failure) resulting in the indication that the oil reservoir is low/empty. Funny, my last outing my 2002 115 dropped it's magnet.
Per Faztbullet, correct repair method is to cut the tank open, do the fix and glue the tank back together.....or buy a new tank. I chose to disconnect the sending wires and just keep an eye on my oil consumption. I really doubt that I have the expertise to do the repair and be successful!