It's more common than you think. I don't know what the actual percentage is but on a different forum someone asked if other women had this problem and 60+ percent of women said they did. I was really surprised because it wasn't a forum for older women. I don't have this problem.
Kegels absolutely will help. In fact, they were originally proposed for this exact problem. To do them effectively, set aside a few minutes to kegel where you don't have any other distractions. Then concentrate on squeezing your PC muscles as hard as you possibly can without contracting your thigh, butt or abdominal muscles. Hold for ten or more seconds. Do this 10 or 15 times.
If you try to do this while the TV is on, while you're reading or while anything else occupies your mind, you won't squeeze as hard and won't receive as much benefit.
It will take awhile before you notice the effects of this. Be patient.
BBW is right that you don't need an exerciser, although can feel better to use one. I personally like how it feels to have the Kegelcisor inside me while I kegel.