I had semi low sex drive when I first went on birth control, it lasted about two months. I was pretty young though and my hormones were all over the place which may have been why I bounced back quickly (I was barley 15 when I started birth control). It is possible that your body will react better to different kinds of birth control will effect your body differently, and it might be something you choose to consider if you decide to stick with the pill.
However, I have a IUD, have had it for almost two years, and I am absolutely in love with it. I'm still a crazy sex kitten, it doesn't effect me at all (even though I have the low hormone one), and I have sex twice a day with no pregnancy here!