Rating System Feedback

I really like the rating as a kind of reward for even doing the reviews. I enjoy the process, but there is something satisfactory in seeing that my rank goes up. I have 1 review left until I hit the 50 review cap, and honestly it is cramping my enthusiasm about the review process. If I didn't know that some people may find my reviews helpful, it would be enough to kill the motivation all together for me. The cap is the biggest issue I have with the review program.

Besides the cap, I am pretty happy with the ranking system. I just don't think someone's reviews should have no value because someone has "too many." The cap was created for one person (that is obvious by looking through the contributors) but now that more people are approaching the cap it might be lifted? I don't think there ever should have been a cap. All it does is punish the people who are the most zealous and committed to the program. Why punish the people who contribute the most to the review program or who have been on here the longest? There should be something to show for cranking out reviews, especially if they are quality reviews. I do believe that there should be a score or rank as a kind of reward for doing the reviews. If unverified reviews didn't count toward the rank, would anybody review products that they bought on their own? When I spend my money on a large shipment of toys from this store, I write reviews on them because I know it will affect my rank. I write verified reviews because I got the toy as payment for my honest review. I do think there needs to be incentive for people to review toys they buy on their own. Getting rid of the ranking altogether would kill a lot of the pleasure I receive from writing reviews. I really like providing information that can be useful to others, but I like seeing a tally of my reviews also. I don't know about everyone else, but seeing my rank go up a little bit after a couple of reviews brings a big feeling of accomplishment and satisfaction trumped only by a comment that someone found my review helpful in their purchase.

The review program is the reason I shop from this site (with more loyalty than is probably sane) and the ranking is what helps motivate me to review every single toy I buy. I would like to see it expanded upon and improved, but please don't trash it.

I think something helpful may be having different tiers. Say there could be a tier for people with 1-39 reviews, a tier for people with 40-99 reviews, and a tier for people with 100 reviews. That way people would be ranked with people within their own bracket of review numbers. Just a suggestion anyways.

A while back Essin' Em suggest a "sexpert in training" score where comments and advice could be ranked for usefulness and accuracy. I am hoping that will get included as it is a great idea that would really help shoppers to know who to ask questions of.

Thanks so much for this post, and I'm looking forward to the suggestions and ideas of others as well.

I personally have mixed feelings about the rating system. I understand that there are people who enjoy having a rank but, if it's assessed on a curve which affects everyones rank, I can see where a cap would be necessary because a new reviewer, even though possibly fantastic at it, would find it very difficult to catch up to everyone else if there were no cap and several high number reviewers so, I see that point. I personally feel kind of bad about being higher ranked than other members who have been members longer than me so, I think seniority should maybe have a play in it too. I really don't do my reviews in order to achieve a rank though, I personally am very thankful to be able to know if a toy is going to work out for me before purchasing it and just really want to return the favor. I have saved so much money on toys that had the potential to be a total bomb just from reading reviews and I'm very, very thankful for that because, I can find that perfect toy that's going to work out. As far as sex experts, eden already has that where you can chat or leave a message for them so, I think customers would go there to ask questions. So, I would be happy either way because I just really love this site so much! Thank you to all you wonderful reviewers out there! So many of you have helped me in my decision making that you have no idea! =)

I used to pride myself when my rank would soar and I would remain amongst the best of the best reviewers here on Eden's. I thought "Wow, customers are enjoying what I write, hopefully I'm impacting the way people shop and what they buy as positively as I can." I was disappointed to know that the ranking isn't based solely on what customer's think of our reviews but more so based on our activity and our interaction with other reviewers. I've been insanely busy with work over the past couple of months and have really neglected all the new reviews posted and have only been able to comment on just a few every now and again. If I had known that by pushing myself to be more active on other reviewer's reviews it would have kept my rank solid I would have done more. However, coming back and seeing a much lower rank than I have had before has been disappointing. Not that I'm overly concerned with my score but it was a personal victory to see my reviews become popular, it was a huge reward for me to see them so well received because I absolutely loved the entire process. I suppose I could consider myself a veteran to the reviews, my year mark with you guys will be next month and I'm insanely excited and pleased to have been a part of this phenomenal adult community!

I don't think we should hold one reviewer in esteem and have other reviewers try to live up to their activity, we should be ranked solely on ourselves. I don't exactly know how I feel about the ranking system but I do think having new reviewers that don't get dozens of comments from fellow reviewers is discouraging because their reviews get pushed back and it may keep them from submitting more reviews. It would be pretty neat to see catagories, perhaps use the length of time a reviewer has been here as a bracket. I love the reviews by brand new users but once they're gone off the front page I have to sift through to find their reviews, their lower rankings always manage to push them to the way back. Also, to have the number of reviews as a bracket, I like the idea of 1-25 or 25-50, etc. Also, how about ranks amongst just the male reviewers and a seperate system for the women? It would be awesome to be able to input some information and see reviews only by reviewers within a certain age bracket or with a specific sexual orientation or marital status, or even the option to pick "randomize" and be shown all kinds of different reviews. There's a lot you guys could do with the program and there really is no limit to how far you can take it.

All these ideas could really overcomplicate the system. If it works for the Eden's website then it works for me, while I've had my qualms I'll definitely stick it out. If it ain't broke don't fix it and if you have the time and energy to change a few minor issues then by all means we'd really appreciate it.

Ooooh, that "sexpert in training" is a good idea. Or some way for us to identify ourselves, whether voluntarily or related to how helpful we are.

I think seniority might be a good idea, but it might just discourage newer reviewers. Maybe some way to differentiate between the # of approved reviews and voluntary reviews a reviewer has done?

The reviewer rating should be based on individual quality of the reviewer. It should not be based on a curve, and it should not be based on activity related to other reviewers.

If you're looking at quality, then the number of reviews that you've written will automatically come into consideration. Editorially verified reviews have a higher rank. This makes sense because often the editor may add a quick additional comment. Self published reviews have secondary importance, but are still valuable because those represent the toys that you've purchased on your own. The second component that represents quality is the public rank on usefulness and the number of comments generated on your own reviews.

If I didn't see the rating system as basically useless, I'd find it to be extremely frustrating. Eden Fantasy's is not a social networking site. Why do we get points or lose points on how often we leave comments on other people's reviews? I would prefer to read a well written review and not leave a comment because it wasn't necessary then realize that it's in *my* best interest to leave a relatively pointless remark just to show activity.

The current system-
1. Seems to provide Eden employees with an unfair advantage.
2. Actually penalizes quality writers that lead busy and active lives outside of Eden Fantasy's for not being active in "reading and commenting."
3. Partially penalized those members of the review program that choose not to purchase and write reviews during times of the program that the program was inactive.
4. As several others pointed out, penalizes new reviewers because it is so difficult to see there reviews after they are no longer on the "latest review" page. The idea above about viewing by length of time with the program is fabulous.

I think that I've been with the program for about a year and a half and have completed 23 reviews. At the moment, I rated 4.0- the very same rating as someone with 3 reviews. It looks like I suck at writing reviews, yet I always have comments almost immediately, and I always follow up quickly. I think my rating is primarily due to inactivity outside of my own reviews. (I'm writing a thesis. I could care less.)

Oggins mentioned that customers can go to customer service for help with questions. I don't. I doubt actually that a lot of people do regularly. I love the reviews, and I read the comments. I've learned a lot, and find them informative. They help tremendously in deciding on a particular product- OR a similar product. Especially the well-written reviews which are more than just how good the sex was or wasn't!

I think though, that it's important that there be some reason for the reviewers to actually care about the ratings. Perhaps a coupon a couple times a year, to be used when convenient, where the value depends on the reviewers rating. Eden did that recently and we actually spent close to $250. It was the right timing, although most of the time we'd want to balance it.

I don't really think the rating system works. Of course, i don't really care. I'm involved in the community because I appreciate the info when it's time for me to shop and hope my input helps. There are also a lot of great people involved who have always answered questions when asked and I think there’s a great database of info here.

Do people really think the rating system is accurate anyways? i thought it was cool at first, but quickly decided something was probably off on it. The person ranked ahead of me has written 1 review. I comment fairly often, write follow-up reviews occasionally, am involved in the forums and have 12 reviews. Something seems off about that. Maybe they’re just cooler than me. I wouldn’t doubt it for one second.

Anyways, I encourage everyone to be active in the community because it’s unique and pretty damn cool.

Devilbluedress put it perfectly: "I would prefer to read a well written review and not leave a comment because it wasn't necessary then realize that it's in *my* best interest to leave a relatively pointless remark just to show activity."

I definitely feel the pressure to do this -- leave comments that don't necessarily spark discussion or anything -- just for rank.

This gave me an idea. What if there was a way people could give other people kudos (or whatever you want to call them) for particular features of their review? I'm thinking of something like what they have on Yelp.com, where you can rate reviews as "cool," "useful," "funny," etc. Then you could implement a system where reviews could be sorted "funniest first" or something.

Yelp also has "compliments" you can give to people, such as "Write more!" Something like that would work on the reviewers' individual pages.

Oh, and I agree w/ the coupon idea. That really gave me an excuse to buy something. I had 45% off and it was awesome!

I personally am in favor of just eliminating the ranking system. Just list reviewers by how many reviews they have written, verified or unverified! It makes the most sense to me.

I'm glad I came back to check on this thread, as there is definitely some really great ideas that I hadn't even thought of. The coupon idea is awesome Devilbluedress, and I know with the last coupon I definitely splurged.

I do think that the popularity and activity have too much pull when it comes to the rank though. I like the activity part of it for the reason that it does give people the push to read and comment on reviews. I think it is something that is pretty encouraging to reviews that are new. I know I found the comments left by people with my first few reviews to be extremely helpful.

I have never actually talked with a customer service person or used the live chat feature when it comes to products. I ask people who have the toy or who has a lot of experience with similar toys. I really love being able to ask questions of fellow reviewers and think the "sexpert in training" would only enhance that, for casual shoppers as well as the obsessed (me). I think the activity rating should still have an effect on total rank, but it shouldn't be as high as it is now. I have noticed a sizable drop in my rank when I'm not able to log in for a few days, and it seriously only takes two or three days for it to happen.

Being able to sort through reviewers based on different filtering options sounds like a great idea to me. I can see how this would be very helpful to someone who doesn't have a lot of experience with toys and just doesn't know what to pick. They could pick reviewer criteria to find reviewers who had the same sexual orientation, the same gender, and be able to see the toys that those people enjoy. It would be nice for guys especially, to be able to get a quick route to reviews of toys by other guys. Or say to someone who is gay and wants to find a toy that isn't geared towards hetero couples, but doesn't know where to start. I can see this becoming a huge asset as more and more reviewers join.

There are some very good ideas in this discussion, and I'm glad that this thread was started. I think that in general the review program is great, and the ranking helps with incentive, keeping community active, and adds a little something extra to the process that I feel would hurt the review program if not there. The people that don't want the ranking, don't need to pay any attention to it, but for those of us that do enjoy it, it is just one more reason to write reviews that are helpful and informative. I know if the activity didn't affect the ranking at all, I wouldn't even look at reviews unless I was planning a purchase or the review came from one of the people that I really enjoy reading reviews from. It keeps me involved in the program and helps it from becoming stagnant. This is definitely an interesting thread!

I think sorting by gender or sexual orientation is a good idea in theory, but there are so many genders and sexual orientations... for me, I consider myself female and pansexual. It would bother me if I had to "identify" as bisexual in order to fit into the system. Gender would be even more difficult...

I don't know how we could make it work and not have people feel left out. But it is a good idea, Bulma!

I like the rating system, it just makes absolutely no sense to me. I've mentioned in forum before my rank vs. my boyfriend and in comparison to our activity and review levels. Maybe after a MAJOR overhall it would make a lot more sense. Right now it just seems like a popularity contest of sorts. I don't comment unless I find something particularly neat or want to mention a helpful suggestion. I feel stupid for worrying about my rank, but you just can't help it!! (and the bf teases me. haha really mature, I know!)

The suggestions on here are fabulous. I just can't wait to see what they come up with.

I also agree with the coupons. I had gotten that one (25/35/45) shortly after I started, so therefore only got 25. I've been waiting for another one, but have no idea how often they come out!! I thought it was a great incentive for those who do review more often.

I love the idea of being able to search for reviews by gender or sexuality. it would be tough, but people would probably have to just accept that it would be really hard to include every thing possible and perhaps not that helpful if there were too many categories. maybe a "queer" category could be a general category for those of us who don't want to be put in a box.

I think we should be ranked individually because, otherwise it's just a competition/popularity contest. As far as searching the contributors goes, maybe you could do something like what you already do with the sex toys themselves.....where if you like this contributor, you may also like these contributors who are also in the same age bracket, gender, sexual orientation...whatever . That might help customers to find other contributors who fit their needs...

As far as the ranking system goes--I don't like the way it currently works, if nothing else just because everyone has different time commitments, and also because it encourages meaningless comments just to boost your rank.

I think that it would be more beneficial to have your rating affected by how useful other people feel your reviews are--so if there was someone who had written forty-five reviews, but they were absolutely useless (obviously an exaggeration, nobody does that!), they would still have a lower rating than someone who had only written ten reviews, but the ten that they had written were amazingly fantastically awesome.

Basically, I feel that the rating system should be based on quality, not quantity.

While basing rank off of quality of the reviews, sounds like a great idea, I think it would put too much power in other people's hands. I would like to have at least some control over my rank and not have to worry that if someone else didn't agree with something I said that they would be able to trash my rank by rating all my reviews poorly. It sucks, but this does happen. I do believe that people would also be able to really pad their own rankings this way as well. Just sign on to a computer somewhere else (or delete your cookies and use an ip masking program) and you could go from rank 3 to 10 very fast with little or no work. I don't think we should rely too much on the value others place on our reviews, too much room for spite and cheating. It would be really nice if everyone rated other's reviews honestly, but this doesn't always happen. I have noticed that I have at least one "not useful at all" rating on almost all of my reviews while all the other ratings are "useful" and "extremely useful." Why else would this happen than someone has a beef with me? If they found my review to honestly not be useful at all, wouldn't they at least say what they wanted to know or what would have made it better? This doesn't just happen to my reviews either, as I've read excellent reviews that I rated highly and saw that someone had rated them "not useful at all." While the quality should have value (as it already does), I think that it being the major determining factor would leave too much room for abuse. I think the vast majority of the reviewers are honest and quite professional about the whole deal, but it would only take a handful to find a way to trash someone's ranking to the point it couldn't be repaired. Fact is that you can't always count on everyone else to be fair (especially if trashing someone else's rank would move their own rank up).

I also don't think it would be very fair to make number of reviews not matter at all. If someone could put one hour of time into writing a review, their rank should not be the same as someone who has put over 100 hours into writing their reviews. Do people who write really bad reviews actually get to be advanced reviewers? If someone doesn't take the process seriously and signs up just for free toys, odds are that eden wouldn't sign them up to be advanced reviewers if their reviews sucked and weren't helpful. No free toys, so why bother? Maybe making it more difficult to join and not just letting anybody sign up would help with that. 40 excellent reviews shouldn't be the same as 1 excellent review. As someone with more reviews than average, I would hate to see all my hours of research (I do try to go the extra mile so if there are questions I know enough to be able to answer them) and writing just get flushed down the drain. I think people who have less reviews would really benefit from a cap or the number of reviews not mattering, but someone who had more would basically be getting penalized and their work would become meaningless. I just don't see how it could benefit everyone at the same time.

Wouldn't eden want people to buy more toys and get reviews for as many of their products as possible? Lowering the incentives for people to buy toys from the site would be bad for business. If someone spends upwards of $500 every single month, would eden want them to spend that money at a different site when it could be spent here? I think that if the person has the money, the time, and the desire they should be able to write as many reviews as they want, and have them count. I don't see how it would be fair for a person who has 5 reviews and is on the site once a week to have the same score of someone who has 25 reviews and is on the site every day when they have equally excellent reviews. How could people gain in rank at all if they were held back from doing so? Would people only be able to reach a rank of 5 or so and just get stuck there no matter what they did? Is that 10 out of 10 just an impossible rank that no one will ever achieve? I do think that people who have been with the site longer should get a little boost to their score for seniority (I am not one of those people, as I've been here for just a few months). That is, IF they haven't left the site and they are still fairly active. Obviously it wouldn't make sense to have an abandoned account with a high rank.

Sorry for the super long post I just feel passionately about not letting cheaters prosper by being able to install an IP masking program and pad their own (and friends' too more than likely) rank. Doing a search in google will show you exactly how to do it and where to download the programs from as well as step by step guides. It wouldn't be that hard for someone to do, as some of hubby's ex co-workers used programs like that to cheat in online games. That was my main concern.

All great comments. There's a new thread with details about how the rating system works now and what the current change would be- if you have thoughts to add to that, check it out!