#Announcement - Changes to the Review Program pt.02 (a brief update)

It all sounds great overall, but we are a bit concerned with the fact that the review program (and benefits) will still be based on ranks. To quote: "These levels will be earned by achieving a minimum rank, submitting a minimum # of reviews, and being a contributor for a minimum period of time."

We have been here for almost 5 years and, as we all know, rank is heavily influenced by the number of reviews written. Please keep that in mind as you read.
What we have seen happen is that most people come into Eden pumped up and excited, spitting out loads of reviews, good or bad. We have even seen Eden assign extra free items to our students while they were still in Mentor class, above and beyond the 2 they were given in class. Most of them have been assigned several more in short order after they graduated (like, in less than a month's time.) Even without extra freebies, that initial writing rush pushes most of the newest members higher in rank faster than the others, regardless of the quality of their reviews.

Then, based on the current way that rank is calculated, a year later those no-longer-new reviewers will fall, unless they can keep writing at that same frantic pace to keep their numbers consistent. That is the worst part, because after that amount of time on Eden the people who aren't dedicated to writing good reviews tend to go away, and the ones who stay have refined their technique enough to be really good. And yet some of the newest reviewers on board will outrank them within a few months.

Now, for those of you who are saying "Yeah but it's not just the number of reviews, it's how people vote on them too." think of this:
If you submit one review today that gets 10 "Good" votes, you're still not going to be able to keep up in rank with the person who submits 10 reviews today that gets 2 "Good" votes each. And yes, in our time as Editors we did see some people submitting 10 reviews a day.

Unless there are plans to not weight a reviewer's rank using the current system, a lot of the seasoned reviewers will quickly be at the bottom of the new system's benefits barrel. It just bums us out because we've already seen too many great reviewers (and forum contributors) leave out of frustration. less

As we move forward, rank is less and less of a factor.

It is a difficult subject. The ranking system was in place before I got here, so it is a long standing part of the community structure. On the other hand, the system is so scientific and logical (which is fine) but that seems a disconnect for the activities it rates. Plus it can be inflated by people being more active. (I have never been a fan of the system - but what would work better?) less

You're right, it is a difficult subject. Assuming your question wasn't rhetorical (ignore this if it was), maybe putting the ranking system itself into tiers based on time in the community would help. For example, in the Activity Factor section, writing reviews makes up 28% of the calculation yet every other factor only accounts for 2% (with the exception of the 4% for writing follow ups). Maybe for newcomers "Writing reviews" shouldn't carry so much influence at first, yet could rise over time.

On the wilder end of the spectrum, rankings could be ditched in favor of a 2-tier system; one for very new members, and one for established. The new members would work up to the benefits that the established ones had earned simply by being involved for a set amount of time, because let's face it the people who write very few reviews but are otherwise active in the community are valuable too. Everyone would be on a more level ground. That would all but eliminate several of the current issues.

Just thinking out loud less

You're right, it is a difficult subject. Assuming your question wasn't rhetorical (ignore this if it was), maybe putting the ranking system itself into tiers based on time in the community would help. For example, in the Activity Factor section, writing reviews makes up 28% of the calculation yet every other factor only accounts for 2% (with the exception of the 4% for writing follow ups). Maybe for newcomers "Writing reviews" shouldn't carry so much influence at first, yet could rise over time.

On the wilder end of the spectrum, rankings could be ditched in favor of a 2-tier system; one for very new members, and one for established. The new members would work up to the benefits that the established ones had earned simply by being involved for a set amount of time, because let's face it the people who write very few reviews but are otherwise active in the community are valuable too. Everyone would be on a more level ground. That would all but eliminate several of the current issues.

Just thinking out loud less

I think too, capping the ranks/levels (whatever it would end up being) so one couldn't then fall back down after achieving a level would be a good idea. Less of a community influence and more based on individual contribution and performance. This of course, is only if a similar formula for creating ranks is kept. Maybe allow influence between those within the same rank 5.1-5.9 say, but once you hit 6, you can't then return to 5. less

I was thinking that this would be good for a time frame. Like if you're active but your ranking drops, your ranking will stay 'capped,' but if you're not online for several months the cap will go away. Otherwise we'd bet a ton of people who are no longer on the site, have been banned, etc. staying at the top of the list.

It seems complicated, but like a good idea! less