Wired on Hacking Reputations & Ratings

Wired has an interesting story about how much faith we place in online ratings and reputations, Herding the Mob.

In other words, the crowd matters. Today we harness the masses for everything from choosing the next pop star on American Idol to perfecting open source software and assembling Wikipedia articles. But perhaps the most widespread and vital uses for group input online are in scoring systems. In addition to eBay feedback, these are the customer ratings that Amazon.com and Yahoo Shopping post with product reviews. They’re the feedback scores that Netflix tallies to help subscribers decide which movies to order. And they’re the up-or-down votes that sites like Digg and Reddit (part of the Wired Media Group, which also includes WIRED magazine) rely on to determine which stories to feed Web surfers.

I've always wondered if a studio could convince a large number of people to rate a movie higher than it deserves (or add it to their queue) on Netflix, Amazon, Blockbuster Online, Flixster, IMDB, etc., creating a surge in orders for a title. You can already buy "Diggs" for stories, but what if millions of dollars in DVD sales were at stake? I'm sure Netflix calculates how many copies of a movie to order partially based on the reviews and ratings, so I wonder what would it cost to "hack" DVD sales?

Comments

[Aron]

I have definately seen concerted efforts to hack company review sites related to penny stocks (such as Gameznflix). I think the fairly small chance of getting caught red handed will keep most larger companies away from direct manipulation. It would be hard for them to measure the benefit, and if they were busted, executive heads would roll. Not to say that such deterrents work 100%.

[prozac]

I have no idea but it was an interesting read and since alot of money is at stake I don't doubt ratings can be played with. I rely on them more than I should.

[Hunter McDaniel]

Here is Hunter's Axiom. Every system which is built on trust quickly attracts parasites who profit by abusing that trust.

You can see this all across the Internet. Shill bidding and reviews on e-bay, authors reviewing their own books on Amazon, vandalism of Wikipedia articles, sock puppetry on blog threads. Communication paths are new, but human nature remains the same.

[type-cast]

The ideas advanced are what matter. I want a sort function for Netflix reviews, like IMDB and Amazon have. That prevents ratings fraud once enough people have voted, by allowing a reader to focus on negative reviews. It's an obvious advantage for users. One complaint I have about Amazon is you can't rate an item, unless you write a review. Another problem's their new feature to let people comment on a review. This allows vandalism and ad-hominem slandering. It is constantly abused. Reviews shouldn't be treated like bulletin boards. I would rather they make people write a review of their own than let them leech off others.

[type-cast]

It would be nice if Netflix let you show the average rating for people who write reviews, and people who don't. They are two different demographics. Usually, there is a huge ratio of ratings to reviews - 20:1, 300:1, 4000:1. You frequently notice a trend in opinion one way by review-writers and another by average viewers. They need to let us filter results, or show more information, if we choose.

[WeaselBuddha]

Few years back I started noticing comments on IMDb, for small releases, talking about the flick in glowing terms. No external reviews from the pros, no sort of clue of the real deal beyond the write up.

The movie would be a howling dog, scary bad. Either the reviewer was a Shill, or doing some heavy mind altering drugs.

Now I see the same thing with almost every title, people who've never done a review before suddenly chiming in with how they had a spontaneous orgasm while watching some celluloid bag of crap. Often people just piling on, "oh it really moved me".

This explosion, for me, has really devalued what was a useful fall-back in judging a movie. Who do these guys think they are fooling?

Good one Hunter, I think you are absolutely correct. The cynicism these folks show is depressing. Spaulding Grey, in Swimming to Cambodia, says what hope is there for world peace when you can't get your neighbors to turn down their stereo. I'd go further. When the first Batman movie was released, I stood in line with friends, we confronted someone who jumped the queue behind us big time, the jumper turned to me, with fish cold eyes and threaten my life.

What hope is there for anything when people are willing to do violence for what was a crowded five buck movie? Or make ridiculous liars out of themselves just to pimp some waste of time?

The Times ran an article last week, talking about the disinhibition associated with flaming, and reactionary rants. I'd recommend it. Self respect I think is often the first loss.