1098: "Star Ratings"

This forum is for the individual discussion thread that goes with each new comic.

Moderators: Moderators General, Prelates, Magistrates

asdfzxc
Posts: 60
Joined: Mon Jun 08, 2009 7:04 pm UTC

1098: "Star Ratings"

Postby asdfzxc » Wed Aug 22, 2012 4:01 am UTC

Image

Alt-text: I got lost and wandered into the world's creepiest cemetery, where the headstones just had names and star ratings. Freaked me out. When I got home I tried to leave the cemetery a bad review on Yelp, but as my hand hovered over the 'one star' button I felt this distant chill ...

At some point, it becomes more efficient to just remove the three middle stars altogether and just use a like/dislike system.

User avatar
rhomboidal
Posts: 801
Joined: Wed Jun 15, 2011 5:25 pm UTC
Contact:

Re: 1098: "Star Ratings"

Postby rhomboidal » Wed Aug 22, 2012 4:03 am UTC

And half the time, "1 star" really means "0 stars" was not an option.

User avatar
TheGrammarBolshevik
Posts: 4878
Joined: Mon Jun 30, 2008 2:12 am UTC
Location: Going to and fro in the earth, and walking up and down in it.

Re: 1098: "Star Ratings"

Postby TheGrammarBolshevik » Wed Aug 22, 2012 4:05 am UTC

Is the alt-text a reference to a specific story?
Nothing rhymes with orange,
Not even sporange.

sirono
Posts: 5
Joined: Wed Aug 22, 2012 4:27 am UTC

Re: 1098: "Star Ratings"

Postby sirono » Wed Aug 22, 2012 4:30 am UTC

the alt-text is really great. it gave me the chills :D

User avatar
Quicksilver
Posts: 437
Joined: Wed Apr 29, 2009 6:21 am UTC

Re: 1098: "Star Ratings"

Postby Quicksilver » Wed Aug 22, 2012 4:30 am UTC

At least youtube got rid of that system with it's like/dislike bar. Red and green's eternal struggle being much easier to understand.

User avatar
Djehutynakht
Posts: 1546
Joined: Thu Feb 10, 2011 1:37 am UTC

Re: 1098: "Star Ratings"

Postby Djehutynakht » Wed Aug 22, 2012 4:57 am UTC

Huh. That would be a wierd cemetary. How would we determine peoples' ratings upon death? And what effects would this have on the cemetary, I wonder?

das7002
Posts: 14
Joined: Wed Dec 01, 2010 11:45 am UTC

Re: 1098: "Star Ratings"

Postby das7002 » Wed Aug 22, 2012 5:36 am UTC

Djehutynakht wrote: How would we determine peoples' ratings upon death?


How much debt they died with (higher ratings being how much they screwed the bank)

User avatar
San Fran Sam
Posts: 228
Joined: Tue Nov 15, 2011 5:54 pm UTC

Re: 1098: "Star Ratings"

Postby San Fran Sam » Wed Aug 22, 2012 5:57 am UTC

Then there is zero stars....

Drop everything. Run away. Don't look behind you.

Gye
Posts: 16
Joined: Wed Feb 02, 2011 7:01 am UTC

Re: 1098: "Star Ratings"

Postby Gye » Wed Aug 22, 2012 6:47 am UTC

Quicksilver wrote:At least youtube got rid of that system with it's like/dislike bar. Red and green's eternal struggle being much easier to understand.

Absolutely. The only downside was a hundred billion "X people missed the like button" jokes.

User avatar
StClair
Posts: 409
Joined: Fri Feb 29, 2008 8:07 am UTC

Re: 1098: "Star Ratings"

Postby StClair » Wed Aug 22, 2012 7:42 am UTC

Exactly right, and this was (a major contributing factor to) the downfall of the "Mission Architect" feature (player-created content) in City of Heroes. Good luck wading through the vast 3 and 4 star swamps.

User avatar
AvatarIII
Posts: 2098
Joined: Fri Apr 08, 2011 12:28 pm UTC
Location: W.Sussex, UK

Re: 1098: "Star Ratings"

Postby AvatarIII » Wed Aug 22, 2012 7:49 am UTC

I find this is true with things like Google store/Android Market, but for film reviews (from experience with LoveFilm rental service) I find 3 stars is OK, 2.5 and below is crap, 3.5 is good, 4 is very good, 4.5 is excellent and 5 stars is "only one review"

quorn
Posts: 1
Joined: Wed Aug 22, 2012 8:20 am UTC

Re: 1098: "Star Ratings"

Postby quorn » Wed Aug 22, 2012 8:34 am UTC

LoveFilm allows you to award half stars, so it's really a 10 point system. I imagine some pointy haired boss saw a film with 3.5 stars and wanted to give it the same review everyone else was giving it.

I sort through Amazon's DVD reviews by rating and then page through until it dropped from 5 stars with one review to 4.5 with many reviews. That's where the good stuff lives. The top 5/4.5 star material is related to Stephen King, Steven Spielberg, Rob Reiner, Corey Feldman, and Tom Hanks.

tjunction
Posts: 31
Joined: Wed Aug 18, 2010 11:43 am UTC

Re: 1098: "Star Ratings"

Postby tjunction » Wed Aug 22, 2012 8:42 am UTC

Depends what the website is.

For instance, with TripAdvisor, generally few people write online reviews for something that met their expectations, but people will write reviews when they have a complaint. So you have to take the ratings there with a big pinch of salt. Generally I find if the reviews on TripAdvisor are more than about two-thirds positive, the hotel is OK. (Also, some people have weird standards. "I paid for a budget hotel and although it was clean, comfortable and did everything it promised, the girl on reception didn't smile! One star").

User avatar
AvatarIII
Posts: 2098
Joined: Fri Apr 08, 2011 12:28 pm UTC
Location: W.Sussex, UK

Re: 1098: "Star Ratings"

Postby AvatarIII » Wed Aug 22, 2012 9:31 am UTC

tjunction wrote:Depends what the website is.

For instance, with TripAdvisor, generally few people write online reviews for something that met their expectations, but people will write reviews when they have a complaint. So you have to take the ratings there with a big pinch of salt. Generally I find if the reviews on TripAdvisor are more than about two-thirds positive, the hotel is OK. (Also, some people have weird standards. "I paid for a budget hotel and although it was clean, comfortable and did everything it promised, the girl on reception didn't smile! One star").


Yeah, I definitely feel that hotel reviews should definitely be taken qualitatively rather than quantitatively, I will generally read a couple of reviews from each "star rating" and decide based on the content rather than the rating itself.

TFTD
Posts: 2
Joined: Fri Aug 19, 2011 8:08 am UTC

Re: 1098: "Star Ratings"

Postby TFTD » Wed Aug 22, 2012 9:35 am UTC

Quicksilver wrote:At least youtube got rid of that system with it's like/dislike bar. Red and green's eternal struggle being much easier to understand.


Especially people with a red/green blindness.

User avatar
Rhombix
Posts: 45
Joined: Wed Nov 21, 2007 9:38 pm UTC

Re: 1098: "Star Ratings"

Postby Rhombix » Wed Aug 22, 2012 9:40 am UTC

AvatarIII wrote:I find this is true with things like Google store/Android Market, but for film reviews (from experience with LoveFilm rental service) I find 3 stars is OK, 2.5 and below is crap, 3.5 is good, 4 is very good, 4.5 is excellent and 5 stars is "only one review"


First site I thought of upon reading this comic was Amazon, where many products (especially non-media) are sensetive to low ratings. No-one wants to buy faulty hardware for their computer, or toys with overlooked safety hazards.

User avatar
AvatarIII
Posts: 2098
Joined: Fri Apr 08, 2011 12:28 pm UTC
Location: W.Sussex, UK

Re: 1098: "Star Ratings"

Postby AvatarIII » Wed Aug 22, 2012 10:09 am UTC

Rhombix wrote:
AvatarIII wrote:I find this is true with things like Google store/Android Market, but for film reviews (from experience with LoveFilm rental service) I find 3 stars is OK, 2.5 and below is crap, 3.5 is good, 4 is very good, 4.5 is excellent and 5 stars is "only one review"


First site I thought of upon reading this comic was Amazon, where many products (especially non-media) are sensetive to low ratings. No-one wants to buy faulty hardware for their computer, or toys with overlooked safety hazards.


I think the more something costs, the harsher the reviews are going to be, and an element of value for money goes into the reviews.
Computer hardware is pricey, hotels are pricey, but movie rentals are pretty cheap/free after monthly subscription.
Even android apps are pretty cheap, but on the whole I think people resent paying for android apps when some exist for free, in which case competition fuels star ratings, because people will try a free app, which is OK, probably give it 4 stars, then decide it's not good enough, buy an app that does more or less the same thing but costs £1 for example, and is better so they have no choice but to give it 4.5 or 5 stars. however if there is no free app, if someone pays for an app that is "just ok" or has a few minor faults and/or flaws that would be tolerated in a free app, they will probably fixate on those and give it a lower rating.

User avatar
eran_rathan
Mostly Wrong
Posts: 1846
Joined: Fri Apr 09, 2010 2:36 pm UTC
Location: in your ceiling, judging you

Re: 1098: "Star Ratings"

Postby eran_rathan » Wed Aug 22, 2012 11:16 am UTC

San Fran Sam wrote:Then there is zero stars....

Drop everything. Run away. Don't look behind you.


But there are four lights.


Also, cemetery contained bobcat. would not use again. 1 star.
"Does this smell like chloroform to you?"
"Google tells me you are not unique. You are, however, wrong."
nɒʜƚɒɿ_nɒɿɘ

User avatar
Eternal Density
Posts: 5590
Joined: Thu Oct 02, 2008 12:37 am UTC
Contact:

Re: 1098: "Star Ratings"

Postby Eternal Density » Wed Aug 22, 2012 11:33 am UTC

My family visited a cemetery the other day, and drank lemonade to the memory of Jonathan Sperry. [/obscurereference]
Play the game of Time! castle.chirpingmustard.com Hotdog Vending Supplier But what is this?
In the Marvel vs. DC film-making war, we're all winners.

Eutychus
Posts: 448
Joined: Mon Jan 25, 2010 6:01 am UTC
Location: France

Re: 1098: "Star Ratings"

Postby Eutychus » Wed Aug 22, 2012 11:36 am UTC

AvatarIII wrote:
tjunction wrote:Depends what the website is.

For instance, with TripAdvisor, generally few people write online reviews for something that met their expectations, but people will write reviews when they have a complaint. So you have to take the ratings there with a big pinch of salt. Generally I find if the reviews on TripAdvisor are more than about two-thirds positive, the hotel is OK. (Also, some people have weird standards. "I paid for a budget hotel and although it was clean, comfortable and did everything it promised, the girl on reception didn't smile! One star").


Yeah, I definitely feel that hotel reviews should definitely be taken qualitatively rather than quantitatively, I will generally read a couple of reviews from each "star rating" and decide based on the content rather than the rating itself.


I agree with tjunction, and with AvatarIII, but you need even more discernment than that.

I checked out the last holiday hotel we planned to stay in on TripAdvisor. Along with that mix of reviews was one that said "Run away!!! Everyone is over 30 and there is absolutely no entertainment of any kind!!!!!!" We knew immediately that it was the right hotel for us, and it was.
Be very careful about rectilinear assumptions. Raptors could be hiding there - ucim

User avatar
mojacardave
Posts: 346
Joined: Mon Jul 04, 2011 1:01 pm UTC

Re: 1098: "Star Ratings"

Postby mojacardave » Wed Aug 22, 2012 11:50 am UTC

For critics reviews, a star system is perfect, and works as you would expect. So long as you trust the critic, there's a fair chance that the star system will show you how good the product is.

For weighted user-generated-feedback reviews, Randall is spot on. Presumably that's one reason that most sites have moved towards a more binary system.

For an unusual example of where star ratings are even less trustable, see the Dilbert.com website - it wasn't too long before readers realised that no matter how few reviewers had graded each comic, or how poorly they'd rated it, the average score would never drop below 4 stars. It's a pretty badly designed site though, so I suppose it's possible that it's an unintentional bug.

User avatar
bassguy
Posts: 25
Joined: Wed Jun 27, 2012 11:21 am UTC

Re: 1098: "Star Ratings"

Postby bassguy » Wed Aug 22, 2012 12:13 pm UTC

eran_rathan wrote:But there are four lights.


Also, cemetery contained bobcat. would not use again. 1 star.


+1 for excellent use of STNG/XKCD references simultaneously...

rmsgrey
Posts: 3655
Joined: Wed Nov 16, 2011 6:35 pm UTC

Re: 1098: "Star Ratings"

Postby rmsgrey » Wed Aug 22, 2012 12:23 pm UTC

San Fran Sam wrote:Then there is zero stars....

Drop everything. Run away. Don't look behind you.


Nah, zero stars is another "only one review". If it has a hundred reviews at 0 stars, then it's either a ratings-spammer attack, or it's the default for reviews where the reviewer doesn't pick a rating.

There's always someone willing to go: "Package contained bobcat. AWESOMME!!1 - 5 Starz"

User avatar
Various Varieties
Posts: 505
Joined: Tue Mar 04, 2008 7:24 pm UTC

Re: 1098: "Star Ratings"

Postby Various Varieties » Wed Aug 22, 2012 1:06 pm UTC

Quicksilver wrote:At least youtube got rid of that system with it's like/dislike bar. Red and green's eternal struggle being much easier to understand.


Every time the stupidity of online comments gets to me, I rewatch this in order to restore my faith in humanity's amazing capacity for collaboration. You don't get that with star ratings!

tnvlsrule
Posts: 2
Joined: Wed Jun 29, 2011 12:01 pm UTC

Re: 1098: "Star Ratings"

Postby tnvlsrule » Wed Aug 22, 2012 1:10 pm UTC

It's gratifying when you do find something that has 5 stars and over a hundred reviews. The first time I encountered this was with a water pump on Newegg, and to be fair, it was probably closer to 4.9 eggs. The pump was half the price of the competition, slightly more powerful than the competition, and it could survive being hit by a sledgehammer and pumping nothing but air for several hours.

User avatar
Jackpot777
Posts: 328
Joined: Wed Sep 14, 2011 1:19 pm UTC

Re: 1098: "Star Ratings"

Postby Jackpot777 » Wed Aug 22, 2012 1:20 pm UTC

Eutychus wrote:
AvatarIII wrote:
tjunction wrote:Depends what the website is.

For instance, with TripAdvisor, generally few people write online reviews for something that met their expectations, but people will write reviews when they have a complaint. So you have to take the ratings there with a big pinch of salt. Generally I find if the reviews on TripAdvisor are more than about two-thirds positive, the hotel is OK. (Also, some people have weird standards. "I paid for a budget hotel and although it was clean, comfortable and did everything it promised, the girl on reception didn't smile! One star").


Yeah, I definitely feel that hotel reviews should definitely be taken qualitatively rather than quantitatively, I will generally read a couple of reviews from each "star rating" and decide based on the content rather than the rating itself.


I agree with tjunction, and with AvatarIII, but you need even more discernment than that.

I checked out the last holiday hotel we planned to stay in on TripAdvisor. Along with that mix of reviews was one that said "Run away!!! Everyone is over 30 and there is absolutely no entertainment of any kind!!!!!!" We knew immediately that it was the right hotel for us, and it was.


And then... there are some negative online reviews which persuade very few that read them to see it as a selling point.

endolith
Posts: 229
Joined: Tue Jan 01, 2008 2:14 am UTC
Location: New York, NY
Contact:

Re: 1098: "Star Ratings"

Postby endolith » Wed Aug 22, 2012 1:23 pm UTC

Web programmers please read (Especially if you work for Amazon):

How Not To Sort By Average Rating

also Randall's implementation: reddit's new comment sorting system Posted by Randall Munroe

San Fran Sam wrote:Then there is zero stars....

Drop everything. Run away. Don't look behind you.


No. Zero stars means "Has only one review".
Last edited by endolith on Wed Aug 22, 2012 1:33 pm UTC, edited 2 times in total.

User avatar
cellocgw
Posts: 2067
Joined: Sat Jun 21, 2008 7:40 pm UTC

Re: 1098: "Star Ratings"

Postby cellocgw » Wed Aug 22, 2012 1:25 pm UTC

Eternal Density wrote:My family visited a cemetery the other day, and drank lemonade to the memory of Jonathan Sperry. [/obscurereference]


"When life gives you lemons^H^H^H^Hdeath, make lemonade" ?
resume
Former OTTer
Vote cellocgw for President 2020. #ScienceintheWhiteHouse http://cellocgw.wordpress.com
"The Planck length is 3.81779e-33 picas." -- keithl
" Earth weighs almost exactly π milliJupiters" -- what-if #146, note 7

User avatar
eran_rathan
Mostly Wrong
Posts: 1846
Joined: Fri Apr 09, 2010 2:36 pm UTC
Location: in your ceiling, judging you

Re: 1098: "Star Ratings"

Postby eran_rathan » Wed Aug 22, 2012 1:41 pm UTC

cellocgw wrote:
Eternal Density wrote:My family visited a cemetery the other day, and drank lemonade to the memory of Jonathan Sperry. [/obscurereference]


"When life gives you lemons^H^H^H^Hdeath, make lemonade" ?


"When life gives you lemons, don't make lemonade. Make life take the lemons back! Get mad! I don't want your damn lemons, what am I supposed to do with these? Demand to see life's manager! Make life rue the day it thought it could give Cave Johnson lemons! Do you know who I am? I'm the man who's gonna burn your house down! With the lemons! I'm gonna get my engineers to invent a combustible lemon that burns your house down!"
―Cave Johnson
"Does this smell like chloroform to you?"
"Google tells me you are not unique. You are, however, wrong."
nɒʜƚɒɿ_nɒɿɘ

dedwrekka
Posts: 152
Joined: Wed Dec 15, 2010 10:39 am UTC

Re: 1098: "Star Ratings"

Postby dedwrekka » Wed Aug 22, 2012 2:11 pm UTC

For the video game industry, replace "Crap" with "Buggy". In the gaming industry, there's only three levels of quality. 5, 4.5, and 4. And even then what that translates to is:

5 - Modern Cookie Cutter FPS / Pays for advertising
4.5 - Excellent game to okay game
4 - Buy it on sale
3.5 to .5 - Has a bug or the reviewer has issues with the genre of game

kronak
Posts: 1
Joined: Mon Jan 24, 2011 5:26 pm UTC

Re: 1098: "Star Ratings"

Postby kronak » Wed Aug 22, 2012 2:24 pm UTC

The statistician in me was screaming about the need for Wilson score confidence interval when I read this comic.

B-Fisch
Posts: 18
Joined: Tue Jan 25, 2011 5:05 am UTC
Location: Lafayette, LA

Re: 1098: "Star Ratings"

Postby B-Fisch » Wed Aug 22, 2012 4:13 pm UTC

dedwrekka wrote:For the video game industry, replace "Crap" with "Buggy". In the gaming industry, there's only three levels of quality. 5, 4.5, and 4. And even then what that translates to is:

5 - Modern Cookie Cutter FPS / Pays for advertising
4.5 - Excellent game to okay game
4 - Buy it on sale
3.5 to .5 - Has a bug or the reviewer has issues with the genre of game


Someone needs to watch the Red vs Blue PSA about the game industry. It has pretty much that exact joke. Also, "Stay tuned for Part 2: Sequels. Parts 3 and 4 are about that, too..."

User avatar
San Fran Sam
Posts: 228
Joined: Tue Nov 15, 2011 5:54 pm UTC

Re: 1098: "Star Ratings"

Postby San Fran Sam » Wed Aug 22, 2012 4:31 pm UTC

rmsgrey wrote:
San Fran Sam wrote:Then there is zero stars....

Drop everything. Run away. Don't look behind you.


Nah, zero stars is another "only one review". If it has a hundred reviews at 0 stars, then it's either a ratings-spammer attack, or it's the default for reviews where the reviewer doesn't pick a rating.

There's always someone willing to go: "Package contained bobcat. AWESOMME!!1 - 5 Starz"


Oh really? I guess you never saw Ishtar.

jpers36
Posts: 234
Joined: Wed Apr 14, 2010 2:47 pm UTC
Location: The 3-manifold described by Red and Blue

Re: 1098: "Star Ratings"

Postby jpers36 » Wed Aug 22, 2012 5:18 pm UTC

San Fran Sam wrote:
rmsgrey wrote:
San Fran Sam wrote:Then there is zero stars....

Drop everything. Run away. Don't look behind you.


Nah, zero stars is another "only one review". If it has a hundred reviews at 0 stars, then it's either a ratings-spammer attack, or it's the default for reviews where the reviewer doesn't pick a rating.

There's always someone willing to go: "Package contained bobcat. AWESOMME!!1 - 5 Starz"


Oh really? I guess you never saw Ishtar.


Ishtar has its defenders.

User avatar
da Doctah
Posts: 996
Joined: Fri Feb 03, 2012 6:27 am UTC

Re: 1098: "Star Ratings"

Postby da Doctah » Wed Aug 22, 2012 5:40 pm UTC

Every time I actually want to check out product reviews, it's for something that nobody has bothered with yet. They always invite me "be the first the write a review for this product" as if that's an appropriate substitute.

endolith
Posts: 229
Joined: Tue Jan 01, 2008 2:14 am UTC
Location: New York, NY
Contact:

Re: 1098: "Star Ratings"

Postby endolith » Wed Aug 22, 2012 5:45 pm UTC

da Doctah wrote:Every time I actually want to check out product reviews, it's for something that nobody has bothered with yet. They always invite me "be the first the write a review for this product" as if that's an appropriate substitute.


Sounds like you need a time machine.

User avatar
pkcommando
Posts: 574
Joined: Fri Feb 10, 2012 3:22 pm UTC
Location: Allston, MA

Re: 1098: "Star Ratings"

Postby pkcommando » Wed Aug 22, 2012 7:16 pm UTC

San Fran Sam wrote:
rmsgrey wrote:
San Fran Sam wrote:Then there is zero stars....

Drop everything. Run away. Don't look behind you.


Nah, zero stars is another "only one review". If it has a hundred reviews at 0 stars, then it's either a ratings-spammer attack, or it's the default for reviews where the reviewer doesn't pick a rating.

There's always someone willing to go: "Package contained bobcat. AWESOMME!!1 - 5 Starz"


Oh really? I guess you never saw Ishtar.

Anyone who says they saw Ishtar, never saw Ishtar.

JimsMaher
Posts: 145
Joined: Wed Mar 14, 2012 5:14 pm UTC

Re: 1098: "Star Ratings"

Postby JimsMaher » Wed Aug 22, 2012 7:44 pm UTC

Can't decide if he's being cynical or elitist, when marginalizing 3½ stars to the "crap" category.
Everyone has their own interpretation I suppose.
Does this mean that anyone rating a 3 star for something they use but think could be better, is actually conveying an incorrect assessment of their own experience?

This comic gets exactly ½ star. HA!

User avatar
SlyReaper
inflatable
Posts: 8015
Joined: Mon Dec 31, 2007 11:09 pm UTC
Location: Bristol, Old Blighty

Re: 1098: "Star Ratings"

Postby SlyReaper » Wed Aug 22, 2012 8:00 pm UTC

JimsMaher wrote:Can't decide if he's being cynical or elitist, when marginalizing 3½ stars to the "crap" category.
Everyone has their own interpretation I suppose.
Does this mean that anyone rating a 3 star for something they use but think could be better, is actually conveying an incorrect assessment of their own experience?

This comic gets exactly ½ star. HA!

People tend to rate things at one extreme or the other. If it's 3 and a half stars, chances are there are a lot of 5 star reviews and a lot of 1 star reviews. The 1 star reviews are the ones you want to pay attention to.
Image
What would Baron Harkonnen do?

endolith
Posts: 229
Joined: Tue Jan 01, 2008 2:14 am UTC
Location: New York, NY
Contact:

Re: 1098: "Star Ratings"

Postby endolith » Wed Aug 22, 2012 8:19 pm UTC

SlyReaper wrote:The 1 star reviews are the ones you want to pay attention to.


"I can't believe this broke after I ran it over with my SUV! I'm outraged! 1 star!"


Return to “Individual XKCD Comic Threads”

Who is online

Users browsing this forum: Google [Bot], hetas and 88 guests