Overall Rating Score Should Have Different Methodology - High-Def Digest Forums
Reply
 
Thread Tools Search this Thread Display Modes
  #1  
Old 03-27-2013, 11:24 AM
Senior Member
Thread Starter
 
Join Date: Feb 2011
Posts: 3,004
Default Overall Rating Score Should Have Different Methodology

From Josh Z: "The Overall score is a mathematical average of the Movie Itself,
Video, Audio and Supplement sections. In the case of The Hobbit, we have 3 + 5 +
5 + 2.5 = 15.5. Divided by 4, that comes to 3.875, which rounds up to 4 stars.
The High-Def Exclusives section counts as "extra credit" that the reviewer can use
to nudge a star rating up a notch if he feels that's warranted."

Should Movie Itself = Video = Audio? For most review sites, that doesn't make
sense. But for high def video and audio enthusiasts, equaling the three is
understandable.

But here's where it gets most troubling. Supplements are treated equally, too.
The "extras" should not get the same billing as the headlining stars.

Do I like special features? Yeah. For beloved films, I love to get all the behind
the scenes details. But are the special features as special as the main feature?
Hell no!

Some suggestions:

1) Re-weight the component scores to something like:
30% Movie Itself
30% Video
30% Audio
10% Supplements

Film itself to the Extras is a 3 to 1 ratio. We spend about that same ratio on
the main course versus dessert; that sounds fair.

2) Supplements and Hi-Def Extra are grouped together as reviewer-choice bonus
points.

I prefer not having more "arbitrary" reviewer bonus points. It might not be
random-arbitrary, but this method is less transparent. If reviewer loves a movie,
the supplements might equal 1 bonus star, and the same amount of supplements
for a crappy movie (in the reviewer's eyes) might subconsciously only equal to
half a bonus star or no bonus at all.

Perhaps this bonus calculation should have defined rules, too. What is the
maximum bonus that can be rewarded? If Battlefield Earth came with a
personal visit from Travolta who will view the film with you in your living room,
is that 5 bonus stars?

-----

Look I understand nothing replaces the actual review itself. The explanations,
the examples, the details of a review is best in describing what is it worthy/
unworthy about the disc. But since we are using a methodology in producing
rankings, why not improve our methodology?
Reply With Quote
  #2  
Old 03-27-2013, 12:09 PM
Boston007's Avatar
Senior Member
 
Join Date: Jan 2009
Posts: 11,210
Default

That's why I love when the reviewers utilize the

BAD MOVIE, GREAT DISC or GREAT MOVIE, BAD DISC
Reply With Quote
  #3  
Old 03-27-2013, 12:46 PM
Senior Member
Thread Starter
 
Join Date: Feb 2011
Posts: 3,004
Default

Quote:
Originally Posted by Boston007 View Post
That's why I love when the reviewers utilize the

BAD MOVIE, GREAT DISC or GREAT MOVIE, BAD DISC
I agree; let's keep the Bottomline subjective to the reviewer's choice. "You'll feel
better setting your money on fire instead of buying this disc" can be the reviewer's
Bottomline, I say.

But it is hard to distinguish which is better: "Highly Recommended" or "Must Buy"?
I do know which is a better rating, if the same reviewer says 4.5 versus 5 stars.

No methodology is perfect, so yes, let's keep the Bottomline comments for the
reviewer. But the current methodology is flawed, so let's improve it.

-----

If this website is called DVDExtras.com, then I understand that Supplements will
get an equal weighting. But it's not, so let's weight our ratings accordingly.

-----

My suggestion #1 of the re-weighting the component scores, is also an easy fix.
No reviewer has to change what he/she is doing. One formula in one cell of the
spreadsheet is changed, and it's fixed.
Reply With Quote
  #4  
Old 03-27-2013, 01:16 PM
eNoize's Avatar
Senior Member
 
Join Date: Nov 2006
Posts: 5,598
Default

MOD NOTE:

Moved discussion to the Feedback section.
__________________
M. Enois Duarte
High-Def Digest Contributor
Hi-Def Collection
Unfollow me at twitter: @MEnoisDuarte

Movies are so rarely great art that if we cannot appreciate great trash
we have very little reason to be interested in them.

~ Pauline Kael
.
Reply With Quote
  #5  
Old 03-27-2013, 01:27 PM
Attebery's Avatar
Senior Member
 
Join Date: Jan 2008
Posts: 9,716
Default

Thanks for the suggestion.
Reply With Quote
  #6  
Old 03-27-2013, 01:43 PM
Josh Z's Avatar
Senior Member
 
Join Date: Mar 2007
Posts: 11,925
Default

We have thousands of existing reviews in the database that were rated with the current methodology. It really wouldn't be practical to change our "Overall" score calculation method now.

I don't necessarily disagree with the points in the original post, but I also don't necessarily agree that a change is needed. If you personally weight the Movie Itself portion higher than any other (or Audio, or Video, or Extras), all you have to do is look at that individual score. It might be an issue if we only listed the composite score and didn't show you how it breaks out per section, but we do. Each category is individually rated. You can choose to pay the most attention to whichever you want.
__________________
Josh Z
Writer/Editor, High-Def Digest (Blog updated daily!)
Curator, Laserdisc Forever
My opinions are strictly my own, and do not necessarily reflect those of this site, its owners or employees.
Reply With Quote
  #7  
Old 03-27-2013, 01:52 PM
MrAngles's Avatar
Senior Member
 
Join Date: Apr 2008
Posts: 3,388
Default

My recommendation to improve the rating score methodology is for people to read the actual review. The star section is the least important section of the review page, I don't understand why people care enough to even think about the methodology involved in determining it.
Reply With Quote
  #8  
Old 03-27-2013, 01:55 PM
Boston007's Avatar
Senior Member
 
Join Date: Jan 2009
Posts: 11,210
Default

Quote:
Originally Posted by MrAngles View Post
My recommendation to improve the rating score methodology is for people to read the actual review. The star section is the least important section of the review page, I don't understand why people care enough to even think about the methodology involved in determining it.
Do away with that star chart and put that in words IN the review, forcing people to read it all the way through, hehehe
Reply With Quote
  #9  
Old 03-28-2013, 11:47 AM
Josh Z's Avatar
Senior Member
 
Join Date: Mar 2007
Posts: 11,925
Default

Quote:
Originally Posted by Boston007 View Post
Do away with that star chart and put that in words IN the review, forcing people to read it all the way through, hehehe
Various critics have tried to do away with star ratings in their reviews over the years. Readers go ballistic, and the publication inevitably brings the star ratings back.
__________________
Josh Z
Writer/Editor, High-Def Digest (Blog updated daily!)
Curator, Laserdisc Forever
My opinions are strictly my own, and do not necessarily reflect those of this site, its owners or employees.
Reply With Quote
  #10  
Old 03-28-2013, 12:33 PM
Senior Member
Thread Starter
 
Join Date: Feb 2011
Posts: 3,004
Default

Quote:
Originally Posted by MrAngles View Post
My recommendation to improve the rating score methodology is for people to read the actual review.
(Don't want to be accused of repeating, so I'll just repost this answer.)
"Look I understand nothing replaces the actual review itself. The explanations,
the examples, the details of a review is best in describing what is it worthy/
unworthy about the disc. But since we are using a methodology in producing
rankings, why not improve our methodology?"

Quote:
Originally Posted by MrAngles View Post
The star section is the least important section of the review page...
The least important part is probably telling what kind of keepcase or what the
trailers are, though I do appreciate the thoroughness of the reviews. But the
Overall Stars Rating is treated with importance, have you not seen the posts
agreeing & disagreeing with the ratings? It's like saying sports are the least
important part of people lives, but have you seen what the world is like during a
Super Bowl, March Madness, or World Cup?

In this world, we keep score. We just do it. Number of stars do matter. Some
philosophers have tried for merit, for based-on-needs, etc. Those philosophers
usually get killed or die hungry. Small steps first. Let's improve our methodology,
instead of changing how the world is.

Quote:
Originally Posted by MrAngles View Post
I don't understand why people care enough to even think about the methodology involved in determining it.
I have the exact opposite view. If someone disagrees with an Overall Rating (and
who hasn't?), to have critical viewpoint you DO need to know the methodology
behind it. How can I complain about the results of a Monopoly game, if I don't
know the rules of Monopoly? Yes, I can wildly, mindlessly complain about a film's
star rating; or instead, I can critically understand it then attack/defend it.
Reply With Quote
Reply

Related Topics
Thread Thread Starter Forum Replies Last Post
Hairspray overall score doesn't add up glocks out Feedback 9 11-15-2007 12:04 AM
What movie will have the first 5 star overall HD disc rating? tardsgonewild High Definition Smackdown 20 06-04-2007 07:09 PM
our review methodology jed Feedback 5 04-24-2007 02:01 PM
Sony's Methodology Dragonbait High Definition Smackdown 36 02-09-2007 03:51 PM
Sort Reviews by Overall Rating booner Feedback 4 12-06-2006 12:04 PM


Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off
Trackbacks are Off
Pingbacks are Off
Refbacks are Off