our review methodology - High-Def Digest Forums
Thread Tools Search this Thread Display Modes
Old 03-22-2007, 02:56 PM
jed's Avatar
jed jed is offline
HDD Founder
Thread Starter
Join Date: Oct 2006
Posts: 1,621
Default our review methodology

More than any other feature on High-Def Digest, our software reviews generate by far the most feedback. Our forums (and our email box) are routinely flooded with reactions as varied as our readers themselves, whether they're writing in to praise our glowing review of a given title, or to lament how we in the world could possibly give another title a poor rating. Amid all this, the most oft-asked question we receive isn't about the details of any one particular review, but the hows and whys of our approach to disc reviewing itself.

To address all those inquiries, we've created this detailed overview of our Review Methodology. Breaking down each component of our reviews, we hope this will offer a thorough explanation of our systematic assessment of any given disc title. From how we approach dissecting the components of next-gen video and audio performance to the application of our (sometimes controversial) star ratings, it explains our philosophy toward balancing disc quality, artistic intent and supplemental content to formulate a ranking of overall value for your consumer dollar.

Reply With Quote
Old 03-22-2007, 09:28 PM
Bruuce's Avatar
Senior Member
Join Date: Dec 2006
Posts: 1,006

Thanks for posting this. I'm sure alot of people will find it interesting and full of information just like I did.
Reply With Quote
Old 03-23-2007, 09:50 AM
Tejstar's Avatar
Super Moderator
Join Date: Oct 2006
Posts: 2,891

A very thorough article and interesting read. Thanks Jed & Peter!
Reply With Quote
Old 03-24-2007, 01:13 AM
Senior Member
Join Date: Jan 2007
Posts: 1,381

This was a good writeup. Its good to see what the metholody is.

I am curious about the "Detail" section.
Detail: How well are fine textures and objects visible in an image? Do close-ups look natural, life-like and vivid? And in long-shots, can you make out far-away-objects, or are they just a mass of undefined blobs?
I don't think this is a correct way to evaluate detail. Many films and graphical content intentionaly blur the background either by choice of lens or in the case of CGI a depth of field effect.

If the point of interest in the scene has detail but other points do not, thats a good indication it is as intended. The example shown for "Detail" could easily have been intentional. Its not something that should be marked down.

Now if the entire scene is devoid of detail then there is a problem and the score should reflect this.

[edit] For those unfamiliar with Depth of Field and why parts of the images are expected to be blury, please refer to this link

Last edited by enigma; 03-24-2007 at 04:24 AM.
Reply With Quote
Old 03-29-2007, 10:50 AM
nelll's Avatar
Senior Member
Join Date: Jan 2007
Posts: 4,606

A very good article.
Reply With Quote
Old 04-24-2007, 02:01 PM
HomeTheatreFreak's Avatar
Senior Member
Join Date: Feb 2007
Posts: 2,898

One problem I have seems to be your rating every release on the same scale and the problem with such practices is now showing up in the Planet Earth review. Nature documentaries should not be put on the same scale as Hollywood films period. Planet Earth should have received at least a 4.5 overall rating with maybe a half point being deducted for no lossless audio when in reality it received a 3.5 rating which is utterly unbelievable. What this site is ultimately saying is the National Geographic release of Relentless Enemies for HD is the same score as Planet Earth and that again is utterly ridiculous at best.

First documentaries should have separate rating scales that are not the same as movies. I am not saying that they should be given leeway for lesser quality sound and audio, just that they are very different releases and the expectation for documentaries are very different from hollywood movies. First extras are very rare in documentaries and are not expected to be mandatory like with movies. If they have extras great but if they dont they should not have points taken away because of that fact. In all honesty I own about 200 documentaries from the BBC, National Geographic, Horizon, Nova and PBS and out of all of them I think maybe 10 have legitimate extras included. Again its just a very different release structure than movies and these differences should be taken into account.

The fact that Planet Earth scored a 3.5 should be a sure fire sign that something is wrong with your system for judging documentaries and the fact that Relentless Enemies scored the same score as Planet Earth should be another huge sign that something is seriously wrong with the current system for rating documentaries. I for one certainly hope you do something regarding this because any bad score that 99% of the public would disagree with will ultimately hurt the overall respect people have for this site and its reviews.

Just my 2 cents.
Reply With Quote

Related Topics
Thread Thread Starter Forum Replies Last Post
Overall Rating Score Should Have Different Methodology timcharger Feedback 19 04-16-2014 02:59 PM
They may kill our format, but they will NEVER take our freedom! Trenchalicious High Definition Smackdown 114 02-18-2008 04:14 AM
Sony's Methodology Dragonbait High Definition Smackdown 36 02-09-2007 03:51 PM

Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off
Trackbacks are Off
Pingbacks are Off
Refbacks are Off