our review methodology

[jed]

More than any other feature on High-Def Digest, our software reviews generate by far the most feedback. Our forums (and our email box) are routinely flooded with reactions as varied as our readers themselves, whether they're writing in to praise our glowing review of a given title, or to lament how we in the world could possibly give another title a poor rating. Amid all this, the most oft-asked question we receive isn't about the details of any one particular review, but the hows and whys of our approach to disc reviewing itself.

To address all those inquiries, we've created this detailed overview of our Review Methodology. Breaking down each component of our reviews, we hope this will offer a thorough explanation of our systematic assessment of any given disc title. From how we approach dissecting the components of next-gen video and audio performance to the application of our (sometimes controversial) star ratings, it explains our philosophy toward balancing disc quality, artistic intent and supplemental content to formulate a ranking of overall value for your consumer dollar.

http://www.highdefdigest.com/reviewmethodology.html

[Bruuce]

Thanks for posting this. I'm sure alot of people will find it interesting and full of information just like I did.

[Tejstar]

A very thorough article and interesting read. Thanks Jed & Peter!

[enigma]

This was a good writeup. Its good to see what the metholody is.

I am curious about the "Detail" section.

Detail: How well are fine textures and objects visible in an image? Do close-ups look natural, life-like and vivid? And in long-shots, can you make out far-away-objects, or are they just a mass of undefined blobs?

I don't think this is a correct way to evaluate detail. Many films and graphical content intentionaly blur the background either by choice of lens or in the case of CGI a depth of field effect.

If the point of interest in the scene has detail but other points do not, thats a good indication it is as intended. The example shown for "Detail" could easily have been intentional. Its not something that should be marked down.

Now if the entire scene is devoid of detail then there is a problem and the score should reflect this.

[edit] For those unfamiliar with Depth of Field and why parts of the images are expected to be blury, please refer to this link
http://en.wikipedia.org/wiki/Depth_of_field

[nelll]

A very good article.

[HomeTheatreFreak]

One problem I have seems to be your rating every release on the same scale and the problem with such practices is now showing up in the Planet Earth review. Nature documentaries should not be put on the same scale as Hollywood films period. Planet Earth should have received at least a 4.5 overall rating with maybe a half point being deducted for no lossless audio when in reality it received a 3.5 rating which is utterly unbelievable. What this site is ultimately saying is the National Geographic release of Relentless Enemies for HD is the same score as Planet Earth and that again is utterly ridiculous at best.

First documentaries should have separate rating scales that are not the same as movies. I am not saying that they should be given leeway for lesser quality sound and audio, just that they are very different releases and the expectation for documentaries are very different from hollywood movies. First extras are very rare in documentaries and are not expected to be mandatory like with movies. If they have extras great but if they dont they should not have points taken away because of that fact. In all honesty I own about 200 documentaries from the BBC, National Geographic, Horizon, Nova and PBS and out of all of them I think maybe 10 have legitimate extras included. Again its just a very different release structure than movies and these differences should be taken into account.

The fact that Planet Earth scored a 3.5 should be a sure fire sign that something is wrong with your system for judging documentaries and the fact that Relentless Enemies scored the same score as Planet Earth should be another huge sign that something is seriously wrong with the current system for rating documentaries. I for one certainly hope you do something regarding this because any bad score that 99% of the public would disagree with will ultimately hurt the overall respect people have for this site and its reviews.

Just my 2 cents.