Attn: All (Self-proclaimed) Golden Ears - High-Def Digest Forums
Reply
 
Thread Tools Search this Thread Display Modes
  #1  
Old 11-16-2008, 04:24 AM
vinnie97's Avatar
Senior Member
Thread Starter
 
Join Date: Sep 2007
Posts: 9,897
Default Attn: All (Self-proclaimed) Golden Ears

The 2005 double-blind ABX test to which I commonly allude has finally been followed up with another (this time with different MP3 encoders, all at 128kbps). It is currently ongoing and is scheduled to be finished in a week (if enough results are obtained). This is an invitation to those here who would automatically claim such a test would involve easy differentiation between the lossy and lossless sources. Read all about it and get set up here:

http://www.listening-tests.info/mp3-128-1/
with more information contained within the readme: http://www.listening-tests.info/mp3-128-1/readme.htm

Excerpt:

Quote:
A lot of development happened since the last public MP3 listening test @ 128 kbps conducted by Roberto Amorim. All encoders experienced major or minor updates that should improve audio quality or encoding speed and we have a totally new encoder on board.

The purpose of this test is to find out which popular MP3 VBR encoder outputs the best quality on bitrates around 128 kbps.
Test discussion: http://www.hydrogenaudio.org/forums/...howtopic=66564
Reply With Quote
  #2  
Old 11-16-2008, 05:12 AM
Senior Member
 
Join Date: Feb 2008
Posts: 509
Default

OK I read all the links but I am still not sure how this test works and how the results are empirically relevant. There are zero factors under control other than the rip settings. IMO all this crap is virtually impossible to prove.
Reply With Quote
  #3  
Old 11-16-2008, 10:35 AM
damonous's Avatar
Senior Member
 
Join Date: Jan 2007
Posts: 1,600
Default

Quote:
Originally Posted by oxbladder View Post
OK I read all the links but I am still not sure how this test works and how the results are empirically relevant. There are zero factors under control other than the rip settings. IMO all this crap is virtually impossible to prove.
Add to that, that the first link with the details on the actual test say that the testing ended 11/03/2008.

Also, even the 'rip settings' aren't really controlled, since they're doing a VBR test. ...And that's "Ok", too. --But, if these results are going to be used to say "MP3 achieves transparency with lossless", it's a little bit of a dubious prospect: there are a large amount of CE devices which support MP3 playback, but either do not support Variable BitRate, or perform poorly with VBR (such as improper playback speed, or timecode descrepancies). --Meaning that I would think most people listen to CBR MP3's, etc.; not VBR.

I would also agree that as a test, it's rather hard to prove a point when the test subjects are using varied audio systems, computers, playback devices. That forum-link even had a post with someone asking if a prior poster had used headphones. ---And, with this, I merely mean to point out that (on some "weak" audio-playback systems), the playback device itself may not have the range to properly reproduce the differences in audio quality. (In other words, if the speakers & amplification aren't "full range", then the test is already 'skewed' in favor of MP3/compression.)
Reply With Quote
  #4  
Old 11-16-2008, 11:29 AM
crazzeto's Avatar
Senior Member
 
Join Date: Nov 2007
Posts: 14,885
Default

It would be interesting to test this out, but anyway for those people that don't understand how a double blind test works and assuming they wrote a good double blind test here's the deal

The application should randomly switch audio files, these files should be the same segment of an audio clip mastered from the same source with only the compression codec being different. When the codec changes, it's your job to pick out which audio clip is the actual master source. If you're not able to hear a difference then that means all encodes are transparent to the master, if you are able to hear a difference then you grade which clips you like the best in order to determain how well the clips repersent the master.
Reply With Quote
  #5  
Old 11-16-2008, 11:46 AM
Malanthius's Avatar
Senior Member
 
Join Date: Nov 2007
Posts: 6,577
Default

Quote:
Originally Posted by crazzeto View Post
It would be interesting to test this out, but anyway for those people that don't understand how a double blind test works and assuming they wrote a good double blind test here's the deal

The application should randomly switch audio files, these files should be the same segment of an audio clip mastered from the same source with only the compression codec being different. When the codec changes, it's your job to pick out which audio clip is the actual master source. If you're not able to hear a difference then that means all encodes are transparent to the master, if you are able to hear a difference then you grade which clips you like the best in order to determain how well the clips repersent the master.
Problem is is that when you throw BR into the mix and how codecs pertain to some arguments. Then these tests are no longer valid.

Last edited by Malanthius; 11-16-2008 at 01:27 PM.
Reply With Quote
  #6  
Old 11-16-2008, 01:26 PM
vinnie97's Avatar
Senior Member
Thread Starter
 
Join Date: Sep 2007
Posts: 9,897
Default

Well, I suppose they could have all testers register what systems they are using. Generally speaking, though, for those who have familiarity with codec artifacts, they could get away with this test on a fairly menial setup. It seems to me that frequency response might be the only caveat with low-end systems.

The great thing is those whose testing results show that they've ranked the reference (aka master source) are discarded, so this helps to maintain the test's validity.

Damonous, I think the test was extended due to an unfortunate lack of interest. The tester is attempting to get at least 20 valid results per sample.

It's really a great forum and the tester is open to any issues, complains or concerns the forum members may have.

Malanthius, their validity doesn't need to be translated to BD codecs.
Reply With Quote
  #7  
Old 11-16-2008, 01:52 PM
Sbert's Avatar
Senior Member
 
Join Date: Jul 2007
Posts: 6,069
Default

please keep in mind that the brain doesn't work strictly vision and hearing. It integrates the two. If you eyes perceive something that will make a sound somewhere else, your brain will expect to hear such things. The nuances of the human brain can not be divided purely on isolated tests.

The brain also integrates body location with vision along with tactile sense. There's just too much that you loose taking out the visual affects.

Perhaps vinnie, you need golden eyes with golden ears
Reply With Quote
  #8  
Old 11-16-2008, 02:05 PM
vinnie97's Avatar
Senior Member
Thread Starter
 
Join Date: Sep 2007
Posts: 9,897
Default

Well, don't forget smell, touch and taste while you're at it. The presence of other senses won't diminish the findings of these tests. Perception is one thing but isolation is important to make any meaningful, scientific conclusions.
Reply With Quote
  #9  
Old 11-16-2008, 02:25 PM
Sbert's Avatar
Senior Member
 
Join Date: Jul 2007
Posts: 6,069
Default

Quote:
Originally Posted by vinnie97 View Post
Well, don't forget smell, touch and taste while you're at it. The presence of other senses won't diminish the findings of these tests. Perception is one thing but isolation is important to make any meaningful, scientific conclusions.
You're right, smell, touch and taste, smellovision (ketucky fried movie anyone)?

Anyways, I'm not saying the findings are diminished but it's just like medicine, you can always to a double control double blinded placebo trials and try to tease out the information you need, but unfortunately, real life doesn't work like that.

I'm sure eating a large meal, smoking a cigarette, drinking coffee, or doing drugs alter any of these situations.
Reply With Quote
  #10  
Old 11-16-2008, 02:29 PM
vinnie97's Avatar
Senior Member
Thread Starter
 
Join Date: Sep 2007
Posts: 9,897
Default

True! Hard to isolate all those variables...nay, next to impossible.
Reply With Quote
Reply

Related Topics
Thread Thread Starter Forum Replies Last Post
Puro Sound Reveals BT5200 Healthy Ears Headphones for Adults scohen Home Theater Gear 0 12-11-2015 12:39 AM
'That Man from Rio / Up to His Ears' is coming April 14 Landy Blu-ray Software General Discussion 0 02-05-2015 11:21 AM
Is there even one single, reputable, non-biased source that has ever proclaimed... brian451 High Definition Smackdown 42 11-28-2007 07:58 PM
Specs lie, your eyes and ears don't james High Definition Smackdown 25 11-03-2007 07:32 PM


Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off
Trackbacks are Off
Pingbacks are Off
Refbacks are Off