Hello, Steve,
I will tell you a little story about a bad test design and a good test
design.
The bad test started at a lecture I was giving. Part of my standard
lecture shows "what is possible" at a given era by highlighting
high-quality sound from a given era (going back to a 1935 steel tape
copy). Someone asked about MP3 and I had one of the 1980s selections
both in my demo as a WAV file and on my Palm T3 as an MP3. The Palm
sounded way worse than the Sony CD walkman I was using for the rest
of the demo.
When I got home, I took the original file, made a high-quality MP3
within Samplitude, converted it back to WAV and then cut between the
two recordings. I now demo the cut recording and the MP3 is almost
identical to the WAV file.
So, just as with A-D and D-A converters and even CDs themselves which
over time have, for some people, received the reputation of "not
sounding good" for perhaps the wrong reasons. Clearly, here we were
hearing the deficiencies in the Palm T3 audio system as opposed to
the deficiencies in the MP3 format.
I believe that my test is one of the few ways that one can do a test
and remove most of the external variables. I'm passing this story on
as an object lesson and as a caveat to anyone doing a listening test:
make sure that you're really listening to what you think you're
listening to and do NOT make assumptions. I believe that it is almost
impossible to do the test that you describe using A/B hardware
without the hardware differences influencing the rating of the format.
Oh, and I emailed the organizer of the lecture this explanation and
requested she mail it to all attendees. I think she did.
If you want the resultant WAV file I would be happy to share with you.
Cheers,
Richard
At 09:32 AM 2007-06-13, you wrote:
>Has ther been anything published in recent years that addresses
>actual listening comparisons between MP3 and CDs? I'd prefer they
>have split the panels' sources into those that are acoustical (i.e.
>begin by pushing air)from those which start by exciting
>electrons. It would be helpful if those doing the reacting were
>identified as professional or casual listeners as well.
>
>I 'm not looking for indivual reactions in print but a designed and
>controlled test. Anything out there?
>
>Steve Smolian
Richard L. Hess email: richard@xxxxxxxxxxxxxxx
Aurora, Ontario, Canada (905) 713 6733 1-877-TAPE-FIX
Detailed contact information: http://www.richardhess.com/tape/contact.htm
Quality tape transfers -- even from hard-to-play tapes.