Quantifying Digital Audio Player Performance

by Bill Machrone
Copyright 2005 Bill Machrone

This Web page is a companion to my column, "Shuffle's Got a Secret" in PC Magazine.

It's an attempt to quantify the performance differences among several digital audio players. While all have essentially flat (or flat-enough) frequency response with sine wave sweep tones and less than 0.1% THD at 1KHz, there are audible differences when driving professional headphones or self-powered studio monitors. All can drive a standard set of earbuds (in this case, the standard Apple earbuds) to 100+dB in-canal loudness using a rock test track that fills virtually the entire audible spectrum with a constant din.

Most of the difference is in the bass response, but no differences are discernible from one unit to the next with a 40Hz sine wave (roughly low E on the electric bass). After performing many different tests, I came up with the following test tracks, which I loaded onto each of the players:

  • Pink noise, first with no load, then the same pink noise file with standard Apple earbuds plugged into the unit
     

  • a 40Hz square wave, with and without the earbuds.
  • The 40Hz square wave is an extreme test, since it asks the output stage to act like a servo amplifier, driving to a full voltage, holding it at that level, then driving back the opposite way.

    This is really a test of the output stage and possibly the power supply, but it's been noted many times that an amplifier is nothing more than a modulated power supply.

    All of the digital audio players do a decent job of reproducing the square wave without a load. With the earbuds plugged in, they generally have difficulty sustaining the voltage, and it collapses back towards zero. Some sustain better than others. One in particular sustains remarkably well and subjectively has noticeably better bass response.

    The five players I tested, in order the quality of their perceived bass performance in listening tests were:
     

  • iPod Shuffle
  • Zen Micro
  • Dell DJ 20GB
  • 15GB iPod (3rd Generation)
  • iPod Mini
  • All but one of these players use single-ended, capacitively coupled output stages. It's an inexpensive and effective way to deliver acceptable, if not superb performance, but the size of the coupling capacitor and the impedance of the headphones have a significant effect on the player's ability to sustain a complex bass tone.

    The screen shots below are just a small portion of the testing I did, but should give you an idea of the differences among players. The spectrum analyzer and oscilloscope are part of the Acoustic Analyzing System 5E, from www.ymec.com.

    I used an M-Audio Transit USB, an external USB sound card, as the audio interface. It has a high-impedance input that doesn't load the circuit under test, and it has no external adjustments for gain--no knob settings to screw up; identical results from test run to test run. It also has about 120dB signal to noise ratio and can sample at 96KHz, sufficient headroom for tests like these.

     

    This is the Dell DJ20, the latest model. Apart from some overshoot, player 1 does a pretty good job of generating an unloaded square wave. It has good subjective bass performance. Unloaded square wave performance doesn't really tell you all that much--unless the player can't form a decent square wave, as one below couldn't.

     

    This is the Dell DJ20 with the headphones plugged in. You can see how the voltage can't be sustained with the load, but players with curves like this still manage to sound pretty good.
    This is the Dell DJ20 with pink noise. The top line is the response without the headphone load; the bottom line is the response with the headphones plugged in. The load of the headphones has little effect on the upper frequency response, but the drop is noticeable at lower frequencies.
    The 15GB iPod does a near-textbook job of generating the square wave...
    ... and falls down badly when it comes to sustaining it under load. In my testing, players that were dragged down to zero by the load of the earbuds didn't sound as full as the ones that had a little left in in the tank at the end of the cycle.
    My son took the 15GB iPod back to college with him before I could run the pink noise tests. I did, however, record a 1/3 octave sweep test. It doesn't show no/load load. While the iPod is generally well-regarded, bass performance is not stellar. This sweep was run with the headphone load, and you can see the fall-off in the lower octave. The rest of the frequency response curve is flat, however.
    This is the original iPod Mini (not the new 6GB Mini) It has trouble making a clean square wave or even sustaining it when the player is unloaded.
    Once I plugged in the earbuds, the waveform of the iPod Mini really deteriorated.

    Audiophiles and high-end headphone manufacturers agree that Apple undersized the output capacitors on the Mini, whether for space, cost, or power reasons.

    With the pink noise test, this Player 3 shows a larger variance between the load and no-load lines than any other player in the test. The divergence, as expected, is greatest at low frequencies.
    The Zen Micro does a pretty good job of generating the square wave.
    The Zen Micro's signal sinks like a stone when it hits the earphone load, but it doesn't sink to zero. Subjective bass response on this player is pretty good.
    The Zen Micro shows little divergence under load from its no-load performance. This player did very well in listening tests and is well regarded.
    This is the iPod Shuffle. The square wave is just about perfect--the overshoot might be heard as a small amount of harmonic distortion, but is immaterial.
    With earbuds, the iPod Shuffle's signal looks darn near identical to the no-load signal. I checked and rechecked this result because I couldn't believe my eyes. The iPod Shuffle sounds great, with a solid low end, and no need for bass boost.

    The reason for this sterling performance is that the left and right channels each have two transistors, one pushing, one pulling, and no capacitor that gets discharged over time.

    The pink noise performance of the iPod Shuffle is, as you might expect, exemplary. The load/no load performance is very close, even at the deepest bass frequencies.

    You might be wondering why these pink noise charts seem to sink down as frequency goes up. The reason is that you're looking at a logarithmic chart, and the intensity of pink nose tapers off as frequency goes up. This is to avoid the problem of white noise, which has the same amount of power at every frequency. That sounds like a good thing, but in fact would mean that fully half of the sonic power would be above 10KHz, and sound just isn't like that in the real world. Pink noise is more representative.

    Update

    Since I wrote the column, Apple introduced the iPod Mini 6GB. I subjected it to some of the same tests as the other players, and as you can see from the screen captures, Apple has obviously improved the audio amplifier. The 6GB Mini sounds noticeably better than the original Mini.

    The curve gets pulled down to zero with standard Apple 32 ohm earbuds. We could have hoped for more capacity, but it's still better than the original iPod Mini.
    I didn't get a chance to run the load/no load sweep test on the 6GB Mini, but I did run a 1/3 octave sweep with the earbud load. You can seen from the graph that the lowest frequencies are still somewhat deficient, but there's definitely more bass from 30Hz on up.

    Copyright 2005 Bill Machrone