How to Evaluate CD Players -- and How NOT to Enhance Their Performance by Robert Thompson, "The $ensible Sound" magazine, Issue #72, Nov/Dec 1998, page 10-18. Copyright 1998, "The $ensible Sound" magazine

Assuming that you have read Part I of this series on CD players (Issue 71) and now know what really matters in player performance, I'll introduce you to some techniques for testing players in order to assess their tracking and error correction and concealment capabilities. Plus, I'll take a look at some interesting products -- some useful, some not. As an audio reviewer, I'm always on the lookout for tools that can aid in the evaluation of products. For the past few years, for example, whenever I have considered the purchase of a new CD player or been asked to advise a friend on the purchase of one, I would make sure to arm myself with a few different useful evaluation CDs. The first CD set I use for this purpose is from a company called Pierre Verany (PV.788031/788032). This set has a number of technical tracks for testing a CD player's capabilities. My favorite tracks on this particular CD set are tracks 25--50, which are designed to produce various dropouts to simulate CD scratches, stains, fingerprints, manufacturing defects, etc. (another similar CD I use is from the late Sound & Vision magazine's Audio Test CD, tracks 56--71). No player I am aware of is capable of cleanly playing through all of these tracks without some type of error(s). The second CD I use is CBS CD-1. I use track 20 of this disc to evaluate the low-level conversion linearity in players, (i.e., track 20 is a 500 Hz dithered sine wave that fades from --60 to about --120 dB). This track is perfect for conducting a subjective evaluation of the linearity of players at very low levels and helps to zero in on potential problems with the D/A converters, etc., (although this is rarely a problem in modern day players, if it is, this test finds it). Another useful low-level conversion linearity test disc is available from Sheffield Labs, My Disc (tracks 33--40). Another of the test CDs I often use is a music disc through which I have purposely drilled holes of various sizes (i.e., 0.750 mm, etc.). A variation of the homemade drill test CD is one I made out of a duplicate CD I had of Beethoven's Ninth Symphony. On the underside (the side the music is encoded upon) of this CD I have attached six 1/32" wide black-matte tape strips. These tape strips are arranged running on the bottom of the disc from the inside edge to the outer edge, (it looks somewhat like a pie). The purpose of these last two CDs is similar to that of the Pierre Verany and Sound & Vision CDs, i.e., to determine a player's servo-controlled tracking abilities and its error correction/concealment capabilities. Although these purposely-devised CDs are useful, they are not quite as accurate in the exact magnitude of the errors as the Pierre Verany CD set is. The last test CD I use is disc #1 from the Denon Professional Test CDs set (PG-6013-15). I am also currently in the process of making my own mathematically correct, dithered digital test signals. The CDs made from these will be used in the future evaluation of disc players. These signals are relatively easy to make on a personal computer (PC) using software such as Gold Wave, etc. Once the signals are generated, they are transferred from the computer's hard drive to a CD-R disc with a CD-R recorder. A New Tool for the Toolbag A couple of new tools have found their way into my bag of tools. The first is a CD titled CD-CHECK ($24.95 plus shipping, 902/425-1154) from a Canadian company, Digital Recordings. CD-CHECK has quickly proven itself to be every bit as useful -- in fact, more so-- as my previously mentioned CDs. This CD has uniquely engineered tracks designed to evaluate CD, DVD, and CD-ROM players for their error correction and concealment capabilities as well as a player's laser-servo tracking abilities. The truly nice thing about CD-CHECK is that its use is not contingent upon other test equipment beside that which God gave us two of -- our ears. All you need to do is to put this CD in your player and press the play button. CD-CHECK has five tracks, with track 1 being the easiest for any CD, DVD, or CD-ROM player to handle and track 5 being the most difficult. Each track has a 20-second continuous tone, with a 15-minute buffer in between tracks. According to the manufacturer, the 20-second tone is based on a complex signal that is designed to stress a player's error correction/concealment capabilities. The manufacturer states that the signal has embedded patterns that force a player into error correction and concealment. The continuous tone combined with the specially embedded signals allow a user to hear when a player's error correction and concealment circuitry are being pushed beyond their limits, (i.e., clicks, interruptions, skipping, looping, or complete player lockup). If you look at the bottom of the CD-CHECK disc, you will see something akin to my previously mentioned homemade CD of Beethoven's Ninth Symphony. The main differences between my homemade CD and CD-CHECK have to do with the far greater precision in which the errors are laid out on the CD-CHECK disc. My homemade CD has six strips of tape which act as dropouts, whereas CD-CHECK uses four precision-cut tapered strips per revolution. The CD-CHECK disc's precision tapered strips increase in size as a user progresses to a higher track. Another difference is the signals on the CDs, i.e., music on my homemade CD as opposed to the continuous tone encoded on CD-CHECK. Track 1 of CD-CHECK has no tape errors (the term "tape errors" refers to the physical blockage via precision cut and placed tape strips on the playing side of the disc). Track 2 has an error size of 0.375 mm; track 3, 0.75 mm; track 4, 1.125 mm; and track 5, 1.50 mm. In theory, an audio CD player should be able to handle burst errors of up to 4000 consecutive bits, i.e., this equates to an error which is approximately 2.38 mm in size. However, if attempts to correct the burst error are made via the CIRC system's interpolation, then errors of up to 12,300 consecutive bits are correctable, i.e., this equates to an error of approximately 8.16 mm in length. In reality, this type of performance is rare. I found CD-CHECK to be a superb complement to the Pierre Verany test set. After testing a number of players with both discs, the last two tracks on CD-CHECK proved to be the more difficult for the players to cope with. One thing I have never liked about the Pierre Verany or Sound & Vision CD sets is the fact that the dropout areas are simulations of disc damage. These disc sets have their dropouts encoded within the disc surfaces. The disc surfaces are not actually physically damaged with true scratches or inclusions. Although they work well, these discs are not as truly representative of real-world disc errors as they could be. CD-CHECK, on the other hand, has small perfectly sized and spaced taped markings on the playing side of the disc. In my opinion, these marks are more truly representative of real-world errors than the Pierre Verany or Sound & Vision disc sets are. Furthermore, the audibility of errors forced upon a player by CD-CHECK is much easier to hear than with the Pierre Verany or Sound & Vision CD sets. I was very curious as to the exact makeup of the special signals that were encoded on CD-CHECK, but the manufacturer of the disc did not wish to disclose this information. To gain a little insight, I performed Digital Audio Extraction (DAE) with my computer's CD-ROM drive of a 20-second segment off one of the tracks from the disc. DAE is the process of digitally extracting the exact data directly from an audio disc to one's computer hard drive, whereby the signal is stored as a wave file. After obtaining this wave file, I performed a Fast Fourier Transform (FFT) on the data and saw a clean 200 Hz tone clearly on the spectrum analyzer display. I strongly suspect that this signal is not just a 200 Hz tone, but rather a signal that is designed in such a way so as to cause the interpolation to incorrectly guess at any missing data and thereby give a more clearly audible error. In any case, this is a "stab-in-the-dark" on my part and the signals worked as advertised. Another New Tool for the Toolbag I guess the folks at Digital Recordings weren't content to have the best error correction and error concealment test CD on the market. Shortly after developing CD-CHECK, they developed and are now selling a unique device called the "Digital Error Detector" or "DED" for short (consumer version $199.95, professional version $299.95 plus shipping; cost includes one copy of CD-CHECK, 902/425-1154). The DED is a small, palm-sized electronic box designed to hook up to a DVD or CD player's digital output. In brief, the DED is able to visually show errors visually as they occur during playback. On the consumer model, errors are shown via a red LED, while on the professional version a 6-digit LCD counter has been added to "count" the total errors detected. The DED absolutely requires that your player have either the consumer type SPDIF (Sony/Philips Digital Interface), or the professional type AES/EBU (Audio Engineering Society/European Broadcasting Union) digital output. DED accepts both optical and coaxial digital outputs from players or DAT decks. DED can also be used in series with other digital devices if so desired, (such as a line tap). Both versions currently support a maximum sampling frequency of 48 Khz but new models are being developed which support up to 96 kHz, (upgrading will be made available to existing DED owners for a nominal cost). DED requires a wall wart (power supply) that supplies between 6--12 volts of DC power with a minimum of 100 Ma capability in order to operate. These type units can be purchased at Radio Shacks throughout the country for approximately $10. It's important to realize that DED can operate on a continuous basis and its use is not limited to detecting errors with special test discs. DED can be used with any music type CD disc and detect errors on a continuous basis. DED will not detect errors with either DVD Dolby Digital encoded or DTS type discs. However, DED can be used to detect errors on Digital Audio Tape (DAT) players. A Little Technical Sleuthing on How DED Works When I first received the DED and started using it I was curious as to what the device was actually "detecting." The manufacturer of the device did not wish to disclose this type of information. I understood this, as the manufacturer wants to protect the intellectual property. However, I was not dissuaded from my quest for knowledge, since I knew that the data stream output of both SPDIF and AES/EBU standards are available to anyone from the AES or IEC respectively, (for a fee of course). In order to determine what the DED was detecting I had to put on my detective's hat and search out the standards for both SPDIF and AES/EBU for digital output data streams. I was able to obtain some answers after much searching and asking a multitude of smart individuals on both the SPDIF and AES/EBU committees a number of questions. I also owe a debt of thanks to the applications engineers at Crystal Semiconductor, who provided a lot of helpful information (Crystal Semiconductor's digital audio transmitters and receivers are used in most of the players). My reason for being a little suspicious of the DED at first was that I originally thought that the error correction and error concealment flags to be "internal" to the player. By the time the data left the player, any error correction or error concealment had already been accomplished and the data was leaving the digital output of the player in as good of condition as possible. If the data on the disc were so poor that they could not be corrected via error correction or error concealment then the player would "mute" the data rather then send out grossly bad data. I had originally thought that the device was counting the "muting" signals in the data stream. However, more research changed my thoughts on this matter. Based on my research I strongly suspect that the device is using the V-bit (validity bit) on the interface. This is often flagged by devices when uncorrected errors are detected, but is very widely open to interpretation within the standards, because the V-bit was originally intended to indicate that the sample was suitable for conversion to analog audio. In fact it is more appropriate to call the validity bit an "invalidity bit," because it is true when the sample is invalid. It's important to understand that not all devices raise the V-bit when uncorrected errors are detected. However, it is the only widely used error flagging method used on both the AES/EBU and IEC 958 that I was able to find. It appears that the validity bit is one of the most controversial and misused parts of the interface. A second -- though less likely -- way that DED might be able to detect errors is via the user-data bits. In brief, the AES/EBU standard allows for a CRC error correction code to be transmitted in every frame of the AES3 interface to allow the digital receiver to correct for transmission errors. It is possible for a player to put data in the user-data bits of the AES3 frame to indicate any errors encountered during the reading of the disc, but that is not mandated by the standard. If DED were to use this method of error detection it would be dependent on how the manufacturers choose to use the user-data bits, (i.e., each manufacturer could configure this differently). The important point to remember with regard to my second hypothesis on how DED works is that I am referring to the CRC bits in the channel status and the possible use of user-data bits which could, theoretically be used to send error flags. This approach however, is unlikely, and it is more likely that the device is using the aforementioned V-bit for error detection. The bottom line is similar to my guesses about CD-CHECK: I'm not 100% certain that I'm correct with regard to how either work. I suspect that I am correct, or close to being correct, but cannot say this with 100% certainty. In any event, both devices work as advertised. How Well Did CD-CHECK & DED Work? To evaluate the usefulness of CD-CHECK and DED I went to my local audio shops and tried them on a multitude of CD and DVD players. Both CD-CHECK and DED were both very informative and fun to use. I found the majority of CD and DVD players were able to pass through tracks 1--3 without any errors. Players began to show audible and visual (via DED) differences starting with track 4 of CD-CHECK. When playing track 4 of CD-CHECK, each time an audible error occurred the DED device caught the error as well. However, when playing tracks 25 through 32 of the Pierre Verany test disc with its specially constructed dropouts, errors were caught by the DED prior to their becoming audible. On the subsequent tracks of the Pierre Verany test disc, errors became audible and their audibility coincided with the errors caught by the DED. What really separated the men from the boys among players was when I played track 5 of CD-CHECK. Playing this track really pushed the error correction and concealment capabilities of the players to their utmost. Using a combination of test discs such as CD-CHECK, Pierre Verany, and my homemade test discs made it easy to determine which CD or DVD player had the best error correction and concealment capabilities. Of all the test discs I owned, however, CD-CHECK proved to be the most revealing and hence has become my overall favorite. Both CD-CHECK and the Pierre Verany sets did a great job of finding the "dud" players; however, CD-CHECK proved to be the most challenging for players to cope with and the errors were more audible. CD-CHECK therefore proved to be the most revealing of the differences in tracking ability and error correction and concealment between the top-model players. After trying CD-CHECK and DED on various players at the local audio shops I decided that I would try a different experiment. For this experiment I took a CD player and put a very small amount of dust from my vacuum cleaner directly onto the laser diode. I then played CD-CHECK and noticed visual (via DED) and audible dropouts on track 1, which as you'll recall, is the least stressful track for a CD player to pass. After this, I removed CD-CHECK, unplugged the power to the unit and cleaned the optical mechanism. To clean the optical pickup I used a small burst of compressed air. After blowing off the majority of the dust from the optical pick-up, I physically cleaned the optical pick-up with a camera lens tissue and some optical cleaning fluid I purchased from a local camera store. As a note of caution, readers should normally not try this type of cleaning at home. If you forget to unplug the unit and the optical pick-up comes on and emanates light as you are looking directly upon it, eye damage could occur. Also most home CD, DVD, or CD-ROM players are designed so that you would have to take a portion of the unit apart in order to get to the optical pickup. Doing this may void any warranty you may have. Last but not least, there's always the potential problem that static electricity presents, i.e., it's easy to destroy sensitive electronic components. If you're interested in cleaning your laser assembly I recommend the Geneva "Turbo Disc--CD Lens Cleaner." This disc is specially constructed with small fins that "blow" dust and particles off the optical pickup. This is far and away superior to any type of CD player "cleaner" that uses any type of physical contact via brushes, etc. These latter items can actually do more harm then good -- so buyer beware! After cleaning the laser diode I again played CD-CHECK. The CD player's performance improved, as it played through tracks 1--3 of CD-CHECK without any audible errors. From this little experiment I was able to verify one of the claims made by the manufacturer: that CD-CHECK and DED could be used to verify improvements made by laser cleaning. Admittedly, this type of test was somewhat unrealistic, as I can envision no circumstance whereby a player's laser diode would become so dirty. This test does, however, highlight the fact that a dirty laser diode can lead to incorrect playback of discs. The last experiment I did was to take a CD containing music and purposely damage it by scratching the playing surface. The scratching of the surface was accomplished by rubbing the playing side of the CD back and forth on a block of wood. I proceeded to scratch the CD in stages, starting slowly at first, playing the disc and noting the error count with DED. During the fist stage, where only light scratches were on the CD, the DED was showing errors, but the player's error correction and concealment circuitry were able to compensate for these without any audible signs of error. However, as I proceeded to stage two, which of course resulted in still more scratches to the playing side of the CD, the error count on the DED went up and the scratches became slightly audible. Going to stage 3, wherein the disc had numerous scratches all over its playing surface, the DED counted still more errors and the disc errors became clearly audible. This experiment clearly demonstrated to me that the DED device worked as advertised. One interesting note is that the error counts did not remain absolutely consistent even when playing the exact same track of the disc for a timed duration. This probably relates to the fact that the players' servo-tracking systems do not track exactly the same way each and every time they are confronted with a damaged disc. To get around this I would take an average of three different error rates on the same track over a predetermined time duration. Comparing averages this way showed good test/re-test repeatability. Summary of CD-CHECK and DED's Value Digital Recording's CD-CHECK proved to be an outstanding tool for checking the quality and serviceability of CD players. It has found a welcome home in my bag of tools when evaluating players. Highly recommended! DED is a unique and useful product as well. The nicest thing is that it can be used with any audio CD, be it music or a specially constructed test disc. For my own purposes, I prefer the professional version to the consumer version, as the pro version's LCD counter helps to quantify errors and makes comparisons between players easier. I only found one DVD player that the DED stated it had correctly "linked to," but that received no error signals whatsoever, (even though the errors were grossly audible). This device was the Sony DVP-5500D DVD player. I think the reason the DED was unable to receive the signal relates to the loose standard which manufacturers have regarding the use of the previously mentioned V-bit. Even so, I tested over 20 different DVD and CD players and only one did not work correctly with DED. I therefore conclude that this type of performance is atypical and that most manufacturers' players provide the necessary data in their digital output that allows the DED to work correctly. Another plus is that DED can detect errors with DAT players as well. I therefore give an enthusiastic "two thumbs up" to DED as well. It's a super product and is the only one of its type of which I am aware. "Disc Enhancement" Products -- Do They Really Work? Having satisfied myself that CD-CHECK and DED worked as claimed, I decided to give some of the more esoteric audio products a run for their money. After all, now that I had a way of objectively quantifying the errors introduced using these tools, it made sense to substantiate or refute the claims made by various manufacturers of disc enhancement products. Note that I did not personally purchase any of the following products myself (I don't like to throw my money away). Different audiophile friends who were nice enough to loan them to me for these tests purchased the products. Also note that each test was tried in three different manufacturer's CD players. This was done to rule out the possibility of one player not allowing the products to be shown in a favorable light. Last, but not least, I was unable to obtain all the tweak disc enhancements I've seen advertised. Since I was relying on the kindness of my audiophile friends and did not wish to purchase the products myself, I was at the mercy of what they had to offer. For example, I had wished to conduct this test with CD-Backlight and some CD stabilizer rings I had seen advertised. Alas, none of my friends had these items and therefore they were not tested. I don't suspect I would be overly impressed even if they had been available. (OK, I'm a cynic). The first and second products I tried were something called "Statmat" and a product called the "Orpheus CD tuning mat." The marketing types who push the "Statmat" would have you believe that the Statmat's specially engineered conductive polypropylene film is specially formulated with conductive inks that somehow mean that you obtain vastly better sound. Supposedly, this diaphanous sheet of plastic sits on top of your CDs and helps to control static build up with can degrade the sound of your player. In much the same vein, the makers of the "Orpheus CD tuning mat" claim that its use will result in a vast sonic improvement. Using the DED I tried the Statmat and the Orpheus CD tuning mat with three different discs both with and without the mats. In each case, I noticed absolutely no audible improvement in the sound quality with the Statmat or with the Orpheus CD tuning mat. Conversely, I did not notice a decrease in listening quality either. To my ears, the sound remained unchanged. However, the DED error count rose higher whenever either the Statmat or the Orpheus CD tuning mat was used. This happened in every case. Apparently, the use of the Statmat or the Orpheus CD tuning mat actually resulted in a decrease in data quality, although the player's error correction and concealment circuitry compensated for this with no audible differences being noted. I cannot recommend that anyone buy or use either of these products. The third through sixth treatments I tried were "coating oil" made by Yamamura Systems; "anti-static spray" made by Nordost; and "boundary layer optical fluid" and "compact disc edge treatment"made by Marigo Audio Labs. The Yamamura Systems "coating oil" is said to form a thin film surface on the disc which is claimed to improve the disc's "transparency and light permeability." The Nordost "anti-static spray" is said to collapse the magnetic field generated by signals passing through metal, (but they advertise it can also be used on CDs as well). The "boundary layer optical fluid" is said to result in increased clarity, dynamics, and low-level resolution from discs that have been treated with it. The "compact disc edge treatment" is a green marking pen which can supposedly improve the performance of any player when a disc is played which is so treated. I tried all of these products on a number of discs and in no case did any of them result in an audible improvement in the sound quality or in a decrease in the error count as evidenced though the use of the DED. On some of the products I was suspicious of the chemical reaction between the disc surface and the treatments themselves. What would be the long-term consequences of using these chemicals on a plastic disc surface? Would their use promote the breakdown of the disc surface itself? My recommendation is that you leave these products on the shelf as well. The last product I tried was the "Bedini Ultra Clarifier." The product looks like a small box. You put your CD inside and then push a button. The device spins the disc at a high rate of speed over magnets, supposedly "degaussing" the disc, the end result purport to be a disc with increased fidelity. Despite the fact that this entire process seems to make absolutely zero scientific sense I tried it anyway. As I expected, treated discs resulted in zero difference --- both audibly as well as quantitatively as shown by DED. My recommendation is that you stay away from this device and don't waste your money. The bottom line on all these supposed disc enhancements is the fact that they don't work --- period! As the saying goes, a fool and his money are soon parted. If any of these devices had actually proven to result in better sound, (when proven via double-blind testing) or in reduced error rates on DED I may have felt differently. Readers would do better to hold on to their hard-earned cash and invest the difference in more discs.

Excerpted with permission from Issue #72, Nov/Dec 1998, of "The $ensible Sound" magazine. tel. (716) 833 - 0930, fax. (716) 833 - 0929, E-mail: SensiSound@aol.com Subscriptions to "The $ensible Sound" magazine are available for $29 US per year (6 issues) by check to 403 Darwin Drive, Snyder, NY 14226 or through Visa/MC by phone ( 800 - 695 - 8439 ). You may also purchase a set of all available back issues (currently 50+ issues) for $89 US, shipping included.

Terms of Use | Return Policy | Privacy Policy

Copyright (©) 1996-2014 by Digital Recordings. All Rights Reserved. No part of the information provided on this www page may be reproduced for any purpose, in any form, without prior written approval.

This site uses frames. To enjoy them your screen's resolution should be at least 800 x 600, preferably 1024 x 768. To invoke frames click here.