If you accept the conventional wisdom, Consumer Reports is the best product review publication in the U.S. The reason is that it’s run by a non-profit corporation that not only doesn’t accept advertising, but actually purchases the items it tests, most often from regular dealers. In other words, it’s incorruptible.
So how can you beat that?
Unfortunately, there’s also that old saying about being a jack of all trades and the master of none that applies very much to Consumer Reports.
Yes, they seem to do a credible job at auto reviews, and on such ordinary consumer products as dishwasher detergent, washing machines, and even digital cameras and flat screen TVs. But the situation becomes troublesome when they confront such sophisticated gear as personal computers and smartphones.
Even though they seem to have a more tech-savvy staff nowadays, their reviews remain useless when it comes to the promised intent, which is to guide you towards making the right purchasing decision.
Take the eternal Mac versus PC dilemma. You never see detailed descriptions of the differences between the Mac OS and Windows in the pages of CR. They may pay lip service to major upgrades, such as Snow Leopard and Windows 7. But when confronted with a choice between the two, which should you pick?
CR won’t tell you why one might be better than the other, or even, in their collective opinion, that there’s no practical difference. Of course, I’d dispute that heavily, as would most people who have had a fair amount of experience with the Mac and the PC, but the question isn’t even asked in their annual reader surveys.
You do see Apple getting high marks, because Macs contain premium hardware. But you might come away with the impression that the Mac is nothing more than a high-priced PC, so why even give it a second thought if you’re not prepared to pay extra for one?
To CR, it’s the same as choosing whether to buy a Toyota or a Lexus. In this case, it’s the same company, sometimes similar platforms, but it’s mostly about style, right?
CR does, however, have a knack for getting solid press. You’ll find their spokespeople on many TV shows, not to mention being quoted regularly in the mainstream press. But they are also given softball questions and never pressed to confront the serious lapses in their testing process.
That takes us to Antennagate, where CR became a major offender, along with the Web-based tabloid publication that is credited with (or blamed for) disclosing the notorious signal attenuation problems encountered on the iPhone 4. Now maybe the official offender, Gizmodo, has an ax to grind, having gotten themselves embroiled in that notorious missing or stolen iPhone prototype. It stands to reason that they are no longer included on Apple’s official invite list for media events, and good luck to them if they hope to get review products in the future.
Unfortunately, the Antennagate controversy quickly became a YouTube sensation, then got picked up by the major media, particularly after CR was able to duplicate the phenomenon in a specially-configured test that an independent antenna expert regarded as essentially junk science.
Although the iPhone 4 got the highest rating among all tested smartphones, by a small margin, CR won’t recommend it. It doesn’t matter that you can pretty much take any smartphone, as many people have done, and find an appropriate death grip to cause the signal to dip.
Apple has been busy exposing blatant offenders from rival companies on their site, as the executives from those smartphone makers continue to cry foul.
But you wonder how CR might have missed the obvious warnings in manuals and, on occasion, affixed to the smartphones themselves, which explained the potential for signal loss and perhaps lower battery life if you hold those gadgets the wrong way. You’d think a publication that prides itself on being thorough (not to mention fair) would actually read the manuals and make an effort to understand a product’s known limitations.
On the other hand, posting a blog about the iPhone generates far more attention, and hits of course, than if they identified a similar shortcoming with a gadget from HTC, Motorola, Nokia or Samsung. After all, we’re talking about Apple’s “Jesus phone” here, the one that’s magical, mystical and therefore cannot possibly obey the laws of physics.
What’s even more troubling is that CR refuses to admit the testing was not properly conducted. Rather, they imagine that Apple will have an unknown fix at some undefined moment in the future, and, while saying that the offer of free bumpers or third-party cases is a good first step, it’s not enough.
If CR wants to be fair and balanced, they should be demanding the same of every other smartphone maker; that is, free cases. That way, all the products will be on an equal footing, and not susceptible to signal variation under normal use and service.
But I’m not expecting that to happen anytime soon. As long as Apple continues to have difficulty meeting iPhone 4 demand, there’s little to be concerned about. But my respect for CR has gone down several more notches.