No, that’s going way too far, but Consumer Reports still deserves more well deserved criticism. I want to expect better, because CR is run by a non-profit corporation that does not accept outside advertising and buys tested products at regular retail outlets. They aren’t getting freebies from a PR agency or a manufacturer’s corporate communications department, so you have to believe that the tested product is the same as regular people buy. No ringers allowed.
Despite those qualifications, CR just can’t get tech gear right. More to the point, their undefined test methods result in skewing the ratings against Apple in curious ways, without an iota of apparent support with facts and figures. Worse, ratings are buried under generic labels, with no indication how the conclusions were reached.
So in the August 2012 issue, for example, the iPhone 4s gets, at best, middling ratings, a 67, compared to the other smartphone offerings from the major wireless carriers in the U.S. Now such ratings would be deserved if all the other products examined were demonstrably superior, but there are serious questions about CR’s test standards and results.
Take the ratings of 11 Verizon Wireless smartphones, topped by the Droid Razr Maxx with 78 points. The main advantage of the Maxx line is the larger battery, which, on the basis of its high capacity, offers more talk time than any other tested smartphone, even on an LTE network. All right, that’s a positive. But the rest of the ratings make far less sense. So all the Android smartphones are credited with better “Messaging” than the iOS, even though Apple’s exclusive iMessages feature, which will come to the Mac when OS 10.8 comes out, ought to count as a major advantage for Apple. Although the Safari Web browser is generally regarded as equal to our superior to the browser on Android phones in the detailed tests I’ve located, CR considers it merely Good, compared to the Excellent ratings on Android gear.
According to just about every benchmark comparing iOS versus Android browser performance, Safari is demonstrably better in HTML5 fidelity. Or maybe CR hasn’t a clue what HTML 5 is. In terms of rendering speeds, they are usually close, meaning that differences aren’t going to be noticeable. It’s more about the speed of your Internet connection. Or maybe CR doesn’t understand that either.
As usual, CR doesn’t rate the quality of the OS. They are just included among the features for reference, with an “A” for Android, an “I” for iOS, a “W” for Windows Phone, and a “B” for BlackBerry. Which is better? CR won’t tell you. Which version of Android do you get on those smartphones? CR overlooks the fact that many owners of Android gear are saddled with OS versions over a year old, with no possibility of getting an over-the-air update, even to receive a critical security fix.
It’s also true that CR just adores larger screens. They are wowed by the “superb 4.7-inch” display on an HTC One X, without saying anything about the convenience of using such a large smartphone with one hand or taking it on the road with you. You’d think a publication that is so concerned about the failings of autos when it comes to ride, steering, handling, seat comfort, and the ease of use of dashboard controls, would also consider the possibility that a smartphone’s display might be too large, that the case might be unsuitable for convenient storage in a pocket or a purse. Or maybe they hired a bunch of clowns to perform customer evaluations, or require their testing staff to wear clown uniforms, so they can carry the larger stuff.
When it comes to tablets, the ratings might be expressed with ditto marks. Regardless of OS, every tested model gets a Very good rating for “Ease of Use,” whatever that’s supposed to signify. It doesn’t seem as if the OS has anything to do with it. In the end, among the larger tablets, the differences in scores from the highest rated third generation iPad to the worst rated Motorola Xoom are not significant. The iPad’s Retina display appears to be the only defining difference, and not one that seems significant to CR’s editors, despite their love of large displays on smartphones. Worse, CR seems oblivious to the well-known problems with touch responsiveness on Android gear.
But at least they aren’t complaining about the new iPad running too hot. They tried to get away with that scheme a few months back, but the test results were still acceptable. Of course, iPads ran a lot cooler when tested by other publications, but CR is not always about consistency or reliability.
To the CR reader, it would appear that all tablets rate very close to one another, and you might as well get the model that’s cheapest. Why bother to consider any other factor in deciding which one to buy? Well, the public has decided, despite CR’s unfortunate efforts to find equality where it doesn’t exist. Android tablets are still failures.
There’s also a brief survey entitled, “Faster laptops make a debut,” but CR included last year’s MacBook Pros among the ratings. I’ll cut them a break here, because the 2012 Mac note-books no doubt arrived too late to be included on the list. I could tell you that CR doesn’t understand the differences between OS X and Windows, but you knew that already.
And did I tell you that CR also reviewed chain restaurants in that issue?