Is Consumer Reports Having its Revenge Against Apple?
July 10th, 2012No, that’s going way too far, but Consumer Reports still deserves more well deserved criticism. I want to expect better, because CR is run by a non-profit corporation that does not accept outside advertising and buys tested products at regular retail outlets. They aren’t getting freebies from a PR agency or a manufacturer’s corporate communications department, so you have to believe that the tested product is the same as regular people buy. No ringers allowed.
Despite those qualifications, CR just can’t get tech gear right. More to the point, their undefined test methods result in skewing the ratings against Apple in curious ways, without an iota of apparent support with facts and figures. Worse, ratings are buried under generic labels, with no indication how the conclusions were reached.
So in the August 2012 issue, for example, the iPhone 4s gets, at best, middling ratings, a 67, compared to the other smartphone offerings from the major wireless carriers in the U.S. Now such ratings would be deserved if all the other products examined were demonstrably superior, but there are serious questions about CR’s test standards and results.
Take the ratings of 11 Verizon Wireless smartphones, topped by the Droid Razr Maxx with 78 points. The main advantage of the Maxx line is the larger battery, which, on the basis of its high capacity, offers more talk time than any other tested smartphone, even on an LTE network. All right, that’s a positive. But the rest of the ratings make far less sense. So all the Android smartphones are credited with better “Messaging” than the iOS, even though Apple’s exclusive iMessages feature, which will come to the Mac when OS 10.8 comes out, ought to count as a major advantage for Apple. Although the Safari Web browser is generally regarded as equal to our superior to the browser on Android phones in the detailed tests I’ve located, CR considers it merely Good, compared to the Excellent ratings on Android gear.
According to just about every benchmark comparing iOS versus Android browser performance, Safari is demonstrably better in HTML5 fidelity. Or maybe CR hasn’t a clue what HTML 5 is. In terms of rendering speeds, they are usually close, meaning that differences aren’t going to be noticeable. It’s more about the speed of your Internet connection. Or maybe CR doesn’t understand that either.
As usual, CR doesn’t rate the quality of the OS. They are just included among the features for reference, with an “A” for Android, an “I” for iOS, a “W” for Windows Phone, and a “B” for BlackBerry. Which is better? CR won’t tell you. Which version of Android do you get on those smartphones? CR overlooks the fact that many owners of Android gear are saddled with OS versions over a year old, with no possibility of getting an over-the-air update, even to receive a critical security fix.
It’s also true that CR just adores larger screens. They are wowed by the “superb 4.7-inch” display on an HTC One X, without saying anything about the convenience of using such a large smartphone with one hand or taking it on the road with you. You’d think a publication that is so concerned about the failings of autos when it comes to ride, steering, handling, seat comfort, and the ease of use of dashboard controls, would also consider the possibility that a smartphone’s display might be too large, that the case might be unsuitable for convenient storage in a pocket or a purse. Or maybe they hired a bunch of clowns to perform customer evaluations, or require their testing staff to wear clown uniforms, so they can carry the larger stuff.
When it comes to tablets, the ratings might be expressed with ditto marks. Regardless of OS, every tested model gets a Very good rating for “Ease of Use,” whatever that’s supposed to signify. It doesn’t seem as if the OS has anything to do with it. In the end, among the larger tablets, the differences in scores from the highest rated third generation iPad to the worst rated Motorola Xoom are not significant. The iPad’s Retina display appears to be the only defining difference, and not one that seems significant to CR’s editors, despite their love of large displays on smartphones. Worse, CR seems oblivious to the well-known problems with touch responsiveness on Android gear.
But at least they aren’t complaining about the new iPad running too hot. They tried to get away with that scheme a few months back, but the test results were still acceptable. Of course, iPads ran a lot cooler when tested by other publications, but CR is not always about consistency or reliability.
To the CR reader, it would appear that all tablets rate very close to one another, and you might as well get the model that’s cheapest. Why bother to consider any other factor in deciding which one to buy? Well, the public has decided, despite CR’s unfortunate efforts to find equality where it doesn’t exist. Android tablets are still failures.
There’s also a brief survey entitled, “Faster laptops make a debut,” but CR included last year’s MacBook Pros among the ratings. I’ll cut them a break here, because the 2012 Mac note-books no doubt arrived too late to be included on the list. I could tell you that CR doesn’t understand the differences between OS X and Windows, but you knew that already.
And did I tell you that CR also reviewed chain restaurants in that issue?
Print This Post
The nasty little parasites at CR need to have all things equal – requiring the uninformed to use CR’s daft, inadequate ‘research’ to tell them what to buy.
It would be too easy if there were outright winners…
CR does not take advertising dollars but its goal is to prove that less expensive products can be better than more expensive ones. Apple is only the latest to suffer with this agenda. Nikon and Sony products were also given middling ratings in the past.
CR seems to have a real problem with Apple ever since they made unsubstantiated claims and were called on it by lots of people. I think they have doubled down because of this rather than back off and be more even handed. Really reduces my respect for CR because I wonder what other products they have done this with.
I object to this article’s misleading title. Not only does the author negate the title’s premise in his very first sentence, he goes on to provide no context whatsoever to support the idea that Consumer Reports is seeking “revenge” against Apple. Revenge for what? Even if you ultimately conclude they do not have such a motive, how do you justify suggesting so in the title, if you don’t plan to at lay out a rationale in the article?
This piece is a decent critique of CR’s seeming inability to fairly and accurately review consumer electronics. It’s a shame the author thought he had to saddle it with a click-baiting title that is in no way supported by the article itself. Trolling is not cool.
@Dan Berks, It is very clear Consumer Reports has an agenda against Apple. Consider their false claims about the original iPhone 4 being the only smartphone among the ones they tested that had the so-called Death Grip. That was a lie. Publishing reviews with demonstrably false ratings clearly indicates they have something to prove, they believe. Whether that’s an act of revenge because Apple succeeded with the iPhone 4 despite their complaint is certainly something to put on the table and discuss.
My problem is that most of the media gives CR a pass. I will continue to call them out on the faults with their testing of tech gear.
Peace,
Gene
Thanks for taking this on, glad to know that others view them with hesitation, now. I gave up on the rag after their antennae issue with the iPhone. I pretty much ignore them now and research products through various other methods. Oh, their entertaining enough to look over when I view my parents back issues (I cancelled my subscription), and the glossy paper feels nice and some pictures look kinda pretty. I don’t think they will ever address their foibles, hence the irrelevancy.
Consumer Reports started to its relevance a long time ago, particularly in the case of computers!!!
Its reviews are for the less than average mentally agile consumer!!!
@Viswakarma, I assume you mean, “irrelevance.”
In any case, that CR’s readers aren’t necessarily tech savvy doesn’t mean they have a right to publish false information about products.
Peace,
Gene
“Apple’s exclusive iMessages feature, which will come to the Mac when OS 10.8 comes out, ought to count as a major advantage for Apple”
Are you serious? CR should score products on features that aren’t available yet?!
I agree that CR does a pretty lousy job of rating high-tech things. They’re great on peanut butter and laundry detergent, and their automobile reliability surveys are tremendously valuable, but on computers they don’t do a very good job.
@David, OK, you need an Apple 101:
1. iMessages is an EXISTING feature of the iOS. The review I questioned was about mobile gear, not OS X. I simply said it was coming to OS 10.8.
2. Even without iMessages, the iOS offers an equal or superior SMS messaging experience compared to Android. End of story.
Peace,
Gene
As a tree-based product, CR needs many months of “lead time” before publishing articles. As a result, especially in the fast moving world of tech, the reviews found in CR are often hopelessly out of date. They often list products that are no longer available. And for some reason, they always include products from Sears and give them good ratings.
The concept of CR is great. In practice, however, they fall far short of their goals. And Gene, their disdain for all things Apple goes back far beyond the iPhone 4 lie. They would often put obsolete or EOLed Apple products against the latest Mactel offerings, making Apple look bad and often far worse than simply being a generation behind the others would indicate.
@Don108, Nothing stops CR from reviewing a product online, which they have done, and later placing it in the print version. They actually did that with the third generation iPad, after ginning up the nonexistent overheating controversy.
Peace,
Gene
When I was a student in the late 1960’s, CR reviewed the VW Beetle. It was the cheapest to run, repair, etc. But, they slammed it for noise and wind buffeting with the windows open (may be a bit wrong as this is memory). I wish I still had it as it was the cheapest to run car I have ever had and absolutely perfect for a university student. The starter failed and I could push it with my shoulder in the door and jump in to start it with the clutch. The battery was weak and I added sulphuric acid from the lab and it worked fine. Front suspension needed work, so I took off the spindles, bought new bushings and link pins for a few dollars, took it to a local machine shop and had a new front end for under $50. CR did not look at the different needs of people, I needed a cheap, reliable car, not a silent one.
I cancelled my subscription when they slammed the iPhone for “antenna gate”
I still like CR for info on autos and household appliances, but I’ve often thought they should have rated all versions of Windows since XP as outright unacceptable for security, the same as a car that tips over in testing.
P.S. @Russ Brown, I also have fond memories of single-handedly pushing my ’59 VW, jumping into the driver’s seat, and popping the clutch to overcome a dead battery.
@degrees_of_truth, CR is getting better with auto reviews. There are still a few question marks, but ratings for ride, handling, safety, steering and seat comfort are still of critical importance. I’m less concerned about their complaints about today’s complicated dashboard controls. That’s largely a first-look response, since you soon become accustomed to how those controls work. When the salesperson delivers the vehicle, they are usually expected to go over all of the features, and make sure you understand the fine details. Or at least the salespeople who care about the customer after you sign the contract and hand them a check.
Wait, the finances are handled in another department, where they usually try to see you some useless add-ons to fatten the profit margins, which are fairly thin these days.
Peace,
Gene
CR’s Review format is really only useful for comparing like items. Neither OS X nor iOS are like items to Windows or Android. So, that which makes Apple special is left out. But they can’t leave Apple out, so they compare Macs and iDevices with Wintel and Android with simple items like screen size.
Computers are just too complicated for a comparison matrix of 5 items and the price to be meaningful.
AND they are curiously blind to Apple’s dedication to build quality. Laundry soap yes. Computers, smart phones, tablets – clueless.