Well, my friends, the folks at Consumer Reports are at it again, making pronouncements born of ignorance, or inadequate testing methods. In the September 2011 issue, CR is focusing on such topics as cars, “top coffees” and tablets and e-readers.
While, the iPad 2 rates a tad better than the competition in the CR feature — and this fact will be touted to the skies by Mac sites — the test methodology remains highly flawed as usual. CR concentrates on the raw hardware features and performance, which actually puts the top-tier tablets on a fairly similar footing. But where it counts, such as the iOS versus Android, Web OS, and so forth and so on, CR hasn’t a clue. “Consider software” is the last criterion in choosing a tablet, where common sense should place it pretty much near the top, just below the actual performance and reliability of the product.
CR also offers precious little information about software. You know that “Apple has provided upgrades for the iPad’s iOS.” They correctly point out that upgrades for Android aren’t always available. But app selection is pretty much ignored beyond the presence of an “Approved app market.” How the App Store differs from other app repositories isn’t mentioned, nor is the number and quality of the apps themselves. So you are never informed that there are over 100,000 iPad apps, but only a tiny number for the Android OS and other tablet-based systems. What’s the good of having a tablet with a pretty interface and snappy performance if the app selection is minuscule?
Maybe CR expects you to just look at them.
Well, maybe not. In a section entitled “12 apps that make the most of a tablet,” only two are available for the Android OS. CR’s excuse, “As noted, many are available only for the iPad OS, at least for now.” Clearly they assume that Android will ultimately catch up. Curiously, the iPad’s rich selection of games is barely touched upon. Just two, specifically “Pocket Frogs,” a free game, and “Infinity Blade, available for $6, are supposedly the best of the breed. Pathetic.
CR also fails to pay attention to the marketplace, where those other tablets have, in large part, been total failures. One of the most widely promoted, the Motorola Xoom, moved just 440,000 units in the same quarter in which the iPad sold 9.25 million. Clearly the customers know what’s best for them, even if CR cannot understand the ingredients of a good tablet computer.
But none of this should come as a surprise. CR has also failed big time to figure out what makes a top flight personal computer. They rate products in arbitrary categories with little regard as to how the raw specs beyond display size relate to one another. So you are left with the impression that a Mac is just a pretty computer with a fancy price. The distinctions between OS X and Windows are utterly lost on them; they never truly compare the two in terms of usability and security.
When it comes to autos, CR has always been the standard barer. You take them seriously, particularly when they claim that a vehicle has a tendency to tip over when making an abrupt maneuver to avoid a collision.
Well, this time, CR has aimed its arrows at the venerable Honda Civic, for years one of the best rated and best selling compact cars in the U.S. But the redesigned 2012 model isn’t getting the love from CR. The rating, compared to the previous model, dropped from 78 to 61, and is thus classified as not recommended, even though the Civic is expected to be a very reliable car.
The major criticism is that it isn’t as agile as its predecessor, has a choppy ride and long stopping distances. The last may be the unkindest cut of all, because it means that brakes may not stop you in time to avoid a serious accident.
Honda has hit back, claiming they disagree, and you have to expect that. However, you wonder why such glaring defects eluded the commercial auto magazines, none of whom report serious problems with ride, handling and braking. Is it because they take ads from Honda and other car companies, and are thus more inclined to give a marginal car a pass? CR claims not to be influenced by manufacturers because they do not accept advertising and buy the gear they test direct from regular dealers.
Now speaking as someone who has owned two Honda Accords — the mid-sized step up from the Civic — I think I have a fair sense as to the company’s design objectives. I also spent about 15 minutes test driving a 2012 Civic, one of the high-end EX models, and found the ride and handling to be mostly typical of other Honda vehicles, meaning firm, comfortable, with reasonably responsive steering and braking. No, I didn’t have the opportunity to test the car on rough roads or attempt to stop on wet pavement, where the Civic supposedly suffers.
Typical of most of the published reviews, Car and Driver spoke of the Civic’s “smoother ride,” which is the polar opposite of choppy. Among the notable criticisms has it that Honda excised some of the character out of the vehicle in terms of delivering somewhat numb steering (I didn’t drive the car long enough to notice one way or the other). The increased use of hard plastics in the interior was another negative, particularly at a time when the competition, particularly the Ford Focus and Hyundai Elantra, have gone upscale in terms of looks, ride and interior amenities.
Sure, one is entitled to their opinion, but the differences between what CR says about the Civic, and pretty much all the major car review outlets, are serious enough that you have to wonder if the former reviewed the same car as the latter. As I said, the ride didn’t seem at all choppy to me.
But as long as CR is taken seriously by the media, and they aren’t asked to account for the discrepancies in their reviews, nothing’s going to change.