As most regular readers know, I’m not enamored of Consumer Reports and the way the magazine handles personal technology. Their efforts to simplify and generalize possibly complex information ends up helping nobody. While they no doubt accurately represent reader surveys about product and service reliability, the actual reviews are tragically flawed.
I’m also concerned why so many members of the media accept CR at face value, and don’t question the clear and present flaws in their test methodology. It can’t just be me!
Consider the February 2011 issue, which was just dropped into my mailbox a short while ago. One of the feature articles is entitled: “Tablet computers,” and the subtitle pretty much sums up the conclusions: “Apple and Samsung offer the most for your money.”
The tested products include, of course, the iPad, and the Samsung Galaxy Tab, along with a smattering of lesser products from Huawei, Archos, Coby and Maylong. The later, with a 7-inch screen, retails for a mere $100, and that may be too much, considering that CR concludes that its “Touch screen has poor response. No access to a standard app market.”
Unfortunately, the key areas where you might compare tablets of different sizes and configurations are omitted. Almost as bad, CR tested the iPad with the older 3.2 OS, even though version 4.2, with multitasking and loads of other features, shipped in November. Then again, the differences between the iOS and Android, which powers all but one of the remaining tablets in this lineup (the Archos 9 PCtablet uses Windows 7), are never explained. The closest you get are some generalities about the app markets. iOS and Android are good, but “Third-tier app stores we visited were simply not up to par.”
Maybe so, but CR has an extremely low level of acceptance, failing to point out the obvious differences between the App Store and Android Market, such as far more substantial apps, including games, in the former’s repertoire. Worse, nothing is said about the fact that Google has never certified current Android OS versions for use in tablets. What this means is that the apps are designed for smartphone screens, not even the 7-inch displays tested by CR. As a result, the images, particularly text, might be pixellated when expanded to fill a larger screen.
Then again, the difference between the smartphone display and a 7-inch tablet isn’t such a stretch. Going to 9.7 inches on the iPad required compromises for existing iOS apps, which open in a smaller window, but can optionally zoom to fit the larger borders, with the expected sacrifices in quality. But with tens of thousands if apps that now support the iPad, you’re more likely to find what you need without compromise.
Alas CR, which has managed to stick feet in the mouths of its tech crew with that lame review of the perceived flaws of the iPhone 4’s antenna, doesn’t attempt to examine the usability of a 7-inch versus 9.7-inch tablet. Steve Jobs had made a large issue of this perceived disparity, pointing out that the 7-inch screen gives you only 45% of the real estate of the iPad’s display, which he believes is too small. CR didn’t test this contention, although they cover the difference in passing.
So the iPad “is a tad clunky to carry around,” and I would agree with that statement. The 7-inch Galaxy Tab “is more portable, even if the smaller screen won’t wow you the way the iPad screen will.”
Well, I suppose that sort of covers some of the differences, but the actual experience of consuming content and performing some basic productivity chores isn’t evaluated. The issue of usability is seldom on CR’s radar when it comes to personal computers and related products. They seem too focused on the hardware differences, real or imagined, and the most basic functionality, emphasizing the usefulness of the touch screen on tablets. Beyond that, they have nothing to say.
But that’s been true about CR’s personal computer reviews, where they compare Macs with under-equipped Windows PCs, which simply makes the Mac seem a lot more expensive. Windows? Mac OS X? To CR, they are two flavors of the same ice cream, with only minor differences, or at least that’s the impression the lack of a meaningful comparison conveys.
I suppose if there’s anything you can take away from CR’s mediocre coverage of tablets is that they don’t recommend paying extra for 3G capability, because they believe Wi-Fi access is ubiquitous enough for most users. I don’t disagree, but that’s just me. It’s also clear that price does impact quality. For the most part, the cheapest tablets were seriously downgraded for one reason or another. However, the Archos Windows 7 model, listing for $430, got dinged for poor app access and a subpar touch screen that isn’t “very responsive.” That, however, is likely as much the fault of Windows 7 being nudged into a direction for which it wasn’t designed, as the design of the touch screen itself. But CR doesn’t consider those possibilities.
The review will, naturally, be outdated when the next iPad debuts, probably in a month or two. But I don’t expect CR’s coverage of the next model, and its competitors, to be any better.