So as we speak — more or less — Consumer Reports is rerunning its MacBook Pro battery tests. These are the ones that resulted in a non-recommended rating. That’s the first time such a thing happened to an Apple product, so it had to be extremely important. Potentially, it could hurt sales, and it sure brought lots of attention to CR, which garnered major headlines as a result.

Thus it came as no surprise when Apple marketing VP Philip Schiller responded — a rarity when it comes to a negative review — and explained that the company was working with the publication to see what was up.

On Tuesday, both Apple and CR posted updates on the controversial test results. Those posts got coverage in the tech media, but not so much in the mainstream press that made a huge deal of the original report, flawed as it was. What do I mean by flawed? Well, it seems that CR’s battery test consists of downloading 10 sites from a default server using a notebook’s default browser. This process is repeated until the battery dies.

But here’s the curious part: At the same time, CR disables caching, supposedly to have consistent results among all equipment. But this comes at the expense of failing to match a user’s actual use pattern, which would normally be to download some sites repeatedly, meaning they’d be cached. To shut off caching in Safari, CR needed to switch on the Develop menu in Advanced preferences. So not only was the use pattern not consistent with reality, but they also managed to trigger a rare bug in Apple’s Develop menu. The end result was that logos would be reloaded inconsistently, thus hurting battery life.

While it is Apple’s bug to fix, it is triggered by a test routine that doesn’t reflect real world use. Regardless, Apple fixed it — it’s in the macOS Sierra 10.12.3 public betas that will be made available to all Mac users soon — so the MacBook Pro should pass CR’s battery test.

Otherwise, this bug would probably have have gone unfixed and had no discernible impact.

Now the MacBook Pro has been an erratic performer when it comes to battery life. While most tests from tech publications show it is respectable, usually within the range of Apple’s 10-hour claim, some users are complaining of subpar results; sometimes it’s half what Apple claims, although it’s possible the recent 10.12.2 update may have helped some. But the fix mainly addressed a graphics bug, impacting the switch between integrated and discrete graphics, which would only affect the 15-inch model.

Still, Apple’s claims are thought to be controversial. Are they somehow stacking the deck in claiming 10-hour battery life and 30 hours idle? Did they devise some wacky test scheme that users wouldn’t normally use?

If you look at the fine print on Apple’s specs page for the MacBook Pro (if you can actually read anything that small), you’ll see what they did to come up with their battery life figures:

Testing conducted by Apple in October 2016 using preproduction 2.0GHz dual-core Intel Core i5-based 13-inch MacBook Pro systems with a 256GB SSD and 8GB of RAM (wireless web test, iTunes movie playback test, and standby test). Testing conducted by Apple in October 2016 using preproduction 2.9GHz dual-core Intel Core i5-based 13-inch MacBook Pro systems with a 512GB SSD and 8GB of RAM (wireless web test and iTunes movie playback test) and preproduction 2.9GHz dual-core Intel Core i5-based 13-inch MacBook Pro systems with a 256GB SSD and 8GB of RAM (standby test). The wireless web test measures battery life by wirelessly browsing 25 popular websites with display brightness set to 12 clicks from bottom or 75%. The iTunes movie playback test measures battery life by playing back HD 1080p content with display brightness set to 12 clicks from bottom or 75%. The standby test measures battery life by allowing a system, connected to a wireless network and signed in to an iCloud account, to enter standby mode with Safari and Mail applications launched and all system settings left at default. Battery life varies by use and configuration.

All right, let’s take a quick look. Some will quibble over using a display brightness of 75%. CR uses 100%, but I’ve never seen the need to run my Mac notebooks that high. Indeed, my aging 2010 17-inch MacBook Pro is set the 75% range, and since the new models have brighter displays, I accept Apple’s setting as realistic, and CR’s choice as somewhat on the extreme side. To each his/her own.

It would, however, be nice to know which 25 sites Apple selected, but that many should offer a representative sample of what a typical user might access during the course of a day. The iTunes movie playback rating is based on, obviously, playing back an iTunes movie. What else could it be?

You may have other ideas as to what should be tested to measure battery life more accurately. Maybe devise a canned routine using email access, word processing, and web access. Regardless, what Apple is doing isn’t out of the range of what a normal user might do. Compared to the CR methodology, it makes much more sense since it doesn’t require switching on an obscure setting.

None of this actually explains what is hurting the battery life on some MacBook Pros. Maybe the upcoming macOS Sierra update will address such problems too, since I don’t think the Develop menu bug is having much impact outside of CR. Maybe Apple has more work to do. Meantime, they have posted suggestions on what to do to maximize battery life on your equipment.

Macs tend to fare second best in Consumer Reports testing, partly because the magazine lives in ignorance of the differences between Apple’s computers and Windows boxes. But they’ve always been recommended, until recently. I can quibble about the way the tests appear to emphasize features over performance, usability and reliability. In fact, I have.

But it took a poor rating by CR to trigger a dialogue that revealed a serious flaw in their testing. The tests also triggered an obscure bug in Safari for macOS Sierra that might otherwise have remained undiscovered and unfixed.

It all started when CR reported wildly divergent battery life results, ranging from 3.75 hours up to 19 hours over three tests for each product. The latter is way more than Apple’s estimates, which range up to 10 hours.

Now all three MacBook Pro models exhibited similar behavior. A clue that something might be amiss was the fact that CR uses the default browser, in this case Safari. When the tests were rerun in Google Chrome, battery life was within acceptable limits.

Now Apple usually ignores test results from the media, but not CR, which has a circulation of millions of consumers and is highly influential when readers make buying decisions. A bad rating can kill or seriously hurt sales of some products. It can also accomplish good things, such as when an auto manufacturer has to go back and modify a faulty suspension system that might cause a rollover during a rapid maneuver to avoid an accident.

This time, Apple was in the hot seat. Even though a number of owners of the new MacBook Pros have reported an assortment of battery issues, CR’s results were unique. The inconsistency didn’t make sense, and thus marketing VP Philip Schiller posted a tweet — the new normal for getting the word out nowadays — saying that the results didn’t jibe with Apple’s own field tests. Apple was working with CR to figure out just what was going on.

Now CR’s tests are intended to be consistent from notebook to notebook. It involves downloading 10 sites from the company’s in-house server until the battery is spent. So just what was going on here, and was the test deliberately designed to leave Safari — and Macs — second best?

Well, that’s debatable, but to achieve consistent results, CR turns off caching on a browser. With caching on, the theory goes that the sites would be retrieved from the local cache, which presents an anomalous situation since different computers — and operating systems — might do it differently. On the other hand, it would also be using the computer normally, not in an artificial way. CR’s excuse, by the way, is that the test sequence puts greater stress on the battery: “This allows us to collect consistent results across the testing of many laptops, and it also puts batteries through a tougher workout.”

But how can such a test possibly produce results that in any way reflect what a typical user would encounter? After all, normal users might check a site several times a day, rather than constantly bring up new uncached sites. While all notebooks are being evaluated the same way, it’s a curious choice. Unfortunately, CR would have to go back and retest hundreds of computers to switch the testing scheme.

On Safari, caching is switched off via a seldom-used menu bar command, Develop, which is available in the apps preferences under the Advanced category. Clearly this is not a feature most users will ever use — or even know about. I use it to access the “Show Page Source” command from the context menu when I’m examining a site’s coding.

Now I suppose using a non-standard test scheme of this sort shouldn’t have had a disastrous effect, but it did. It seemed that the action triggered an obscure and inconsistent bug in Safari. With caching turned off, logos would reload, thus unnecessarily taxing the battery. It’s a bug that Apple discovered and fixed in the latest beta for macOS Sierra 10.12.3. You can download it if you’re a public beta tester or developer, and it will be made available for general distribution in a few weeks.

In the meantime, CR has accepted Apple’s findings: “According to Apple, this last part of our testing is what triggered a bug in the company’s Safari browser. Indeed, when we turned the caching function back on as part of the research we did after publishing our initial findings, the three MacBooks we’d originally tested had consistently high battery life results.”

It would have been nice if they said that before the review appeared, because that clearly indicated there was some sort of software issue that might be unnecessarily impacting the tests in a way that customers wouldn’t encounter. In other words, it’s an admission the test was unfair, and that the results didn’t in any way reflect a normal use case. After all, CR is testing a notebook’s battery life, not the capabilities of the default browser to render pages without caching.

In any case, CR is retesting the MacBook Pros with the revised macOS, and it shouldn’t take more than a few days to deliver the results. Assuming battery life is normal, the rating will be changed accordingly, and the new notebooks will be added to the recommended list.

Of course, CR should have realized something was amiss as soon as the battery life normalized with caching on. They could have reached out to Apple before the results were published for clarification. As it was, CR got a boatload of publicity for its decision not to recommend the MacBook Pros. Of course, that result will soon be changed if all goes well.

Will CR learn a lesson from this debacle? Probably not. After all, few companies would dare protest a bad rating. Indeed most companies who build products that don’t past muster probably deserve it.

From the very first day when Steve Jobs unveiled the iPhone at a Macworld Expo keynote in 2007, the critics were vocal. Apple had no business whatever building a cell phone. I suppose they also felt the same when Apple released the iPod. At the time, it was regarded by many as an overpriced indulgence.

Funny how that turned out.

That feeling might have resulted from the fact that Apple once supported Motorola in building iTunes-compatible feature phones, such as the infamous ROKR. But it’s not that Motorola made bad phones in those days. All three members of the Steinberg family used Motorola RAZRs for several years before iPhones showed up. It was a pretty good phone for its time if you just wanted a fairly basic cell phone with a rudimentary contact list.

In any case, Steve Jobs denigrated cell phones after the ROKR tanked, which should have given a clue.

When the iPhone arrived, Microsoft’s then-CEO, Steve Ballmer, quickly put it down: “There’s no chance that the iPhone is going to get any significant market share. No chance. It’s a $500 subsidized item. They may make a lot of money. But if you actually take a look at the 1.3 billion phones that get sold, I’d prefer to have our software in 60% or 70% or 80% of them, than I would to have 2% or 3%, which is what Apple might get.”

Now this wasn’t the first time Ballmer was totally wrong about something. But even after he left Microsoft and became the owner of the Los Angeles Clippers, his mistakes have continued to haunt Microsoft. To this day, the company has been unable to get any traction out of mobile handsets. Even buying Nokia’s handset division failed; the latest Nokia actually runs Android.

After Google altered its Android roadmap from a BlackBerry wannabe to an iPhone wannabe, the critics said it was Apple versus Microsoft all over again. Well, the version from the 1980s and 1990s, in which Apple fared second best for a variety of reasons having nothing to do with product quality. But Microsoft has earned hundreds of billions of dollars from Windows over the years regardless of what you think about Windows.

Even though there are far more Android handsets out there than iPhones, most are cheap. Apple dominates in the high-end smartphone marketplace and earns over 90% of the profits. What’s more, Google earns very little from the platform. Android is given away free to licensees, and the profits, such as they are, come as a result of people clicking on targeted ads in the various Google apps that are installed on those handsets. Yes, there is revenue from paid apps in the Google Play store, but it’s still mostly about the ads.

Yes, I suppose Google takes in a decent profit from the sale of Pixel smartphones, at the expense of other mobile handset makers. But it’s a fraction of what Apple earns from iPhone sales.

Today, the iPhone is, to some Apple skeptics, on its last legs. It’s all about the fact that, for the first time in years, Apple reported reduced earnings. Fewer iPhones were sold, but the same was true for iPads and Macs. And iPods too. Yes, Apple still sells a small number of iPods, though the total sales are buried in the same “Other” category as the Apple Watch.

The company is, however, hugely profitable, and its guidance for the December 2016 quarter called for a slight revenue increase. The NPD Group reports good sales for the iPhone and Macs, so the news when Apple releases its financials at the end of the month may be surprisingly positive.

Oh yes, there’s that published report that Apple recently reduced orders from its suppliers for the iPhone. This is an old story, that iPhone sales must be bad if Apple builds fewer units when going into a new year, but that contention merely defies logic. It’s normal for sales in a March quarter to be far less than the December — holiday — quarter. But many members of the media report such stories uncritically without context.

They also forget what Tim Cook has said on several occasions, that you cannot take a few supply chain metrics and come to any valid conclusions about demand and sales for Apple products.

Apple is, surprisingly, making a huge deal of the iPhone’s 10th anniversary. But it stands to reason, since it’s the company’s most successful product and is responsible for all the huge gains Apple made in recent years. The iPhone is also a cultural icon, and even when people buy other smartphones, they are often viewed in the context of the iPhone.

This all implies that there will be some sort of special 10th anniversary model come this fall. As has been reported repeatedly, the next iPhone — an iPhone 7s or iPhone 8 — may arrive, in at least one configuration, with a glass-based case and a wrap-around OLED display. There may even be some sort of wireless charging scheme. Whether the entire lineup takes this direction, or it’s just a single premium model, is anyone’s guess outside of Apple and its suppliers.

What is clear is that the iPhone has been a far more successful product than Apple’s competitors — and probably Apple itself — expected. So maybe it isn’t hype when Tim Cook promises, “The best is yet to come.”

At an age where some might become a little forgetful, I remember it well: Steve Jobs taking the stage and announcing what ended up being the successor to the iPod, which offered your music library in your pocket. The iPhone also offered the Internet in your pocket; essentially a tiny personal computer masquerading as a cell phone with a touchscreen.

Sure other devices that came to be called smartphones offered browsers. But the iPhone delivered nearly the full experience, except for Adobe Flash. That was supposed to be a huge negative, but it turned out not to be. With HTML5 and creating responsive sites — web pages that were optimized for both desktop and mobile browsers — you weren’t giving up much. Even email was reminiscent of the Mac experience.

And no wonder. The iPhone OS was built on a slimmed down version of OS X, by, at first, the same development team. That made it quite unlike other mobile operating systems. Instead of managing a clumsy physical keyboard — or the standard telephone’s numeric keyboard — you had a touchscreen with the full monty.

Continue Reading…