Can you use ActiveSync to measure market shares?
Computerworld.com published an article this week called “iPhone, iPad dwarf mobile rivals in small- and mid-sized firms”. It stated:
Apple has won the hearts and minds of small- and mid-sized businesses, which have overwhelmingly adopted Cupertino's mobile devices over rivals powered by Android or Windows, an Exchange hosting vendor said last week.
Well, my interest was peaked. Mobile devices and Exchange, that is a combination for which I have a soft spot! However reading the article and statements made, I found myself particular critical on their methods and conclusions made from the data. I decide to share those views with you, as some of you might use the same techniques in order to report device usage for BYOD policies for instance. It might provide some useful insight in the subject.
The original article used information supplied by Intermedia, an Exchange (2010) Hoster among other services, as a source (unfortunately I couldn’t find the information on their site). They state that most of their customers use mobile Apple devices (76%), with Android as a second (~18%, when added the shares from Samsung and Motorola) and Windows Phones and tablets as 1%, same as Blackberry (not stated whether this is exclusively BB OS10, as previous versions didn't provide ActiveSync). It is not stated what the remaining 4% is. Note that they have data from a longer period (36 months is mentioned), but these percentages only represent a year (from October 2012 till this years October). However, it certain that all devices are ActiveSync capable, because they tallied the activations. They’ve excluded mobile PC’s (notebooks) as they “access the hosted Exchange servers directly, without needing ActiveSync.”
So, is ActiveSync a correct way to count types of mobile devices? Well, I’ve done it myself and the protocol certainly can provide information about the device OS and even device model, but there are some issues with this approach. Not every user wants to connect their device via ActiveSync and there are alternative ways to connect to Exchange (Exchange Web Services, OWA, OWA Mini and less functional IMAP and POP). To be realistic, these alternatives aren’t widely known, certainly not as known as ActiveSync. Simply put, if a user has an mobile device it’s probably using ActiveSync to connect to company mail, it is still however an assumption. Using this method means that you can only say something definitive about active connected devices. Those that are not connected, but still bought and used possibly with other protocols than ActiveSync, are not counted.
Obviously one alternative to ActiveSync is Outlook (with direct MAPI and/or Outlook Anywhere). This is the reason why they excluded notebooks. But they are making statements about Microsoft Surface tablets:
“During the last three months of 2012 and the first ten of 2013, Intermedia counted just 1,700 Surface activations of all types, or less than half the number of iPads for October alone. Surface RT tablets, the lower-priced of the two lines, dominated, accounting for 62% of all Surface activations.”
The Surface Pro was able to run Outlook from the start and has a trail edition pre-installed. When used in a business context it’s possible it’s fully licensed. Although it’s possible, I don’t think a lot of users would use Outlook concurrently with ActiveSync (via the considerable less capable Mail app) on their Surface Pro.
In any case, determining Surface Pro and Pro 2 usage via ActiveSync will probably count less devices than actually present within a business. Since Windows 8.1 and the addition of Outlook for Surface RT/2 this will also be the case for those tablets. Unfortunately, there is no mention of this in the article. And on a side note, it might be very possible that users also preferred OWA than the Mail App before Outlook was available which could also undercount Surface (RT) usage.
Back to how they measured:
Intermedia measured mobile device preferences by tallying activations of ActiveSync…
Unfortunately there is no more information than this. I assume that they mean with activations new created ActiveSync partnerships. The question then is, is this a correct way to determine how popular certain devices are? I know that sometimes in order to troubleshoot, partnerships are removed and a new connection is made. Devices can break and be replaced. Some devices can hold multiple ActiveSync accounts, while some only one (I think iOS was one of the first to be able to have multiple accounts, and as ActiveSync doesn’t support Shared Mailbox/Calendars…). And not uncommon to businesses, devices are wiped and reused by other employees which make a new partnership to their own mailbox.
With this method they would count this device twice or perhaps more, potentially skewing the result towards a particular popular device range with a relative long usability or need-to-have factor (iOS perhaps?). There is no mention that they compensate for these instances, so I assume they do not. Personally, I would periodically check on the last sync time of partnerships of still active accounts and check what kind of device that partnership holds. That would eliminate most caveats I mentioned above. If I would be very thorough, I would even check for duplicate DeviceID’s within those active and previous partnerships.
Another point of discussion is that there is no regional information supplied. Intermedia supplies services to every country, but it’s very possible that they have more customers in one region than in the other and thus skewing the results towards the habits of one region. Unfortunately, they do not provide such information. It is however a fact that market shares of tablets, phone and other devices can vary greatly depending on the region for whatever reason as research by Kantar shows. Because of this I would argue that the statistics are only valid for the customers of Intermedia and probably cannot be applied to a global market.
Can they make definitive statements on device preference within small- and mid-sized business? Perhaps. They might not be too far from the truth and it is probably a relatively fair way to get an indication on device popularity. The keyword for me is indication; the information is mostly valid for Intermedia’s environment and the percentages only apply on connected devices that use ActiveSync.
It is my opinion that this article and it’s author (not necessarily Intermedia) are stretching the usability of the accumulated data too far and the article also lacks skepticism and criticism. I certainly wouldn’t extrapolate this research beyond these confinements without more information. I would argue that the statement quoted at the beginning might be true, but I would need more information to fully support it.
Unfortunately other sites (Surface stort in diep ravijn, Dutch) have copied the information and dropped even more nuances. Well, I hope I have restored some justice and order in the universe with this post. If you have any remarks or questions, feel free to contact me or add a comment below.