A couple of weeks ago we did a “Comment of the Week” blog posting about how the SEC has focused comments on the meaningfulness and reliability along with other issues in company developed metrics. You can find that post below; just scan down for the July 28 post.
Anyway, to help emphasize the importance of these metrics, and improving them as we go along, here is an example of how they need to be reviewed and improved. Twitter included this language as preliminary note in their 10-Q filed August 11, 2014. Check out the third and last paragraphs in particular.
NOTE REGARDING KEY METRICS
We review a number of metrics, including monthly active users, or MAUs, timeline views, timeline views per MAU and advertising revenue per timeline view, to evaluate our business, measure our performance, identify trends affecting our business, formulate business plans and make strategic decisions. See the section titled “Management’s Discussion and Analysis of Financial Condition and Results of Operations—Key Metrics” for a discussion of how we calculate MAUs, timeline views, timeline views per MAU and advertising revenue per timeline view.
The numbers of active users and timeline views presented in this Quarterly Report on Form 10-Q are based on internal company data. While these numbers are based on what we believe to be reasonable estimates for the applicable period of measurement, there are inherent challenges in measuring usage and user engagement across our large user base around the world. For example, there are a number of false or spam accounts in existence on our platform. We have performed an internal review of a sample of accounts and estimate that false or spam accounts represented less than 5% of our MAUs. In making this determination, we applied significant judgment, so our estimation of false or spam accounts may not accurately represent the actual number of such accounts, and the actual number of false or spam accounts could be higher than we have estimated. We are continually seeking to improve our ability to estimate the total number of spam accounts and eliminate them from the calculation of our active users. For example, we made an improvement in our spam detection capabilities in the second quarter of 2013 and suspended a large number of accounts. Spam accounts that we have identified are not included in the active user numbers presented in this Quarterly Report on Form 10-Q. We treat multiple accounts held by a single person or organization as multiple users for purposes of calculating our active users because we permit people and organizations to have more than one account. Additionally, some accounts used by organizations are used by many people within the organization. As such, the calculations of our active users may not accurately reflect the actual number of people or organizations using our platform.
Our metrics are also affected by third-party applications that automatically contact our servers for regular updates with no user action involved, and this activity can cause our system to count the users associated with such applications as active users on the day or days such contact occurs. Historically we tracked and reported in this section all users who accessed Twitter through third-party applications. We have reviewed and refined our processes, however, to calculate a new metric that is comprised of only such active users who have used applications with the capability to automatically contact our servers for regular updates where there was no discernable user action involved. In the three months ended June 30, 2014, approximately 11% of all active users solely used third-party applications to access Twitter. However, only up to approximately 8.5% of all active users used third party applications that may have automatically contacted our servers for regular updates without any discernable additional user-initiated action. The calculations of MAUs presented in this Quarterly Report on Form 10-Q may be affected as a result of automated activity.
In addition, our data regarding user geographic location for purposes of reporting the geographic location of our MAUs is based on the IP address associated with the account when a user initially registered the account on Twitter. The IP address may not always accurately reflect a user’s actual location at the time such user engaged with our platform.
We present and discuss timeline views in this Quarterly Report on Form 10-Q. We have estimated a small percentage of timeline views in the three months ended September 30, 2013 to account for certain timeline views that were logged incorrectly during the quarter as a result of a product update. We believe this estimate to be reasonable, but the actual numbers could differ from our estimate. Further, timeline views in 2012 exclude an immaterial number of timeline views for our mobile applications, certain of which were not fully tracked until June 2012. We present and discuss our total audience based on both internal metrics and data from Google Analytics, which measures unique visitors to our properties.
We regularly review and may adjust our processes for calculating our internal metrics to improve their accuracy. Our measures of user growth and user engagement may differ from estimates published by third parties or from similarly-titled metrics of our competitors due to differences in methodology.
As always, we would love to hear your comments!