While improving "Customer Experience" is clearly top of mind for mobile operators and optimization solution vendors, the methods for measuring how users interact with mobile services and how customers assess and "feel" about that relationship are complex and multi-layered. As optimization solutions become more integral to the delivery of mobile broadband services, defining the impact optimization has on the perceived experience of users becomes all the more important.
Value propositions for vendor solutions that optimize mobile services typically fall into three categories:
- Operational efficiency and savings – by optimizing application, data, voice or video content these solutions reduce the need for investments in mobile core or radio access networks, thus reducing CapEx and OpEx
- New Service Creation – by providing more control and granular visibility, optimization solutions can enable new services that operators can sell or differentiate their offerings
- Improved Customer Experience – by making networks more efficient and avoiding congestion and poor application performance, optimization solutions can improve how customers perceive network performance, thereby reducing churn and increasing customer satisfaction with their operator
The following diagram illustrates one potential model for understanding the relationships among customers, users, applications, service providers and mobile networks as they pertain to measuring and assessing quality.
It is evident that technical measures of network Quality of Service (QoS) represent only one of several inputs to the overall "customer experience" equation. Optimization solutions enhance and improve the traditional QoS performance of existing network deployments. These solutions affect the performance of mobile applications by, among other things, improving throughput, lowering latency and jitter, reducing dropped packets, reducing connection times, increasing coverage, and improving data call success rates.
The user experiences these benefits through the lens of applications they are using on their smartphone, tablet or other mobile device. The measures of Quality of User Experience (QoUE) vary widely by application and device type and tend to be subjective. The most formalized examples of QoUE measures are based on Mean Opinion Score (MOS) tests from the voice and video world. MOS attempts to quantify the user's perception of the quality of a voice call or a video. QoUE can also be broadened to other human factor metrics about the applications themselves (e.g., HTML page abandonment, bad links, shopping cart abandonment, slow search results, etc.) which are beyond the control or influence of the mobile network itself.
One important distinction is that just improving the quality of the user viewing experience and application performance does not guarantee the overall customer experience will improve in all cases. In the above model, the customer is separate from the user. In the case of a consumer these might be the same entity (i.e., you!), but it could also be an enterprise purchasing services from a mobile operator on behalf of its employees. The metrics for Quality of Customer Experience (QoCE) are much different than QoS or QoUE. QoCE focuses on the business level interactions with the service provider (e.g., service mix, pricing, billing, and support) and is measured through metrics such as churn rate, ARPU, and call rates into service centers.
So given the complex and indirect relationship between optimization solutions and a holistic customer experience assessment, what are some of the best ways to measure and assess the value of optimization solutions from a QoUE perspective? First, these solutions should continue to measure traditional QoS metrics (throughput, latency, etc.), but also expand to include metrics that focus on measuring congestion and the avoidance of congestion at critical points in the network. Congestion is the most direct cause of poor network and application behavior – and by association QoUE. Optimization solutions should not only improve network performance but also provide a multi-layered measurement of congestion onset, congestion duration, and congestion impact and then report important metrics such as:
- Minutes per day above a congestion threshold
- Congested minutes avoided by using optimization solutions
- % congestion avoidance
What really matters from a customer experience perspective is to more closely tie optimization effects to QoUE improvement. Sycamore's IQstream solution provides a strong set of optimization capabilities tied with specific KPIs focused on measuring and reporting congestion avoidance in the performance- and cost-sensitive access portion of the mobile network. Armed with these KPIs, mobile operators can directly link the value of content optimization with improvements in their subscribers' QoUE.
For more information about this topic, please see Kalevi Kilkki's "Quality of Experience in Communications Ecosystem" article.