A framework to evaluate performance from an application and user perspective

Abstract

Experience related performance measures are important for operational, commercial as well as usability reasons. Quality of Experience (QoE) is widely discussed as a measure of user perceived performance, no universal definition for QoE are available. Instead of moulding QoE into a one-fits-all parameter, this work identifies and defines four specific performance measures that are more descriptive than traditional QoE. They account for objective and subjective measures and focus on applications and tasks specific performance. The parameters include Quality of (an) Application (QoA) and Quality of Use (QoU). They are further divided into objective and subjective measures, i.e. objective QoA, subjective QoA, objective QoU and subjective QoU. Subjective QoA reassembles QoE widely discussed in technical literature; QoU provides a measure for quality of experience in a more general, non technical sense. It is proposed that these individual performance parameters provide a better picture of performance and allow stakeholders to make more specific decisions. The paper also introduces a set of sample performance evaluations for the Remote Access Laboratories application to demonstrate how this framework can be applied and how such parameters can be measured.