John Heidemann

The Need for End-to-End Evaluation of Cloud Availability

TitleThe Need for End-to-End Evaluation of Cloud Availability
Publication TypeConference Paper
Year of Publication2014
AuthorsZ. Hu, L. Zhu, C. Ardi, E. Katz-Bassett, H. V. Madhyastha, J. Heidemann, and M. Yu
Date Publishedmar
Conference LocationMarina del Rey, California, USA

People's computing lives are moving into the cloud, making understanding cloud availability increasingly critical. Prior studies of Internet outages have used ICMP-based pings and traceroutes. While these studies can detect network availability, we show that they can be inaccurate at estimating \emphcloud availability. Without care, ICMP probes can \emphunderestimate availability because ICMP is not as robust as application-level measurements such as HTTP. They can \emphoverestimate availability if they measure reachability of the cloud's edge, missing failures in the cloud's back-end. We develop methodologies sensitive to five ``nines'' of reliability, and then we compare ICMP and end-to-end measurements for both cloud VM and storage services. We show case studies where one fails and the other succeeds, and our results highlight the importance of application-level retries to reach high precision. When possible, we recommend end-to-end measurement with application-level protocols to evaluate the availability of cloud services.