John Heidemann

Do You See Me Now? Sparsity in Passive Observations of Address Liveness (extended)

TitleDo You See Me Now? Sparsity in Passive Observations of Address Liveness (extended)
Publication TypeTechnical Report
Year of Publication2016
AuthorsJ. Mirkovic, G. Bartlett, J. Heidemann, H. Shi, and X. Deng
InstitutionUSC/Information Sciences Institute

Full allocation of IPv4 addresses has prompted interest in measuring address \emphliveness, first with active probing, and recently with the addition of passive observation. While prior work has shown dramatic increases in coverage, this paper explores \emphwhat factors affect contributions of passive observers to visibility. While all passive monitors are \emphsparse, seeing only a part of the Internet, we seek to understand how different types of sparsity impact observation quality: the \emphinterests of external hosts and the hosts within the observed network, the \emphtemporal limitations on the observation duration, and \emphcoverage challenges to observe all traffic for a given target or a given vantage point. We study sparsity with \emphinverted analysis, a new approach where we use passive monitors at four sites to infer what monitors would see at \emphall sites exchanging traffic with those four. We show that visibility provided by monitors is heavy-tailed–-interest sparsity means popular monitors see a great deal, while 99% see very little. We find that traffic is bipartite, with visibility much stronger between client-"NETWORKS" and server-"NETWORKS" than within each group. Finally, we find that popular monitors are robust to temporal and coverage sparsity, but they greatly reduce power of monitors that start with low visibility.