John Heidemann

Exploring Visible Internet Hosts through Census and Survey

TitleExploring Visible Internet Hosts through Census and Survey
Publication TypeTechnical Report
Year of Publication2007
AuthorsJ. Heidemann, Y. Pradkin, R. Govindan, C. Papadopoulos, and J. Bannister

Measurement studies published in the literature have, for the most part, ignored the population of hosts. While many hosts are hidden behind firewalls and in private address space, there is much to be learned from examining the population of visible Internet hosts–one can better understand network growth and accessibility and this understanding can help to assess vulnerabilities, deployment of new technologies, and improve network models. This paper is, to our knowledge, the first attempt to measure the population of visible Internet edge hosts. We measure hosts in two ways: via periodic Internet censuses, where we query all accessible Internet addresses every few months, and via surveys of a small fraction of the responsive address space, probing each address every 11 minutes for one week. These approaches are complementary: a census is effective at evaluating the Internet as a whole, while surveys validate the census and allow observation of the lifetime of typical address occupancy. We find that only 3.6% of allocated addresses are actually occupied by visible hosts, and that occupancy is unevenly distributed, with a quarter of responsive /24 subnets less than 5% full, and only 9% of subnets more than half full. We establish an upper-bound on the number of servers in the Internet at 36 million, about 16% of the responsive addresses. Many firewalls are visible and we observe significant diversity in the distribution of firewalled block size. While the absolute number of firewalled blocks appears stable, the ratio of coverage of visible firewalls to the number of visible addresses is declining, perhaps suggesting increasing use of invisible firewalls.