A First Look at Commercial 5G Performance on Smartphones


Arvind Narayanan, Eman Ramadan, Jason Carpenter, Qingxu Liu, Yu Liu, Feng Qian, Zhi-Li Zhang

Department of Computer Science & Engineering
University of Minnesota

Abstract


We conduct to our knowledge a first measurement study of commercial 5G performance on smartphones by closely examining 5G networks of three carriers (two mmWave carriers, one mid-band 5G carrier) in three U.S. cities. We conduct extensive field tests on 5G performance in diverse urban environments. We systematically analyze the handoff mechanisms in 5G and their impact on network performance, and explore the feasibility of using location and possibly other environmental information to predict the network performance. We also study the app performance (web browsing, HTTP download, and volumetric video streaming) over 5G. Our study consumes more than 15 TB of data. Conducted when 5G just made its debut, it provides a “baseline” for studying how 5G performance evolves, and identifies key research directions on improving 5G users’ experience in a cross-layer manner.

5G Panel Tower

Experiments


We conduct on-field experiments to evaluate 5G performance in 3 cities in the United States: Minneapolis, MN; Chicago, IL; Atlanta, GA.

 

Arvind Narayanan conducting 5G throughput measurement tests in Minneapolis, Minnesota

TCP throughput test
Minneapolis, MN

Eman Ramadan conducting a walking loop test comparing 4G and 5G performance near US Bank Stadium in Minneapolis, Minnesota

Walking Loop
Minneapolis, MN

Mycroft Liu conducting a throughput testing of all 3 5G carriers in the US at Atlanta, Georgia

Testing Three 5G Carriers
Atlanta, GA

Jason Carpentar conducting obstruction tests of 5G in a hotel in Chicago, Illinois

Obstruction test
Chicago, IL

Paper | Talk | Slides


Paper PDF

Talk

Presentation Slides

BibTeX Citation

@inproceedings{10.1145/3366423.3380169,
  author = {Narayanan, Arvind and Ramadan, Eman and Carpenter, Jason and Liu, Qingxu and Liu, Yu and Qian, Feng and Zhang, Zhi-Li},
  title = {A First Look at Commercial 5G Performance on Smartphones},
  year = {2020},
  isbn = {9781450370233},
  publisher = {Association for Computing Machinery},
  address = {New York, NY, USA},
  url = {https://doi.org/10.1145/3366423.3380169},
  doi = {10.1145/3366423.3380169},
  booktitle = {Proceedings of The Web Conference 2020},
  pages = {894–905},
  numpages = {12},
  location = {Taipei, Taiwan},
  series = {WWW ’20}
}

Dataset


We release the data collected by us in this paper. This data is important in providing a first impression of the world's very first commercial 5G rollouts, and serves as an important baseline of 5G performance. We conduct several experiments to evaluate 5G performance, including but not limited to throughput performance, latency measurements, impact of mobility and obstructions, handoff analysis among many others. Our experiments also illustrate the pros and cons of the different 5G technologies.


Supporters


This project was supported in part by NSF under Grants CNS-1917424, CNS-1903880, CNS-1915122, CNS-1618339, CNS-1617729, CNS-1814322, CNS-1831140, CNS-1836772, and CNS-1901103.