A First Look at Commercial 5G Performance on Smartphones


Arvind Narayanan, Eman Ramadan, Jason Carpenter, Qingxu Liu, Yu Liu, Feng Qian, Zhi-Li Zhang

Department of Computer Science & Engineering
University of Minnesota

Abstract


We conduct to our knowledge a first measurement study of commercial 5G performance on smartphones by closely examining 5G networks of three carriers (two mmWave carriers, one mid-band 5G carrier) in three U.S. cities. We conduct extensive field tests on 5G performance in diverse urban environments. We systematically analyze the handoff mechanisms in 5G and their impact on network performance, and explore the feasibility of using location and possibly other environmental information to predict the network performance. We also study the app performance (web browsing, HTTP download, and volumetric video streaming) over 5G. Our study consumes more than 15 TB of data. Conducted when 5G just made its debut, it provides a “baseline” for studying how 5G performance evolves, and identifies key research directions on improving 5G users’ experience in a cross-layer manner.

5G Panel Tower

Experiments


We conduct on-field experiments to evaluate 5G performance in 3 cities in the United States: Minneapolis, MN; Chicago, IL; Atlanta, GA.

 

Arvind Narayanan conducting 5G throughput measurement tests in Minneapolis, Minnesota

TCP Throughput Test
Minneapolis, MN

Eman Ramadan conducting a walking loop test comparing 4G and 5G performance near US Bank Stadium in Minneapolis, Minnesota

Walking Loop
Minneapolis, MN

Mycroft Liu conducting a throughput testing of all 3 5G carriers in the US at Atlanta, Georgia

Testing Three 5G Carriers
Atlanta, GA

Jason Carpentar conducting obstruction tests of 5G in a hotel in Chicago, Illinois

Obstruction Test
Chicago, IL

Paper | Talk | Slides


Paper PDF

Technical Talk

Presentation Slides


Dataset (5Gophers-v1.0)


We release the data collected by us in this paper. This data is important in providing a first impression of the world's very first commercial 5G rollouts, and serves as an important baseline of 5G performance. We conduct several experiments to evaluate 5G performance, including but not limited to throughput performance, latency measurements, impact of mobility and obstructions, handoff analysis among many others. Our experiments also illustrate the pros and cons of the different 5G technologies.


Comparing 5G and 4G performance (TCP concurrency)
5G signals getting blocked due to human body. Another example shows 5G establishing reflective path due to obstruction.
Walking loop tests



Need more data? Check out Lumos5G dataset or email us at fivegophers@umn.edu

Supporters


This project was supported in part by NSF under Grants CNS-1917424, CNS-1903880, CNS-1915122, CNS-1618339, CNS-1617729, CNS-1814322, CNS-1831140, CNS-1836772, and CNS-1901103.