QueuePilot: Reviving Small Buffers With a Learned AQM Policy
Internet router buffers help deal with congestion and traffic variability. Reducing buffer sizes may be one of the main outstanding open problems in networking, as small buffers would provide lower delays for users and free capacity for vendors.
Unfortunately, with small buffers, a passive policy has an excessive loss rate and existing AQM (active queue management) policies, which signal developing congestion, can be unreliable.
In this talk, we introduce QueuePilot, a reinforcement learning-based AQM that enables small buffers in backbone routers, trading off high utilization with low loss rate and short delay. QueuePilot automatically tunes the probability of marking packets to signal congestion.
After training once offline with a variety of settings, QueuePilot produces a single lightweight policy that can be applied online without further learning. We evaluate QueuePilot on real networks with hundreds of TCP flows, and show how its performance in small buffers exceeds that of existing algorithms, and even exceeds their performance with larger buffers.
* M.Sc. student under the supervision of Professor Isaac Keslassy.