Improving the Broadcast Viewer Experience Through Rapid and Early Elephant Flow Detection by Machine Learning

Abstract

Keeping broadcast IP network latency low is critical in maintaining the immersive viewing experience, especially when delivering high quality broadcast media over the Internet or broadcast IP datacentres. The network and resource requirements of heavy-hitting broadcast media flows with high data rates and temporal longevity clash with the needs of latency sensitive short data flows, leading to switch buffer overload and network congestion resulting in dropped packets and increased latency due to TCP-RTOs (Transmission Control Protocol Retransmission Time Out). Within broadcast datacentres the media flows often fall under elephant flow (EF) classification, with the short flows being classified as mice flows (MF). Rapid and early detection of EFs will allow the SDN controller to re-route them and reduce their impact on the MFs within the broadcast IP network. This reduces packet dropout so that the TCPRTOs are not triggered resulting in latency being kept low and the immersive viewing experience being improved. Although EF detection has been researched extensively, this paper proposes a new approach to detect EFs for the broadcast network SDN controller within 500ms, thus allowing the SDN controller to re-route the EFs and reduce packet loss. This method uses machine learning with ensemble LSTM (Long Short-Term Memory) neural networks, with each LSTM being a different length so the ensemble can capture the non-linear characteristics of the varying flow sizes. The ensemble LSTM outputs are then concatenated and further processed by a neural network. Training is achieved by back propagating through the neural network and then each LSTM resulting in a greater inference EF detection accuracy for the broadcast IP network. Our approach was tested on industry standard datasets and achieved EF detection in under 500ms without needing to be reliant on statistical information provided by network switches thus further reducing latency and improving the immersive viewing experience, unlike other approaches.