Online deep learning methods for Intelligent Threat Mitigation

Project Reference :

AISG-100E-2019-033

Institution :

National University of Singapore (NUS)

Principal Investigator :

Professor Chang Ee-Chien

Technology Readiness :

4 (Technology validated in lab)

Technology Categories :

AI - Online Deep Learning

Background/Problem Statement

The global DDoS Protection and Mitigation market size is expected to grow from USD 3.3 billion in 2021 to USD 6.7 billion by 2026, at a Compound Annual Growth Rate (CAGR) of 15.1% from 2021 to 2026. The rise in multi-vector DDoS attacks and ease of availability of DDoS-for-hire services are major factors fueling the market and provide lucrative opportunities for DDoS protection and mitigation vendors.

Distributed Denial-of-Service (DDoS) attacks continue to pose a significant threat to our present-day Internet network security, denying legitimate users access to shared and essential resources. Adopting deep learning solutions to filter out application layer attack requests is challenging due to ever-changing profiles, lack of labeled data, and constraints in the online setting. Offline unsupervised learning methods can sidestep these hurdles by learning an anomaly detector from the normal-day traffic. However, anomaly detection does not exploit information acquired during attacks and its performance typically is not satisfactory.

Solution

Two online learning methods are proposed that utilise both the normal-day traffic as well as the traffic obtained during attacks, in addition to a machine learning optimization solution that aims to sift out the two types of traffic.

The first method is an LSTM-based training algorithm that mitigates DDoS attacks by contrasting estimated normal and attack network traffic conditional probability distributions and ranking the unidentified traffic. It extends an unsupervised anomaly detector to solve the problem and applies transfer learning to combine the results to obtain an online learner. This approach requires an exact likelihood calculation and there is no joint training of the anomaly detector and transfer learning models.

The second method is an enhanced iterative two-class classifier designed with a specific loss function more suited for deep learning and iterative learning to solve the optimization problem in the online setting.

The online learner achieves a 99.3% improvement on false-positive rates compared to baseline detection methods, on publicly available datasets. In the offline setting, the proposed solution is competitive with classifiers trained on labeled data.

(Filtering DDOS attacks from unlabelled network traffic data)

Benefits

  1. Online deep learning methods can accurately classify safe and malicious traffic that would be very difficult for a human
  2. Both online models accomplish marked improvements in terms of accuracy and false positive rate compared to existing models
  3. Both online methods outperform existing detection methods by a large margin
  4. Better network availability and lesser network downtime leading to better quality of service

Potential Application(s)

The growing attempt by organisations to deliver safety and security convergence, secure IIoT, intrusion/anomaly detection on the network, manage cyberphysical threats, manage behaviourial and organisational changes and ensure security throughout the supply chain will contribute to the increasing demand for DDoS protection and mitigation by Content delivery network (CDN) providers and DDoS protection and mitigation vendors.

We welcome interest from the industry for collaboration/ co-development / customisation of the technology into a new product or service. If you have any enquiries or are keen to collaborate, please contact us.