Skip to Main content Skip to Navigation
New interface
Journal articles

A robust control-theory-based exploration strategy in deep reinforcement learning for virtual network embedding

Ghina Dandachi 1, * Sophie Cerf 2 Yassine Hadjadj-Aoul 1 Abdelkader Outtagarts 3 Eric Rutten 4 
* Corresponding author
1 ERMINE - mEasuRing and ManagIng Network operation and Economic
2 SPIRALS - Self-adaptation for distributed services and large software systems
Inria Lille - Nord Europe, CRIStAL - Centre de Recherche en Informatique, Signal et Automatique de Lille - UMR 9189
4 CTRL-A - Control for Autonomic computing systems
Inria Grenoble - Rhône-Alpes, LIG - Laboratoire d'Informatique de Grenoble
Abstract : Network slice management and, more generally, resource orchestration should be fully automated in 6G networks, as envisioned by the ETSI ENI. In this context, artificial intelligence (AI) and context-aware policies are certainly major options to move in this direction and to adapt service delivery to changing user needs, environmental conditions and business objectives. In this paper, we step towards this objective by addressing the problem of optimal placement of dynamic virtual networks through a self-adaptive learning-based strategy. These constantly evolving networks present, however, several challenges, mainly due to their stochastic nature, and the high dimensionality of the state and the action spaces. This curse of dimensionality requires, indeed, a broader exploration, which is not always compatible with a real-time execution in an operational network. Thus, we propose DQMC, a new strategy for virtual network embedding in mobile networks combining a Deep Reinforcement Learning (DRL) strategy, namely a Deep Q-Network (DQN), and Monte Carlo (MC). As learning is costly in time and computing resources, and sensitive to changes in the network, we suggest a control-theory-based techniques to dynamically leverage exploration in DQMC. This leads to fast, efficient, and sober learning compared to a Monte Carlo-based strategy. This also ensures a reliable solution even in the case of a change in the requests' sizes or a node's failure, showing promising perspectives for solutions combining control-theory and machine learning.
Complete list of metadata
Contributor : Sophie CERF Connect in order to contact the contributor
Submitted on : Friday, September 30, 2022 - 9:47:47 AM
Last modification on : Tuesday, November 22, 2022 - 2:26:16 PM



Ghina Dandachi, Sophie Cerf, Yassine Hadjadj-Aoul, Abdelkader Outtagarts, Eric Rutten. A robust control-theory-based exploration strategy in deep reinforcement learning for virtual network embedding. Computer Networks, 2022, 218, pp.1-27. ⟨10.1016/j.comnet.2022.109366⟩. ⟨hal-03792078⟩



Record views


Files downloads