Entropy Regularized Reinforcement Learning with Cascading Networks - INRIA - Institut National de Recherche en Informatique et en Automatique Access content directly
Reports Year : 2022

Entropy Regularized Reinforcement Learning with Cascading Networks

Riccardo Della Vecchia
Alena Shilova
  • Function : Author
  • PersonId : 1126323
Philippe Preux
Riad Akrour
  • Function : Author
  • PersonId : 910562


Deep Reinforcement Learning (Deep RL) has had incredible achievements on high dimensional problems, yet its learning process remains unstable even on the simplest tasks. Deep RL uses neural networks as function approximators. These neural models are largely inspired by developments in the (un)supervised machine learning community. Compared to these learning frameworks, one of the major difficulties of RL is the absence of i.i.d. data. One way to cope with this difficulty is to control the rate of change of the policy at every iteration. In this work, we challenge the common practices of the (un)supervised learning community of using a fixed neural architecture, by having a neural model that grows in size at each policy update. This allows a closed form entropy regularized policy update, which leads to a better control of the rate of change of the policy at each iteration and help cope with the non i.i.d. nature of RL. Initial experiments on classical RL benchmarks show promising results with remarkable convergence on some RL tasks when compared to other deep RL baselines, while exhibiting limitations on others.
Ce travail étudie l’utilisation de réseaux de neurones en cascade qui permettent une mise à jour en forme close des politiques, dans le cadre d’algorithmes d’itération de politique avec régularisation entropique.
Fichier principal
Vignette du fichier
MirrorRL-9.pdf (1.24 Mo) Télécharger le fichier
Origin : Files produced by the author(s)

Dates and versions

hal-03793130 , version 1 (30-09-2022)


  • HAL Id : hal-03793130 , version 1


Riccardo Della Vecchia, Alena Shilova, Philippe Preux, Riad Akrour. Entropy Regularized Reinforcement Learning with Cascading Networks. [Research Report] 7003, Inria Lille Nord Europe - Laboratoire CRIStAL - Université de Lille. 2022, pp.16. ⟨hal-03793130⟩
113 View
155 Download


Gmail Facebook X LinkedIn More