Quantum-Enhanced Reinforcement Learning Using Causal Models to Improve Resilience under Observational Interference

Abstract

Deep Reinforcement Learning (DRL) is a sub-field of Machine Learning (ML) that combines reinforcement learning (RL) with deep learning techniques. DRL has showcased promising results in gaming environments as well real-world applications such as self driving vehicles and even natural language processing. However, DRL agents are susceptible to faulty observations due to sudden interference like blackouts, frozen observation, or adversarial interference in practical applications. These scenarios can hamper the learning and performance of DRL agents if they are not resilient. Drawing inspiration from causal inference, the Causal Inference Architecture (CIA) explores a Deep Q-Network (DQN) and a Proximal Policy Optimization (PPO) framework that undergoes training with an additional task focusing on training for observational interference. Through an evaluation conducted in the gymnasium Cartpole-v1 environment, the experimental findings demonstrate that the CIA exhibits enhanced performance and greater resilience against observational interference in both DQN and PPO implementations.

The CIA is further explored using quantum networks and hybrid quantum-classical networks and are tested for their resilience in noisy environments. A novel Quantum-Enhanced CIA (QCIA) is proposed which could replicate the functioning of the CIA using Variational Quantum Circuits (VQCs) and is benchmarked against the corresponding classical architectures for its performance using both DQN and PPO implementations.