Explainable Hierarchical RL for Transparent Decision-Making in Digital Advertising Ecosystems

Abstract

The growing demand for transparency in digital advertising decision-making has become a critical concern for industry practitioners and regulators alike. Traditional advertising allocation strategies often rely on black-box algorithms that lack sufficient explainability, posing significant challenges in environments where user privacy and regulatory compliance are paramount. This paper proposes a novel Explainable Hierarchical Reinforcement Learning (EHRL) framework specifically designed for transparent decision-making in digital advertising ecosystems. The framework integrates option-critic architectures with deep Q-networks and incorporates sophisticated state representation mechanisms to achieve both efficient and interpretable advertising strategies. Our approach utilizes a three-tier hierarchical structure that mirrors natural advertising decision-making processes, from high-level strategic planning to tactical execution. Experimental results on large-scale real-world advertising datasets demonstrate that the proposed EHRL framework significantly improves decision transparency and explainability while maintaining competitive performance. Compared to traditional Deep Q-Network (DQN) approaches, EHRL achieves a 12.3% improvement in click-through rate prediction accuracy, an 8.7% increase in user satisfaction scores, and a 34.5% enhancement in human comprehensibility of decision explanations.