Reinforcement learning for delta-hedging in illiquid markets

This paper addresses the problem of delta-hedging in illiquid markets, where transaction costs, limited depth of the limit order book, and market impact of large trades play a significant role. Classical approaches based on the Black–Scholes model assume continuous trading and infinite liquidity, which leads to significant distortions in practice. To overcome these limitations, we propose a reinforcement learning approach with a risk-averse Bellman operator. As the training environment, we employ an agent-based exchange simulator with support for trading the underlying asset and options, which reproduces market microstructure and limit order book dynamics. A DeepLOB convolutional encoder is used to extract order book features and capture hidden liquidity characteristics. Numerical experiments show that the proposed method produces a realized PnL distribution centered around zero with lighter tails compared to the classical Black–Scholes delta-hedger. Furthermore, the risk-aversion parameter $\lambda$ enables control over the trade-off between mean profitability and tail-risk management. The results demonstrate the efficiency of the approach and its applicability for constructing robust hedging strategies in illiquid markets.