← All projects
Quant / AI / Built

Reinforcement Learning Market Making Agent

Learning bid and ask offsets while accounting for the cost of holding inventory.

My contribution
Implemented a PPO agent, a custom simulation environment, and inventory aware rewards.
View repository ↗Open external demo ↗

Python / PyTorch / Gymnasium

Original conceptual illustration, not an application screenshot.
Conceptual policy/environment loop, not measured agent performance.

The problem

Market making balances spread capture against inventory exposure. This project explores that tradeoff with reinforcement learning.

Implementation

A PPO agent acts in a custom Gymnasium environment. Microstructure features inform its state; the reward incorporates an inventory penalty inspired by Avellaneda Stoikov. A dashboard exposes quotes and inventory.

Evaluation & limitations

Simulated comparisons with a TWAP baseline require documented data, fills, costs, seeds, and configuration before publication. Simulation does not establish live trading performance.

All projects ↗