CityLearn is an open source OpenAI Gym environment for the implementation of Multi-Agent Reinforcement Learning (RL) for building energy coordination and demand response in cities [15, 17]. A major challenge for RL in demand response is the ability to compare algorithm performance [19]. Thus, CityLearn facilitates and standardizes the evaluation of RL agents such that different algorithms can be easily compared with each other.

demand response

Districts and cities have periods of high demand for electricity, which raise electricity prices and the overall cost of the power distribution networks. Flattening, smoothening, and reducing the overall curve of electrical demand helps reduce operational and capital costs of electricity generation, transmission, and distribution networks. CityLearn allows the easy implementation of reinforcement learning agents in a single or multi-agent setting to reshape their aggregated curve of electrical demand by controlling the storage of energy by every agent.


CityLearn has been utilized in the following projects and publications:



Incentive based demand-response

Deltetto et al. [1]

Collaborative demand-response

Glatt et al. [3]

Independent energy management

Nweye et al. [7]

Coordinated energy management

Kathirgamanathan et al. [4], Pinto et al. [10], Pinto et al. [12], Vazquez-Canteli et al. [18]

Benchmarking reinforcement learning algorithms

Dhamankar et al. [2], Nweye et al. [6], Pinto et al. [11], Qin et al. [13]

Voltage regulation

Pigott et al. [9]

Cite CityLearn

   doi = {10.48550/ARXIV.2012.10504},
   url = {},
   author = {Vazquez-Canteli, Jose R and Dey, Sourav and Henze, Gregor and Nagy, Zoltan},
   keywords = {Machine Learning (cs.LG), Artificial Intelligence (cs.AI), FOS: Computer and information sciences, FOS: Computer and information sciences, I.2.1},
   title = {CityLearn: Standardizing Research in Multi-Agent Reinforcement Learning for Demand Response and Urban Energy Management},
   publisher = {arXiv},
   year = {2020},
   copyright = {Creative Commons Attribution 4.0 International}


CityLearn is an open-source project that continues to benefit from community-driven updates and suggestion. Please, visit the CityLearn GitHub repository to join the community and start contributing!

Indices and tables