Overview

A Python comparison of four ways to cross slippery FrozenLake (Monte Carlo, Q-learning, SARSA and a genetic algorithm over deterministic policies) behind a single agent interface and charted by success rate.

FrozenLake is a 4×4 grid with holes in it: start in one corner, reach the opposite one, and falling in ends the episode. With the ice slippery the action you pick is only the direction you intended, and the environment is free to push you sideways instead. The reward is 1 for arriving and 0 everywhere else, so there is no gradient to walk down: the agent gets one bit of feedback at the end of an episode and has to work out which of its moves deserved it.

All four agents implement the same interface, which is what makes the comparison honest: they see the same environment, the same episode budget and the same measurement. The genetic algorithm is the odd one out: it never learns a value function, it just breeds whole policies.

Project 2 of 3 for the Artificial Intelligence course at the Universitat de les Illes Balears, the one on reinforcement learning.

pythonreinforcement-learninggenetic-algorithms