Richardson began his professional career with the youth academy of Charlton Athletic, where he honed his skills and developed his passion for the game. After impressing scouts with his performance, he signed his first professional contract with Charlton Athletic and made his senior debut for the club in 2012.