Anderson began his career at Crystal Palace, joining their youth academy in 2013. He made his first-team debut for the club in 2015, but struggled to break into the regular starting lineup. Anderson was loaned out to numerous clubs during his time at Palace, including Doncaster Rovers, Bolton Wanderers, and Swindon Town.