Downes began his footballing career at the Ipswich Town academy, where he worked his way up through the ranks before making his first-team debut in the 2017-2018 season. He quickly became a key player for Ipswich Town and impressed fans and scouts alike with his technical ability and composure on the ball.