Brown began his youth career at Stevenage before going on to play for a number of clubs in the English lower leagues. He made his senior debut for Peterborough United in 2016 after impressing in their youth system. Brown then went on loan spells to clubs like St. Neots Town and Kettering Town, where he continued to showcase his talent.