Smallwood began his career at Middlesbrough, where he came up through the youth academy before making his first-team debut in 2008. He went on loan spells to Rotherham United and Scunthorpe United before signing permanently with Rotherham in 2014. Smallwood spent four seasons with Rotherham, helping the club win promotion to the Championship in 2018.