Patterson began his football career playing college football at the University of Tennessee, where he quickly made a name for himself as a dynamic playmaker. As a junior in 2012, he garnered national attention for his exceptional skills as a wide receiver and return specialist. Patterson was then drafted in the first round of the 2013 NFL Draft by the Minnesota Vikings.