Ivey began his football career in high school, where he quickly gained recognition for his skill and athleticism on the field. He continued to excel in college, playing for the University of Miami Hurricanes. During his time at Miami, Ivey showcased his ability to shut down opposing wide receivers and make crucial plays for his team.