Oliver played college football at the University of Houston, where he quickly gained a reputation as a dominant force on the defensive line. During his time at Houston, Oliver earned numerous accolades, including being named an All-American and winning the Outland Trophy as the best interior lineman in college football.