Swilling attended Brother Martin High School in New Orleans, where he quickly established himself as a standout football player. He was highly recruited by several colleges but ultimately decided to play for the Georgia Tech Yellow Jackets. During his time at Georgia Tech, Swilling showcased his skills on the field, becoming a key player for the team.