Answer by taking a quote from the following article:

Colbert was born in Washington, D.C., the youngest of 11 children in a Catholic family. He spent his early years in Bethesda, Maryland. He grew up on James Island, South Carolina. Colbert and his siblings, in descending order by age, are James III, Edward, Mary, William, Margo, Thomas, Jay, Elizabeth, Paul, Peter, and Stephen.

While at Northwestern, Colbert studied with the intent of becoming a dramatic actor; mostly he performed in experimental plays and was uninterested in comedy. He began performing improvisation while in college, both in the campus improv team No Fun Mud Piranhas and at the Annoyance Theatre in Chicago as a part of Del Close's ImprovOlympic at a time when the project was focused on competitive, long-form improvisation, rather than improvisational comedy. "I wasn't gonna do Second City", Colbert later recalled, "because those Annoyance people looked down on Second City because they thought it wasn't pure improv - there was a slightly snobby, mystical quality to the Annoyance people". After Colbert graduated in 1986, however, he was in need of a job. A friend who was employed at Second City's box office offered him work answering phones and selling souvenirs. Colbert accepted and discovered that Second City employees were entitled to take classes at their training center for free. Despite his earlier aversion to the comedy group, he signed up for improvisation classes and enjoyed the experience greatly.  Shortly thereafter, he was hired to perform with Second City's touring company, initially as an understudy for Steve Carell. It was there he met Amy Sedaris and Paul Dinello, with whom he often collaborated later in his career. By their retelling, the three comedians did not get along at first - Dinello thought Colbert was uptight, pretentious and cold, while Colbert thought of Dinello as "an illiterate thug" - but the trio became close friends while touring together, discovering that they shared a similar comic sensibility.  When Sedaris and Dinello were offered the opportunity to create a television series for HBO Downtown Productions, Colbert left The Second City and relocated to New York to work with them on the sketch comedy show Exit 57. The series debuted on Comedy Central in 1995 and aired through 1996. Although it lasted for only 12 episodes, the show received favorable reviews and was nominated for five CableACE Awards in 1995, in categories including best writing, performance, and comedy series.  Following the cancelation of Exit 57, Colbert worked for six months as a cast member and writer on The Dana Carvey Show, alongside former Second City castmate Steve Carell, and also Robert Smigel, Charlie Kaufman, Louis C.K., and Dino Stamatopoulos, among others. The series, described by one reviewer as "kamikaze satire" in "borderline-questionable taste", had sponsors pull out after its first episode aired and was cancelled after seven episodes. Colbert then worked briefly as a freelance writer for Saturday Night Live with Robert Smigel. Smigel brought his animated sketch, The Ambiguously Gay Duo, to SNL from The Dana Carvey Show; Colbert provided the voice of Ace on both series, opposite Steve Carell as Gary. Needing money, he also worked as a script consultant for VH1 and MTV, before taking a job filming humorous correspondent segments for Good Morning America. Only two of the segments he proposed were ever produced and only one aired, but the job led his agent to refer him to The Daily Show's then-producer, Madeline Smithberg, who hired Colbert on a trial basis in 1997.

What was his first gig?
A friend who was employed at Second City's box office offered him work answering phones and selling souvenirs.