Convy was born in St. Louis, Missouri, the son of Monica (nee Whalen) and Bernard Fleming Convy. Convy's family moved to Los Angeles when he was seven years old. He later attended North Hollywood High School where he was an all-around athlete. The Philadelphia Phillies offered him a contract when he was just 17 and he played two years of minor league baseball in 1951-52.

He turned to acting full-time in 1956 and was in the musical The Billy Barnes Revue in Los Angeles before moving to New York City. He appeared in 10 Broadway shows, including "Nowhere to Go but Up", Cabaret (originator of the role of Cliff, Sally Bowles' boyfriend), and The Impossible Years. He played the reporter Hildy Johnson in a 1969 Broadway revival of The Front Page, which starred Robert Ryan. In the original Broadway cast of Fiddler on the Roof with Zero Mostel, Convy played Perchik the Student and sang "Now I Have Everything".  In the 1960-1961 season, Convy guest starred on Pat O'Brien's short-lived sitcom Harrigan and Son as well as the series 77 Sunset Strip in the role of David. He was also cast on an episode of The Mary Tyler Moore Show as Mary's friend Jack Foster, alongside future Alice star Beth Howland. In 1973, Convy was a guest star in two episodes of Hawaii Five-O. Convy also guest starred in an episode of Perry Mason titled "The Case of the Nimble Nephew". He played Harry Thompson, the defendant.  In 1974, Convy portrayed Lt. Steve Ostrowski, the police officer nephew of elderly amateur sleuths on the short-lived series The Snoop Sisters.  Convy attempted a short-lived variety series called The Late Summer Early Fall Bert Convy Show in 1976. In 1979, he appeared with the Dallas Cowboys Cheerleaders in their eponymous television movie. Throughout his career, Convy was a frequent guest star on series such as Bewitched, Hawaii Five-O, The Partridge Family, Mission: Impossible, The Silent Force, The New Phil Silvers Show, Fantasy Island, Charlie's Angels and Murder, She Wrote (including the pilot episode). In 1983, Convy was cast as Neil Townsend on the sitcom It's Not Easy, playing opposite Ken Howard. Convy had joined the project when it was recast after its intended premiere in the 1982-83 season was delayed; he earned the role originally given to Larry Breeding, who was killed in a car accident in September 1982 after the first pilot was shot.  Convy also starred in several movies, perhaps most memorably Semi-Tough (1977), in which he played a caricature of Werner Erhard named Friedrich Bismark. Other film credits included A Bucket of Blood (1959), Susan Slade (1961), Philippe de Broca's Les Caprices de Marie (1970), SST: Death Flight (1977), the horror film Jennifer (1978), Hanging by a Thread (1979), Racquet (1979), The Man in the Santa Claus Suit (1979), Hero at Large (1980), The Cannonball Run (1981), and the television movie Help Wanted: Male (1982). Convy tried his hand at directing with the comedy Weekend Warriors (1986). In 1980, Convy produced and directed the Goodspeed Opera House premiere of the musical Zapata, with music and lyrics by Harry Nilsson and Perry Botkin, Jr., and libretto by Allan Katz.

Was he in any other musicals?
He appeared in 10 Broadway shows, including "Nowhere to Go but Up", Cabaret (originator of the role of Cliff, Sally Bowles' boyfriend), and The Impossible Years.