Carter was born in Clifton, New Jersey, the fourth of seven children. He acquired a criminal record and was sentenced to a juvenile reformatory for assault, having stabbed a man when he was 11. Carter escaped from the reformatory in 1954 and joined the Army. A few months after completing infantry basic training at Fort Jackson, South Carolina, he was sent to West Germany.

After his release from prison in September 1961, Carter became a professional boxer. At 5 ft 8 in (1.73 m), Carter was shorter than the average middleweight, but he fought all of his professional career at 155-160 lb (70-72.6 kg). His aggressive style and punching power (resulting in many early-round knockouts) drew attention, establishing him as a crowd favorite and earning him the nickname "Hurricane." After he defeated a number of middleweight contenders--such as Florentino Fernandez, Holley Mims, Gomeo Brennan, and George Benton--the boxing world took notice. The Ring first listed him as one of its "Top 10" middleweight contenders in July 1963. At the end of 1965, they ranked him as the number five middleweight.  He fought six times in 1963, winning four bouts and losing two. He remained ranked in the lower part of the top 10 until December 20, when he surprised the boxing world by flooring past and future world champion Emile Griffith twice in the first round and scoring a technical knockout. That win resulted in The Ring's ranking of Carter as the number three contender for Joey Giardello's world middleweight title. Carter won two more fights (one a decision over future heavyweight champion Jimmy Ellis) in 1964, before meeting Giardello in Philadelphia for a 15-round championship match on December 14. Carter landed a few solid rights to the head and staggered Giardello in the fourth, but was unable to follow them up, and Giardello took control of the fight in the fifth round. The judges awarded Giardello a unanimous decision.  After that fight, Carter's ranking in The Ring began to decline. He fought nine times in 1965, winning five but losing three of four against contenders Luis Manuel Rodriguez, Dick Tiger, and Harry Scott. Tiger, in particular, floored Carter three times in their match. "It was," Carter said, "the worst beating that I took in my life--inside or outside the ring." During his visit to London (to fight Scott) Carter was involved in an incident in which a shot was fired in his hotel room.  Carter's career record in boxing was 27 wins, 12 losses, and one draw in 40 fights, with 19 total knockouts (8 KOs and 11 TKOs). He received an honorary championship title belt from the World Boxing Council in 1993 (as did Joey Giardello at the same banquet) and was later inducted into the New Jersey Boxing Hall of Fame.

When did he retire from boxing?