Wilhelm was one of eleven children born to poor tenant farmers John and Ethel (nee Stanley) Wilhelm in Huntersville, North Carolina. He played baseball at Cornelius High School in Cornelius, North Carolina. There, he began experimenting with a knuckleball after reading about pitcher Dutch Leonard.

After his retirement as a player, Wilhelm managed two minor league teams in the Atlanta Braves system for single seasons. He led the 1973 Greenwood Braves of the Western Carolinas League to a 61-66 record, then had a 33-33 record with the 1975 Kingsport Braves of the Appalachian League. He also worked as a minor league pitching coach for the New York Yankees for 22 years. As a coach, Wilhelm said that he did not teach pitchers the knuckleball, believing that people had to be born with a knack for throwing it. He sometimes worked individually with major league players who wanted to improve their knuckleballs, including Joe Niekro. The Yankees gave Wilhelm permission to work with Mickey Lolich in 1979 even though Lolich pitched for the San Diego Padres.  Wilhelm was on the ballot for the Baseball Hall of Fame for eight years before he was elected. After Wilhelm failed to garner enough votes for induction in 1983, sportswriter Jim Murray criticized the voters, saying that while Wilhelm never had the look of a baseball player, he was "the best player in history at what he does." He fell short by 13 votes in 1984. Wilhelm was inducted into the Baseball Hall of Fame in 1985. At his induction ceremony, he said that he had achieved all three of his initial major league goals: appearing in a World Series, being named to an All-Star team, and throwing a no-hitter.  He and his wife Peggy lived in Sarasota, Florida. They raised three children together: Patti, Pam, and Jim. Wilhelm died of heart failure in a Sarasota nursing home in 2002.

Did he enjoying his job coaching?