Wilhelm was one of eleven children born to poor tenant farmers John and Ethel (nee Stanley) Wilhelm in Huntersville, North Carolina. He played baseball at Cornelius High School in Cornelius, North Carolina. There, he began experimenting with a knuckleball after reading about pitcher Dutch Leonard.

Though Wilhelm was primarily a starting pitcher in the minor leagues, he had been called up to a Giants team whose strong starting pitchers had led them to a National League (NL) pennant the year before. Giants manager Leo Durocher did not think that Wilhelm's knuckleball approach would be effective for more than a few innings at a time. He assigned Wilhelm to the team's bullpen.  Wilhelm made his MLB debut with the Giants on April 18, 1952 at age 29, giving up a hit and two walks while only recording one out. On April 23, 1952, in his third game with the New York Giants, Wilhelm batted for the first time in the majors. Facing rookie Dick Hoover of the Boston Braves, Wilhelm hit a home run over the short right-field fence at the Polo Grounds. Although he went to bat a total of 432 times in his career, he never hit another home run.  Pitching exclusively in relief, Wilhelm led the NL with a 2.43 ERA in his rookie year. He won 15 games and lost three. Wilhelm finished in the top ten in Most Valuable Player Award voting that season, becoming the first relief pitcher to finish that high. He finished second in the Rookie of the Year Award voting. Wilhelm made 69 relief appearances in 1953, his win-loss record decreased to 7-8 and he issued 77 walks against 71 strikeouts. Wilhelm was named to the NL All-Star team that year, but he did not play in the game because team manager Charlie Dressen did not think that any of the catchers could handle his knuckleball. The Giants renewed Wilhelm's contract in February 1954.  In 1954, Wilhelm was a key piece of the pitching staff that led the 1954 Giants to a world championship. He pitched 111 innings, finishing with a 12-4 record and a 2.10 ERA. During one of Wilhelm's appearances that season, catcher Ray Katt committed four passed balls in one inning to set the major league record; the record has subsequently been tied twice. When Stan Musial set a record by hitting five home runs in a doubleheader that year, Wilhelm was pitching in the second game and gave up two of the home runs. The 1954 World Series represented Wilhelm's only career postseason play. He pitched  2 1/3 innings over two games, earning a save in the third game. The team won the World Series in a four-game sweep.  Wilhelm's ERA increased to 3.93 over 59 games and 103 innings pitched in 1955, but he managed a 4-1 record. He finished the 1956 season with a 4-9 record and a 3.83 ERA in  89 1/3 innings. Sportswriter Bob Driscoll later attributed Wilhelm's difficulties in the mid-1950s to the decline in the career of Giants catcher Wes Westrum, writing that baseball was "a game of inches, and for Hoyt, Wes had been that inch in the right direction."

Did he like baseball at a young age?