Carr was born Joseph Francis Karr on October 23, 1879, at his parents' home in the Irish neighborhood on the East End of Columbus, Ohio. His father, Michael Karr, was a shoemaker who was born in Ireland in 1841 and immigrated to the United States in 1864. His mother Margaret Karr was born in New York to Irish immigrant parents. Carr had five older siblings, Bridget, James, John, Mary, and Michael, and a younger brother, Edward.

In 1900, Carr organized a baseball team made up of employees of the railroad's Panhandle Division. The team, known as the Famous Panhandle White Sox, played in the Capital City League and the Saturday Afternoon League in Columbus for several years. According to the Chicago Tribune, Carr's Panhandle club "gained a reputation in semi-professional ranks throughout the country."  In 1907, Carr began a long association with the sport of football. He obtained permission from the Panhandle Athletic Club to reorganize the Columbus Panhandles football team, a team that had been formed in 1900 or 1901 and disbanded in 1904. He secured players from the railroad shop where he worked. The core of Carr's Panhandles teams were six Nesser brothers who worked at the shop and were excellent athletes. To save on expenses, the players, who were railroad employees, used their passes to ride the train for free and practiced during the lunch hour on the railroad yards. Over the next 13 years, the Panhandles became known as a traveling team, as Carr saved money on travel expenses and stadium rental by playing mostly road games. In 1921, the Fort Wayne Journal called the Panhandles the "most renowned professional football aggregation in the country."  Carr also continued his association with professional baseball while running the Panhandles, serving for several years as the secretary/treasurer and later president of the Ohio State League, a minor league baseball circuit.  As early as 1917, Carr was one of the leading advocates of a plan to develop a national professional football league. Sources are not in agreement as to what role, if any, he played in the formation of the American Professional Football Association (APFA), which later became the National Football League (NFL). However, once the APFA was formed in 1920, Carr's Panhandles played in the league's inaugural season. The 1920 Panhandles team played only one home game and compiled a 2-7-2 record.

Did that help the team?
The core of Carr's Panhandles teams were six Nesser brothers who worked at the shop and were excellent athletes.