Answer by taking a quote from the following article:

Joe Frazier was the 12th child born to Dolly Alston-Frazier and Rubin in Beaufort, South Carolina. He was raised in a rural community of Beaufort called Laurel Bay. Frazier said he was always close to his father, who carried him when he was a toddler "over the 10 acres of farmland" the Fraziers worked as sharecroppers "to the still where he made his bootleg corn liquor, and into town on Saturdays to buy the necessities that a family of 10 needed." Young Frazier was affectionately called "Billie Boy."

According to an article from The New York Times, "over the years, Frazier has lost a fortune through a combination of his own generosity and naivete, his carousing, and failed business opportunities. The other headliners from his fighting days--Ali, George Foreman, and Larry Holmes--are millionaires." Asked about his situation, Frazier became playfully defensive, but would not reveal his financial status. "Are you asking me how much money I have?" he said. "I got plenty of money. I got a stack of $100 bills rolled up over there in the back of the room." Frazier blamed himself, partly, for not effectively promoting his own image. In a 2006 HBO documentary on the fight in Manila, Frazier was interviewed living in a one-room apartment on the second floor of his gym.  His daughter Jackie Frazier-Lyde is a lawyer and worked on her father's behalf in pursuit of money they claimed he was owed in a Pennsylvania land deal. In 1973, Frazier purchased 140 acres in Bucks County, Pennsylvania for $843,000. Five years later, a developer agreed to buy the farmland for $1.8 million. Frazier received annual payments from a trust that bought the land with money he had earned in the ring. However, when the trust went bankrupt, the payments ceased.  Frazier sued his business partners, insisting his signature had been forged on documents and he had no knowledge of the sale. In the ensuing years, the 140 acres was subdivided and turned into a residential community. The land is now worth an estimated $100 million.

Who were his business partners?