Answer by taking a quote from the following article:

The Osmonds are an American family music group with a long and varied career. The Osmond Brothers began as a barbershop quartet consisting of brothers Alan, Wayne, Merrill and Jay Osmond. They were later joined by younger siblings Donny and Jimmy, both of whom enjoyed success as solo artists as well. With the addition of Donny, the group became known as The Osmonds, and enjoyed its greatest success as both a boy band and a hard rock ensemble.

By 1976, though, the group's record sales were softening; Osmondmania had long faded and been replaced by "Rollermania", as the Bay City Rollers became the hot pop band of choice in the mid-1970s. Their 1975 album The Proud One sold poorly (despite the title track providing a chart-topping easy listening hit and the group's last U.S. top-40 hit to date), and MGM Records was sold to Polydor Records. Their first album under Polydor was the album Brainstorm; that album sold only slightly better than its predecessor, and its lead single, "I Can't Live a Dream," fell short of the top 40. Polydor would release two more albums from the family (a Christmas album that included all of the performing family members, and a greatest hits compilation).  The Osmonds poured themselves into a new venture: the older brothers began producing The Donny & Marie Show which was a hit on ABC from 1976-1979. But the success came at a cost. The family built and operated at great expense a first-class television studio in Orem, Utah, where the show was produced beginning in 1977. As a result, the Osmonds as a performing band became a lower priority to Donny and Marie. The older brothers deferred or gave up their dreams of being a rock-and-roll band. Donny experienced stage anxiety and Marie had a brief bout with an eating disorder after a network executive told her she looked heavy.  When the show was cancelled in 1979, the Osmonds were taken by surprise, as they had believed that the show would be renewed, and found themselves in debt and without a clear direction. The group left Polydor and attempted another album for Mercury Records, Steppin' Out, which was a major failure with neither the album nor any of its singles charting anywhere; it would be their only album on that label. Steppin' Out was a transitional album for the Osmonds and was produced under the auspices of Maurice Gibb; among its tracks was the first recorded version of "Rest Your Love on Me," a country song that would become a hit for Gibb's own band, the Bee Gees, and topped the country charts in a cover version by Conway Twitty. They produced two unsuccessful projects for Marie, a sitcom pilot that never aired and a variety show revival that lasted seven episodes in 1980 and 1981.  They recovered and eventually paid their debts and re-established their careers. Rather than go into bankruptcy, they resolved to honor all of their financial obligations. But the Osmond artists and enterprises began operating separately.

Why were there issues?
Donny experienced stage anxiety and Marie had a brief bout with an eating disorder