Brown was born in Mineola, a small segregated town in east Texas marked by racial tensions, to Minnie Collins Boyd and Lewis Brown. Brown was the fourth of five children. During Brown's childhood, mob violence periodically erupted in Mineola, keeping African- Americans from voting. His first job was a shoeshine boy in a whites-only barber shop.

One of Brown's central campaign promises was his "100-Day Plan for Muni," in which he boasted he would fix the city's municipal bus system in that many days. Brown supported the "Peer Pressure" Bus Patrol program, which paid former gang members and troubled youth to patrol Muni buses. Brown claimed the program helped reduce crime. He fired Muni chief Phil Adams and replaced him with his chief of staff Emilio Cruz. In 1998, Brown was Mayor during the summer of the Muni meltdown as Muni implemented the new ATC system and Brown promised riders there would be better times ahead. A voter approved initiative in the following year would help improve Muni services. Brown increased Muni's budget by tens of millions of dollars over his tenure. Brown later said he made a mistake in over promising with his 100-Day Plan.  Brown helped mediate a settlement to the 1997 BART strike.  During his first term as mayor, Brown quietly favored the demolition and abolition of the Transbay Terminal to accommodate the redevelopment of the site for market-rate housing. Centrally located at First and Mission Streets near the Financial District and South Beach, the terminal originally served as the San Francisco terminus for the electric commuter trains of the East Bay Electric Lines, the Key System of streetcars and the Sacramento Northern railroads which ran on the lower deck of the San Francisco-Oakland Bay Bridge. Following the termination of streetcar service in 1958, the terminal has seen continuous service as a major bus facility for East Bay commuters; AC Transit buses transport riders from the terminal directly into neighborhoods throughout the inner East Bay. The terminal also serves passengers traveling to San Mateo County and the North Bay aboard SamTrans and Golden Gate Transit buses respectively, and to tourists arriving by bus motorcoach. Today, the terminal is being planned for redevelopment as a region wide mass transit hub maintaining the current bus services, but with a new tunnel that would extend the Caltrain commuter rail line from its current terminus at Fourth and Townsend Streets to the site. Once completed, Caltrain riders would no longer need to transfer to Muni in order to reach the downtown financial district. Additionally, the heavy rail portion of the terminal would be designed to accommodate the planned High Speed Rail lines to Los Angeles.  In 1998, The Berkeley, California-based Bicycle Civil Liberties Union, produced a two-hour documentary film in the muckraker journalism tradition, July 25: The Secret is Out, which gives evidence of Brown's designs for the Transbay Terminal site.

Did he improve transit?
Brown increased Muni's budget by tens of millions of dollars over his tenure. Brown later said he made a mistake in over promising with his 100-Day Plan.