Allen was born Allan Stewart Konigsberg in the Brooklyn borough of New York City, New York. He and his sister, Letty (b. 1943), were raised in Midwood, Brooklyn. He is the son of Nettie (nee Cherry; November 8, 1906 - January 27, 2002), a bookkeeper at her family's delicatessen, and Martin Konigsberg (December 25, 1900 - January 8, 2001), a jewelry engraver and waiter. His family was Jewish, and his grandparents immigrated to the US from Russia and Austria and spoke Yiddish, Hebrew, and German.

Allen is a passionate fan of jazz, featured prominently in the soundtracks to his films. He began playing the clarinet as a child and took his stage name from clarinetist Woody Herman. He has performed publicly at least since the late 1960s, notably with the Preservation Hall Jazz Band on the soundtrack of Sleeper. One of his earliest televised performances was on The Dick Cavett Show on October 20, 1971.  Woody Allen and his New Orleans Jazz Band have been playing each Monday evening at Manhattan's Carlyle Hotel for many years (as of 2011, specializing in classic New Orleans jazz from the early twentieth century). He plays songs by Sidney Bechet, George Lewis, Johnny Dodds, Jimmie Noone and Louis Armstrong. The documentary film Wild Man Blues (directed by Barbara Kopple) documents a 1996 European tour by Allen and his band, as well as his relationship with Previn. The band has released two CDs: The Bunk Project (1993) and the soundtrack of Wild Man Blues (1997). In a 2011 review of a concert by Allen's jazz band, critic Kirk Silsbee of the L.A. Times suggested that Allen should be regarded as a competent musical hobbyist with a sincere appreciation for early jazz: "Allen's clarinet won't make anyone forget Sidney Bechet, Barney Bigard or Evan Christopher. His piping tone and strings of staccato notes can't approximate melodic or lyrical phrasing. Still his earnestness and the obvious regard he has for traditional jazz counts for something."  Allen and his band played the Montreal International Jazz Festival on two consecutive nights in June 2008. For many years, Allen wanted to make a film about the origins of jazz in New Orleans. The film, tentatively titled American Blues, would follow the vastly different careers of Louis Armstrong and Sidney Bechet. Allen stated that the film would cost between $80 and $100 million and is therefore unlikely to be made.

When did Allen start playing music?
He has performed publicly at least since the late 1960s,