Born to Italian immigrants in New York City, Gazzara studied at The New School and began his professional career with the Actors Studio, of which he was a lifelong member. His breakthrough role was in the Broadway play Cat on a Hot Tin Roof (1955–56), which earned him widespread acclaim. A memorable performance as a soldier on trial for murder in Otto Preminger's Anatomy of a Murder (1959) transitioned Gazzara to an equally successful screen career. As the star of the television series Run for Your Life (1965–1968), he was nominated for three Golden Globe Awards and two Emmy Awards. He won his only Emmy Award for the television film Hysterical Blindness (2002).

He was a recurring collaborator of John Cassavetes, working with him on Husbands (1970), The Killing of a Chinese Bookie (1976) and Opening Night (1977). His other best-known films include The Bridge at Remagen (1969), Capone (1975), Voyage of the Damned (1976), Saint Jack (1979), Road House (1989), The Spanish Prisoner (1997), The Big Lebowski, Buffalo '66, Happiness (all 1998), The Thomas Crown Affair, Summer of Sam (both 1999), Dogville (2003) and Paris, je t'aime (2006). He also had a successful and prolific film career in Europe, particularly Italy, where he worked with eminent directors such as Giuseppe Tornatore, Giuliano Montaldo, Marco Ferreri, and Lars von Trier.