After a few years of drifting, Borgnine attended the Randall School of Drama in Hartford, Connecticut. Following graduation, he went to the famous Barter Theatre in Abingdon, Virginia. His first role was as the Gentleman Caller in Tennessee Williams' "Glass Menagerie". In 1949, he debuted on Broadway in the role of a nurse in the hit play Harvey. In 1951, he moved to Los Angeles, California, where he received his big break in From Here to Eternity (1953), playing the cruel Sergeant "Fatso" Judson, in charge of the stockade, who taunts fellow soldier Angelo Maggio (played by Frank Sinatra). After Maggio ends up in the stockade for being drunk on duty, Fatso beats him to death. However, Fatso meets his own end in a knife fight with Maggio's best friend, Private Robert E. Lee Prewitt (Montgomery Clift) as payback for Maggio's death.

Borgnine built a reputation as a dependable character actor and appeared in early film roles as villains, including Johnny Guitar (1954) and Bad Day at Black Rock (1955). But in 1955, the actor starred as a warm-hearted butcher in the film version of the television play Marty, which gained him an Academy Award for Best Actor. Because of Borgnine's longevity, Marty currently stands as the oldest film with a Best Actor performance from someone still alive. With the passing of Charlton Heston, Borgnine is currently the only living actor who has won Best Actor for performances given prior to 1960. Later film roles include The Vikings (1958), The Flight of the Phoenix (1965), The Dirty Dozen (1967), The Wild Bunch (1969), The Poseidon Adventure (1972) and The Black Hole (1979).

From 1962 through 1966, he starred in the popular situation comedy television series McHale's Navy, for which he received an Emmy Award nomination for Outstanding Lead Actor in a Comedy Series, in 1963. Borgnine also starred in the 1964 film version of the series. Borgnine's later television work included a co-starring role (with Jan-Michael Vincent) as veteran helicopter pilot Dominic Santini in the action/espionage series Airwolf. The series ran from 1984 to 1986.

For his contribution to the motion picture industry, Ernest Borgnine has received a star on the Hollywood Walk of Fame at 6324 Hollywood Blvd. In 1996, he was inducted into the Western Performers Hall of Fame at the National Cowboy & Western Heritage Museum in Oklahoma City, Oklahoma. Also in 1996, Borgnine toured the U.S. in a bus to meet his fans and see the country. The trip was the subject of a 1997 documentary, Ernest Borgnine on the Bus. He also served one year as the Chairman of the National Salute to Hospitalized Veterans, visiting patients in Department of Veterans Affairs medical centers.

Since 1999, Borgnine has provided his voice talent to the cartoon SpongeBob SquarePants as the elderly superhero Mermaid Man (where he is once again paired up with his McHale's Navy co-star, Tim Conway, who voices Mermaid Man's sidekick Barnacle Boy). Borgnine has also appeared on an episode of The Simpsons as himself in addition to a number of television commercials. In 2000, he was the executive producer of Hoover, in which he is the only credited actor. In 2007, Borgnine starred in the Hallmark Original Movie A Grandpa for Christmas. For his performance, Borgnine received a Golden Globe nomination for Best Actor in a Mini-Series or Motion Picture made for Television. At 90, he was the oldest Golden Globe nominee ever.