Once known for its ad campaigns mostly featuring nearly-nude teen models behaving in sexually suggestive ways with each other, the company has toned down sexually charged imagery and no longer features scantily dressed models in its advertisements. According to then-Chairman Arthur Martinez (in 2016), these changes were implemented to show that the company is evolving with its consumers.

The original Abercrombie & Fitch was founded in 1892 in New York City by David T. Abercrombie as an outfitter for the elite outdoorsman. Ezra Fitch—a wealthy lawyer, real estate developer, and devoted Abercrombie customer—bought a significant stake in the business in 1900. In 1904, it was incorporated and renamed "Abercrombie & Fitch Co." Fitch eventually bought out Abercrombie's share of the business, becoming its sole owner from 1907 to 1928. The company was an elite outfitter of sporting and excursion goods, particularly noted for its expensive shotguns, fishing rods, fishing boats, and tents At one time, it had outfitted Theodore Roosevelt's safari and Adm. Richard E. Byrd's expedition to Antarctica. Ernest Hemingway was also a regular customer; the gun with which he committed suicide in 1961 was purchased from Abercrombie and Fitch. Following Hemingway's death, his wife placed several of his guns on consignment with the company.

By the 1970s, A&F was struggling to compete with lower priced competitors while trying to maintain its high-end image. It was known for holding an extensive inventory of lavish items, but high operating expenses forced A&F to shed its highest priced items, such as an $18,000 (equivalent to $125,599 in 2021) gold and onyx chess set. Cash flow problems forced the company to also cut its inventory of moderately-priced products.

In 1976, Abercrombie & Fitch filed for Chapter 11 bankruptcy protection. In 1977, the company closed its New York flagship store at Madison Avenue and East 45th Street.

While Abercrombie & Fitch went out of business during its bankruptcy, the brand survived: in 1978, Oshman's Sporting Goods, a Houston-based retail chain, bought the defunct firm's name and mailing list for $1.5 million (equivalent to $6 million in 2021). Oshman's relaunched the company as a mail-order retailer specializing in hunting wear and novelty items. Retail stores were also opened in Beverly Hills, Dallas, and (by the mid-1980s) New York City.

In 1988, Oshman's sold the brand and its operations to Columbus, Ohio-based The Limited (parent company of several retail clothing chains, including Victoria's Secret). Under The Limited (which later rebranded itself as L Brands), A&F gradually shifted its focus to young adults, was later spun off as a separate, publicly traded company, and eventually grew into one of the largest apparel firms in the United States.

Since 1997, A&F has kept a high profile in ways both positive (ads, brand expansions, philanthropy) and problematic (legal conflicts, boycotts, ads, clothing style controversies, employment practices). Mike Jeffries became CEO in the 1990s and refocused the brand on the teen customer. In 1998, the company built on its success by launching a children's product line, Abercrombie Kids, for 7-14 year olds. By 1999, A&F was more popular than ever. In 2000, A&F launched its Hollister Co. subsidiary, "a new concept focused on the optimistic, laidback California lifestyle".

A&F encountered headwinds again in the first decade of the 21st century, when the Great Recession battered the company's business as teenagers turned to high-style/low-price "fast fashion" brands like H&M and Forever 21 for their clothing.