Some buildings in New York are so stunning, they make you stop in your tracks. The Knox Hat Factory, at Grand Avenue and St. Marks Place in Prospect Heights, is that kind of building.

Historic brick building with multiple stories, featuring fire escapes and large windows, set on a city street with parked cars and trees.

It almost looks like a castle, with the yellow brick factory walls fortified to protect the interior, and a corner watchtower to help spot and vanquish invaders.

But of course this is Brooklyn, and in 1890 when the hat works opened as the world’s largest hat factory, it was just one more manufacturing building in a city filled with companies that made things.

A close-up view of the top corner of a historic brick building with a tower featuring arched windows and decorative molding against a clear blue sky.

As companies go, however, Knox Hatworks has a long history in Gotham. “Founded by Irish immigrant Charles Knox, the Knox Hat Company began operations in lower Manhattan, selling beaver hats in a small store he opened in 1838,” states the Brooklyn Public Library.

“Through promotion and word of mouth, the business built an impressive clientele, including Abraham Lincoln, who purchased a new Knox stovepipe right before his 1860 speech at the Cooper Institute.”

After Charles Knox—aka “Knox the Hatter”— died, his son, Civil War hero Edward, became the head of the company.

Black and white postcard image of the Knox Hat Factory, featuring a large brick building with a clock tower, situated at the intersection of St. Marks and Grand Avenues. The scene includes several people and a horse-drawn carriage in front.

In the ensuing years, teh original Knox store on Fulton Street burned down. Edward Knox relocated the hat shop to 212 Broadway, then to the posh Fifth Avenue Hotel on 23rd Street.

In the 1880s, he decided Knox should get into the manufacturing business. A letter published in the Brooklyn Daily Eagle in 1952 recalled the factory’s early days.

“When this plant first opened, all the men who worked there wore high hats to and from work,” stated the cheeky letter writer. “What a time we kids had during snowball weather.”

A brick building corner with an engraved stone featuring text that reads 'Hat Works of Knox the Hatter 1890' and 'Founded by Charles Knox 1840'.

Knox Hatworks continued production through World War II, but eventually the building was abandoned, according to the Brooklyn Public Library. “In the mid 1980s developers bought the building and converted it to apartments.”

The Brooklyn Public Library’s post about the factory included a photograph dating to 1905—showing a spectacular clock tower with a mansard roof, now removed.

It’s unfortunate, but at least the developers kept the terra cotta panels at the corner of the building, which gives a little backstory on the building and company that curious passersby love to know. (Though the founding date of the company carved into terra cotta is wrong, per Lost New York City.)

[Third image: Brooklyn Public Library]