Howard Whittemore Memorial Library, built and dedicated on June 1, 1894, was designed by McKim, Mead & White and funded by industrialist John Howard Whittemore as a tribute to his son, John Howard Whittemore, Jr. (1872–1887). This Neo-Classical Revival building is constructed from pink Milford granite and embellished with buff terra-cotta panels, featuring a frieze inscribed with the names of famous authors.

The grand entrance is framed by fluted Ionic columns and bronze lampstands. Above, the design includes a low drum crowning the roof, ringed with anthemion antefixae, and a larger antefix above the pediment.

A discreet modern addition was added in 1962, complementing the original structure while preserving the historical street view.

Designated as part of the Naugatuck Center Historic District on July 30, 1999.