Prior to his retirement in 1850, Mr. Allaire’s main residence was on Cherry Street in New York City. He would only come to the Howell Iron Works periodically for business purposes. During the mid-1830s, a cholera epidemic spread throughout New York City so Mr. Allaire made arrangements to escape the sickness and move his family into the Allaire Residence at the Howell Works. His wife, Frances, had been ill for many years and could not afford to be exposed to the epidemic.
Those living in the house included Frances and Mr. Allaire's cousin Calicia, who would later become the second Mrs. Allaire. Occasionally Mr. Allaire and Frances's daughter Maria would visit and stay with them as well. In March of 1836, Frances died, from what was probably tuberculosis. It was not until 1850 when James P Allaire retired, that he made a permanent residence at the Howell Works with his second wife, Calicia and their young son Hal. Mr. Allaire spent the last several years of his life in this home.
The house was built in three sections. The first section, which contains the front porch, dates back to the 1790s. The kitchen was added on after Mr. Allaire purchased the property, most likely during the 1820s and had a cast iron stove, which was not as common for the time. On the first floor of the house is a mud room, a kitchen, a dining room, a parlor and an office. On the second floor there are three bedrooms and a servants room. A three story brick dormitory was added on to the side of the home as well. This structure housed the workers of the village, guests of the Allaire family and children who attended the school. When Mr. Allaire retired he converted the dormitory for private use. The upper rooms he used for guests while the first floor dining hall was turned into a ball room and banquet hall.