Willow Lake was built in the late 1800s, by Henry Dubois Van Wyck. Mr. Van Wyck was a native son who ventured west, made a fortune in the California gold rush and returned home. Van Wyck served as the first mayor when Fishkill was incorporated in 1899. He also built Van Wyck Hall in Fishkill as a theater. At Willowlake, the site of the current main house was formerly the location of a hotel built by Mr. Du Bois called Knickerbocker Lodge. The Lodge was featured on old postcards and was a very nice place to go for special occasions. Postcards dated 1906 talk about "our special wedding anniversary" at Knickerbocker Lodge.
The hotel burned in the early 1920s, and J. H. Noah Slee, a South African inventor of "3 -in- 1 Oil" , acquired the property and built a stone house for his wife-to-be, Margaret Sanger. Time named Sanger as one of the hundred most important people of the 20th century.
Fieldstone from the area was gathered to construct the main house, and stone gate house. The main house was finished in 1924, and the stone gate house in 1928. Historical lore contends that the water from Willow Lake was, at one time, bottled and sold to cure diabetes. There are many old structures on the property, and "rescued" old farm implements arranged on one hillside, creating a "mini" museum.