Located on one of the highest hills in Columbia, the mansion was built in 1855 and was added to the U.S. National Register of Historic Places in 1970. The Caldwell-Bolyston House is located on the north side of the mansion grounds. The gardens were started by the builder of the Caldwell-Boylston House (John Caldwell) in the 1830s. The pea gravel paths are in their original 1830s location. Basically a green garden, the garden is planned on four tiers and features elegant fountains, rectangular boxwood enclosed beds and some large Magnolia and Crape Myrtle trees.