This is the fourth residence to be built on the site. The first house was built in 1852 and burnt just 3 months later. The second, Cary Castle, was built in 1860 and burned in 1899. The third house, built in 1903, burned in 1957 and was rebuilt by 1959. The gardens have been less troubled. Some of the Garry oaks are older than any of the houses and the the garden has a showy Victorian air. There are 14 acres of gardens and the rest of the grounds consist of the rare Garry oak ecosystem. Theme gardens include the Bruce Pavilion, Rotary Garden of International Friendship, Victorian and Sunken Rose Gardens, Winter, Iris, Perennial, Alpine, Heather, Herb, Great Hall Terraces, Cut Flower, Cottage and English Country Gardens.