In medieval times the prosperous port of Cley was centred near the magnificent church of St. Margaret of Antioch. The town was one of the principal ports of East Anglia, exporting large quantities of wool and grain. The River Glaven was navigable to large ships as far as Glandford and ran into the sea half a mile north west of Cley.
In the early 17th century the river began to silt up, largely as a result of reclamation done by Sir Henry Calthorpe, and new wharves were built near deep water by the present quay. Cley Windmill dates from the 1700s although the tower was not completed until some time later. The Customs House and other prominent buildings were also built at this time after much of the old town was destroyed by fire.
The Windmill was first offered for sale in the Norfolk Chronicle on 26th June 1819 and passed through several hands during the next one hundred years. The best known of the millers was Steven Barnabas Burroughs, whose family worked and owned the Windmill from 1840 to 1919, after which it fell into disrepair. In 1921 it was bought by Sarah Maria Wilson and converted into a holiday home. The conversion involved removing most of the working parts and fixing the cap and sails. The old stones, used for grinding the flour, were set into the ground nearby and the warehouses were converted into stables and boat sheds.
In 1934 the windmill passed to Sarah's grandson, Hubert Blount, who made many improvements, including replacing the sails in 1960. During the war years the windmill was used by his aunt, Sister Rachel, who, with another nun, Sister Catherine, became legends in the local area. The Duchess of Bedford, who was lost at sea flying her own aircraft, was one of the frequent visitors during this period.
In 1953 the sea again came to Cley which suffered its worst flooding for 400 years. The windmill stood firm but much of the furniture was damaged or washed away. A sea wall was built around the remainder of the village. In 1979 the windmill passed to Charles and Jane Blount (parents of James, who is known to many as the singer and songwriter James Blunt). In 1983 the windmill was renovated and converted into a guesthouse. The sails, fanstage and galleries were replaced in 1988.
Colonel Blount and his wife continued to run the windmill until 2005 when it was taken over by their nephew John Woodhouse and his wife Val. They sold the property in 2006 to Dr. Julian Godlee, his wife Carolyn and their four children.