The site of Hallyards Castle, now also known as Camilla, may once have been a hunting seat of Malcolm Canmore, and was later owned by the Bishops of Dunkeld. It was the principle residence in the parish of Auchtertool.

In 1539 it passes into more documented history, when it was granted by the Bishop of Dunkeld – at that time it was George Crichton, although some reports suggest it was a Bishop Brown – to Sir James Kirkcaldy of Grange (also written as Kirkaldie).

Sir James was an influential individual, close to James V, and Lord High Treasurer of Scotland. James V visited Hallyards on the way to Falkland Palace following the defeat at Solway Moss in 1542.

Sir James’s son Sir William Kirkcaldy of Grange was an active participant in the murder of Cardinal David Beaton in 1546, and as such Sir James’s lands at Hallyards were forfeited, and father and son sent into exile in France In 1550 the estates were returned to Sir James, and he died at Hallyards in 1556.

Upon his father’s death Hallyards Castle passed to Sir William, who returned to Scotland the same year having been pardoned for his involvement of Beaton’s murder. After initially fighting against Mary Queen of Scots he eventually ended up allied with her, and as Governor of Edinburgh Castle he held it for the Queen, but was forced to surrender to the Regent Morton in 1573 and was hanged.

The estates then passed to Sir John Boswell of Balmuto (whose seat was the nearby Balmuto Castle), and in 1617 to William Forbes of Craigievar.

Following the marriage of William Forbes’s daughter Barbara to Andrew Skene, the barony of Auchtertool (containing Hallyards Castle) was granted to Andrew Skene on the 18th of January 1628. Skene made Hallyards Castle his main residence, and by 1633, and probably earlier, Skene was styling himself Andrew Skene of Halyards. Andrew Skene was the son of Sir John Skene of Curriehill, the Lord Clerk Register and Lord of Session.

In the mid-17th century Andrew’s sister, Katharine Skene, married Sir James Anstruther of Airdrie, the second son of Sir Philip Anstruther of Anstruther.

Hallyards remained in the Skene family until 1715 when a John Skene was forced to sell the estates to the Charles Stuart, the 6th Earl of Moray, due to debts accrued by his father.

The Earls of Moray changed the name of the castle to Camilla, but they were absentee landlords and it was only occupied occasionally during the 18th century. By 1793 they had apparently departed for good, and when a member of the Skene family visited in 1819 he found it in a poor state. In 1847 the majority of it was demolished. Stone from the castle may have been used to build the nearby stable block which has a 19th century appearance.

The remains of the castle that exist today show that it was a sizeable property, with a tall section of the wall at the south-east corner indicating that it was at least three storeys in height.

A sketch of the castle in William Skene’s “Memorials of the family of Skene of Skene” shows two tall corner towers joined by a central block.

There was a vaulted ground floor, with two or three storeys above. Today’s remains are thought to date mainly from the 16th and 17th centuries, but may have included the fabric of an older building. A carved crest over one of the windows in the central block carried the date 1672.

Projecting from the two wings of the castle was a courtyard wall, which enclosed an area measuring around 38 by 27 metres. While most of the stonework has now gone, the plot is still in existence but very overgrown. As well as the tall thin column of masonry at the south-east corner, there is a more substantial section of wall at the north-east corner, leaning at a considerable angle.

The branch of the Skene family known as Skene of Halyards associated with Hallyards Castle in Auchtertool, Fife, should not be confused with a second branch also styled Skene of Halyards, but associated with Hallyards Castle in Kirkliston, Midlothian.

Interestingly both branches appear to have been started by sons of Sir John Skene. The earliest reference to the Fife one seems to be 1539, while the earliest mention of the Midlothian Hallyards is 1579, so it is unclear which is the older name.