Castle Stuart is an early 17th century tower house which was built on the site of an earlier castle.

Originally known as Halhill, it was owned by the Mackintosh family until it was granted to James Stewart, 1st Earl of Moray, by his half-sister, Mary, Queen of Scots, in 1561. He was murdered in 1570 and succeeded by his daughter, Elizabeth Stewart, who became 2nd Countess of Moray. When she married, in 1580 or 1581, James Stewart, son of James Stewart, 1st Lord Doune, her husband became the 2nd Earl of Moray. The 2nd Earl died in 1592 and their son, James Stewart or Stuart, became the 3rd Earl of Moray.

The 3rd Earl was responsible for remodelling the old castle, work which was completed in 1625. However following the Wars of the Three Kingdoms and the fall of the Stuart monarchy the Stuarts went into exile abroad and the castle was left largely unoccupied for three centuries, falling into disrepair.

In 1869 George Stuart, 13th Earl of Moray, commissioned the architects Brown & Wardrop to restore and remodel the castle. In 1977 Douglas Stuart, 20th Earl of Moray, leased Castle Stuart to Charles and Elizabeth Stuart (no relation) on a lengthy peppercorn rent of £50 a year. Despite not actually owning it the new custodians set about restoring the castle, an endeavour which took 15 years and cost over £600,000. Following the refurbishment, which included rebuilding part of the east tower with newly-quarried stone, the castle opened as a luxury bed and breakfast.

In 2009 Castle Stuart Golf Links opened to the north-west of the castle.