In a charter dated the 14th of May 1491, James IV granted the “the island and rock of Inchgarvie” to John Dundas of that Ilk, Inch Garvie being off the coast of the Dundas estate centred on Dundas Castle.

Dundas was also given “the power for the said John and his heirs to build, construct and erect on high upon the same rock and island of Inchgarvie a castle or fortalice, of whatever height, length and breadth as shall seem most expedient to the said John and his heirs; with iron bolts, ramparts, portcullises, crenellations and machicolations, and with all other reinforcements and fortifications which may be devised or conceived for the keeping of the said castle, with moat and gaol”.

The building of the castle seems to have been interrupted by the death of John Dundas in 1495, and it wasn’t completed until 1513.

Inchgarvie Castle was used as a state prison, then from 1650 to 1651 it was occupied by the Royalists who built a fort around it and used it to defend the Forth against Cromwell. It continued to be used as a prison until 1671, when it was replaced in that task by the Bass Rock.

In Napoleonic times the Civil War fort was repaired and mounted with cannon, then during the Second World War an anti-aircraft battery was installed on the island. As such, the remains of the castle are now incorporated into the later concrete defences.