The lands of Whittingehame are first mentioned in the early 13th century when Patrici de Withichun granted a charter of Lochaneshalech, part of his lands of Spott, to Melrose Abbey. This would seem to be Patrick, Earl of Dunbar, so it seems likely that he would have had a castle here.

In 1372 George de Dunbar, 10th Earl of Dunbar and March, granted Whittingehame to his siter, Agnes, upon her marriage to James Douglas of Dalkeith.

The tower is rectangular in plan, with a protruding staircase jamb to the north, and is three storeys high, with a castellated parapet. It is likely to be of late 15th or early 16th century, however it may incorporate parts of an earlier structure.

In 1576 William Douglas of Whittingehame and George Auchinleck of Balmanno, both Senators of the College of Justice, were responsible for cross-examined one Bessye Dunlop as a suspected witch in Dalkeith.

The tower has been restored and is in use as a residence.