A castle at Blyth was mentioned in 1581 however its exact location has been lost.
Around 1350 John de Graham, “lord of Torboltoun and of Roberestoun in Conyggam”, confirmed charters to his uncle, Henry de Graham, of “the lands of Menarstoun and of Philipstoun in the barony of Abyrcorn and of Blyth in the valley of Twed”. Henry had received the lands from his father, Sir John de Graham, who was the grandfather of John de Graham, with “the reddendo for said lands of Blyth being a pair of gilt spure or 12d. sterling at the option of said Henry ‘in name of feu-ferme’ and for the other lands one penny each ‘in name of feu-ferme'”.
I’ve placed the location at Blyth farm pending further research.
Alternative names for Blyth
Blythe