Dryburgh Abbey House incorporates a sixteenth century tower house within an 18th and 19th century mansion.

The tower house, part of the Mertoun estate owned by the Haliburton family from the 15th century, dated back to at least 1572 and possibly earlier. Little is known about the tower or what it looked like, but it may have been contemporary with its near neighbour Mantle House (now the Dryburgh Abbey Hotel). Both occupy good defensive positions within a bend of the River Tweed.

In 1784 it was demolished by David Erskine, 11th Earl of Buchan, who had recently acquired the Dryburgh estate, and the remaining parts incorporated into a new mansion. The rectangular pitched roof entrance wing on the north side may possibly pre-date the 1784 house and it is thought that this may have formed the rear elevation of the old tower. The south front is thought to date from 1784 and probably covers the former entrance to the tower.

The house was largely destroyed by a fire in 1892 and was rebuilt in the Scottish Renaissance style by the architect Henry Francis Kerr between 1892 and 1894 for the then owner Oswald Erskine. A staircase survives from the older (presumably 1784) building.

A cylindrical doocot, thought to date to the 16th century, stands within the grounds of the house to the south-west (a carved date stone marked 1828 is a later insertion). A 17th century sundial from Dryburgh Abbey House stands to the east of Harden in Roxburghshire.

Dryburgh Abbey House is now a private family home owned by the Scott-Plummer family, formerly of Sunderland Hall.