The Statistical Account of Scotland describes “a tower at Dolphiston” which seems to correspond to the location of Dolphinston Farm (however it may be mistaken with Dolphiston Tower nearby). But the description sounds like it might be a broch rather than a castle or peel tower:

“The chief fortification is a tower at Dolphiston, said to have been built by one Dolphus, from whom it took its name. The walls are from 8 to 10 feet thick, built of hewn stone, and so closely cemented with lime, that it is found more difficult to obtain stones for building from it, than from a quarry. It has been extensive, and divided into small apartments by stone partitions. Several vaulted apertures are in the middle of the walls, large enough for a small bed, and some of them so long, as to be used by the tenants for holding their ladders.”

That sounds very like the description of a broch to me. I will keep it listed as both a castle and a broch until I can do more research to try and figure out which it is. It may be that a castle and a broch both existed on the site.

The Statistical Account refers to a “watch tower or light house” standing on rising ground to the south of this tower or broch. Assuming that it is referring to the supposed site of Dolphiston Tower, this suggests that the tower or broch it is describing did indeed occupy the site of Dolphinston Farm slightly to the north.