From the harbourside backyard of his Point Piper mansion, Robert Magid enjoys an impressive vista of Sydney – although not of the three buildings for which the 81-year-old Rich Lister is most famous.

As Magid and The Australian Financial Review Magazine stroll one perfect autumn afternoon across the lawn by his pool, we gaze at the thicket of skyscrapers to the east and try to make out Highgate. It used to be the Esso office building, until Magid’s father reluctantly bought it for $29 million in 1992, and the son converted it into the first residential apartments in the heart of Sydney’s central business district.