Before Society had a single tile laid in the squillion-dollar 80 Collins development, it was one of the most talked about restaurants in town. What a burden to carry. And what a time to have to build such a thing.

To say Society hasn't had an easy start is an understatement. The pandemic, devastating to the whole industry, was catastrophic for a restaurant designed to run with a brigade of 450 souls. To make matters worse, in August 2021, after just one week of service in the middle of multiple lockdowns, the restaurant's headline acts, chef Martin Benn and partner Vicki Wild, exited stage left.

To say that was a shame is also an understatement. Because in that shining week, with Benn commanding this almighty ship, Chris Lucas' dream about Society becoming one of Australia's best restaurants came true.

I walked in worried. Mine and everyone else's expectations for this restaurant were stratospheric. But I walked away filled with excitement. Relief. Benn's dishes, painstakingly shaped over two years of lockdowns, showed the work. The beauty. The detail. The absolute mastery of technique.

Sure, the premium ingredients came thick and fast – osetra caviar perched on a pretzel with your martini makes quite the opening statement. But all were used with absolute purpose and precision.

The room? I wasn't a huge fan from pre-opening press. Cool, austere. The soaring glass cage shrouded in misty drapes, ringed with navy banquettes and black tables, features no colour. Instead, it is overhung by sharp, mirrored chandeliers of a scale that dazzle and dwarf all who sit beneath them. But as a backdrop to Benn's dishes, the space morphed into something dramatic, precise.

There is also the matter of Society's cellar. Deep as the ocean, with a sky's-the-limit price range (a 1945 Chateau Margeaux for $44,000, madam?), compiling the list was a dream come true for sommelier Loic Avril. The former Dinner by Heston star was essentially given a blank cheque to secure high-roller essentials, including vintages of Chateau d'Yquem you won't find anywhere else in the country.

Six months later, Avril and his wonder cellar remain. As does that room. And despite Benn's departure, so does most of his menu, overseen now by his second-in-command, Luke Headon. But my worry has returned.

On two subsequent visits the food was still very good, often great. A seafood platter featuring raw scallops and caviar mounted on richly buttered crumpets, curried crab in crunchy little cups and cured ocean trout is an incredible, albeit pricy, $125 entry.

Lobster ($47.50) comes in ravioli form, caviar dancing in its buttery sauce. It's perfectly lovely, but it lacks the daring and drama of the original lobster course.

Of course, replacing Benn's menu is going to take time. Come March, I'm told it will make way for Headon's vision.

On my most recent visit, I try one of his creations. Still-translucent butter-poached scallops rest on sheep's curd, with wafers of apple and a vibrant pistachio oil. Each element is lovely, but for me, it's not cohesive. The result is strangely sweet, almost tropical.

It may seem unfair to dwell on the departure of Benn and Wild. But a restaurant of this scale and ambition needs visionary but also extremely seasoned leaders at the helm.

Case in point, Benn's wagyu rib cap, a highlight of round one with its antler-like spikes, rich jus and tiny mushrooms. When I revisit the dish, the beef isn't rendered sufficiently, losing that visceral sensation of melting meat.

Headon has plenty of skill. He's been trained by the best, and has runs on the board at the Fat Duck and Ezard at Levantine Hill. But Society didn't just promise good, or even great. It promised revolution.

Society, like all restaurants, is doing the best that it can in the current climate. The senior staff here are unimpeachable. Unsurprisingly, it is they who serve this restaurant critic on each visit. But the many greener hands that my friends and colleagues have been served by slipped and stumbled as they tried to learn this serious art on the fly.

In the face of its challenges, there is still much to love about the place. What a wine list. What a room. But life ain't easy at the top. Even though Society is a la carte, prepare to spend $200 to $300-a-head plus extra depending on your thirst. At this level, slips sink ships.

You cannot blame us for having such great expectations. Big promises were made. We can only pin our hopes on a better future where talented staff flow freely back into our country; where Headon finds his voice; and where everyone's dream for this world-class restaurant is realised. I've seen how good it can be. And I believe it can rise again.

• Gemima Cody reviews Society's sibling, Lillian Terrace

Cost: Entrees $32-$50; mains $60-$73; large shared dishes $185-$330.

Drinks: Show-stopping Australian and international wines, sakes, everything.