Paynesville by name, but not by nature. Here in this popular retirement town, superyachts trawl the waters alongside super-pelicans, made enormous by the same thing that makes chef Mark Brigg's newish seafood diner so good. That would be the goods plundered from just outside – black bream, Lakes Entrance sardines, surf clams – which Briggs is tweaking and serving in a sleek room of buffed grays, blond woods and succulents.

Sardine Eatery is fighting the good fight for sustainable seafood eating, though in a different way to envelope-pushing Saint Peter in Sydney. There, chef Josh Niland is feeding his diners everything from fin to gill. Here, Briggs is championing the lesser-loved fish.

Briggs has slowly but surely pared back his cooking style over the years. For the better. Once the head chef at Vue de Monde, he left to set up Paul Mathis' Sharing House, where he once served ocean trout gravlax outlandishly dressed in freeze-dried and jellied apple (a tasty, but busy dish). These days, Briggs seems content to let the produce do the talking. As he should. East Gippsland might be one of the tastiest patches in the state.

Snacking looks like crisp whitebait dusted with a fragrant orange salt to swipe through jalapeno aioli – fishy chips with eyes. Fresh-baked bread comes with a seaweed-flecked butter, setting an umamiful ocean tone. A lot of what you're eating depends on who's knocked on the kitchen door that day. Often, the one-bite bubbly wonton crisp is loaded with ponzu-dressed yellow tuna tartare. Today it's alfonsino, a deep-sea white fish so fresh it's almost squeaky, lifted further with a zingy kick of the Japanese citrus fruit yuzu.

You don't need to love things that swim. The fish bits are bolstered by local charcuterie, sticky wedges of Gippsland brie, Mount Zero olives and a super juicy salt-and-pepper quail, brined then fried in the finest lacy batter and finished with saltbush crisps.

This is refined but vitally, likeable food. That matters. This 60-seater on the water has destination dining potential, especially since the seafood market in Australia is so strangely sewn up that it's near-impossible for chefs to get at the goods being pulled in on their doorsteps. Local seafood on the seashore is stupidly rare. But Sardine will largely be serving its local community. Hence: lamb shoulders and oxtail ragu, and at weekends, breakfast that takes the form of fresh-baked crumpets and sardines on toast.

Speaking of, those little fish have a big influence. Briggs claims they're his favourite and he makes a case here. If you've only experienced them as mega pungent things in tins, or heavily salt-cured, eating them fresh is a revelation. These Lakes Entrance silvertails have been butterflied, somewhat creepily looped closed with tails in mouths, and grilled with a buttery-fresh parsley and lemon filling. It's a full-flavoured but not overly "fishy" time.

Other likeable wins: a warm jumble of tender octopus, firm kipfler potatoes and salty pork belly, bright with capers, dill and gherkin. Then so-soft black bream, another under-loved local, lives its best death on a creamy spiral of mash, girt by big surf clams, a classic beurre blanc and out-of-season, but still sweet, peas.

On the floor, Briggs' partner Victoria Hollingsworth calmly works a room that's not always as relaxed. She fields a lot of questions about portion sizes and in one notable case whether the scallops are "real". Dish by dish, you see the room relax as the Four Pillars gin and tonics, and the warmth of the fire, sinks in.

Dessert is where Briggs brings some of that fine-dining razzle to the table, which will thrill fans of desserts that look like Pollocks. A sherbert-like mandarin sorbet comes dressed in the fresh fruit, airy bits of chocolate, dabs of lemon curd and – most interesting – crunchy, fizzy house-made honeycomb using local honey. Or there's always a wedge of cloth-bound cheddar from Maffra if you'd prefer something less cerebral. Chuck in a hot toddy made with ginger tea, you've got a well-baited hook.