Police have sharply criticised protesters in Melbourne’s CBD after two officers were taken to hospital and rocks, glass and rotten fruit were thrown at officers during violent scuffles.

Police spent hours on Sunday afternoon trying to keep anti-immigration protesters and counter-protesters apart. The counter-protesters were accused of fuelling the violence, although they disputed that claim on Sunday night.

A visibly frustrated Commander Wayne Cheesman said protesters had pelted police with bluestone rocks, similar to those found on railway tracks, and bottles filled with glass shards, set bins on fire and even threw a cantaloupe as tensions boiled over at the rallies on Sunday.

“Melbourne has had a gutful,” Cheesman said moments after holding up a large rock, which he said had been thrown at police. “The disruption to Melbourne, to the general public who want to come into our fine city, the disruption to businesses … we really need to find an answer of what we do. Enough is enough.

“It appals me, really. This could kill someone. That’s the bottom line.”

He tipped out a cardboard box filled with other rocks during a heated press conference following the protests.

“I show you the rocks because I think the public needs to see what the police are being targeted with,” he said.

“It’s a bad day for Victoria Police. It’s a bad day for Melbourne. It’s a bad day for the community.”

Rally organisers hit back via social media on Sunday night, claiming some protesters were hospitalised due to police actions.

“We condemn the violent acts of Victoria Police,” rally organiser Yasmin said.

“We are appalled by the comments of police commander Wayne Cheesman who praised the far right hate march. It is not peaceful to organise a rally calling for Black and Brown people to not be allowed in this country.

“Police deployed violence which injured activists, with nearly five hospitalised. Police denied thousands the right to protest, while facilitating the racist, hateful demonstration March for Australia, exercising white sovereignty to defend a white Australia ideology.”

Cheesman praised March for Australia organisers for being “peaceful, engaging and they did what they were told”.

Premier Jacinta Allan thanked police on Sunday night and said some people “just want to protest for the sake of protest”.

“While they have the right to do so in a democratic system, they have an obligation to do it peacefully,” Allan said.

“The moment they get violent like they did today – picking up rocks and hurling them at police – they deserve to face serious consequences.”

Opposition Leader Brad Battin said Victorians would be rightly angry about the protests and demanded greater action from the government.

“This was a disgraceful attack on our police officers,” he said. “Being pelted with rocks and bottles isn’t free speech – it’s criminal violence.”

Lord Mayor Nick Reece accused protesters of ruining the city and pledged to work with the state government to end the “cycle of madness”.

“What we are seeing on the streets of our great city is a form of civic insanity,” he said.

Hundreds of police were on the streets on Sunday as the crowds of opposing demonstrators, estimated at about 1000 people each, gathered at the rallies. Racist slogans were chanted at an anti-immigration protest, while Australian flags were burnt by pro-immigration counter-protesters.

One female sergeant was taken to hospital with a suspected broken hand after being kicked by protesters, police said.

A male senior constable sustained a non-life-threatening cut to the lower leg and was taken to hospital.

“I have a member with a nasty gash on his leg from Corio,” Cheesman said.

Three other officers were hit by rocks but not injured. One 30-year-old woman from Brunswick West was arrested.

Cheesman blamed dozens of demonstrators from “issue-based groups on the left” for the violent clashes.

He accused some protesters of coming to the rallies to fight police, labelling them cowards who hid behind face coverings, masks and hooded jumpers.

“They pull the rocks out of their bags, and they throw them as hard as they can at the police, and it’s got to stop,” he said.

“Some of them are professional protesters who certainly don’t come to protest. They come to fight.”

He said the right-wing March for Australia rally had largely listened to police instructions and made their way peacefully through the city to the steps of parliament.

He said some of the counter-protesters were known to police and more arrests could follow as investigations worked to establish the identities of some in the crowd.

Cheesman estimated there were about 1000 protesters on both sides, but only about 50 “hardcore protesters” were behind much of the violence.

The veteran police officer said that while he respected people’s right to protest peacefully, violent clashes at protests were draining police resources from all around the state every week.

“We have enough problems with the crime crisis, and I’ll call it a crisis because ... we don’t have enough police,” he said.

“Our police should be out on the street. They shouldn’t be here today, dealing with people who are trying to harm us.”

Police were largely successful in keeping the two groups separate, with a barricade erected on Bourke Street and riot police using flash bangs and rubber bullets to disperse counter-protesters.

Medics were seen treating and bandaging the legs and chests of those who sustained suspected injuries from what eyewitnesses said were police-deployed flash bangs.

Cheesman said no injuries from the devices were reported to police.

He defended the use of force by police and said the level of aggression by protesters was almost on par with the violence that unfolded during ugly clashes in Melbourne outside a major weapons expo last year.

”We have them as an option now for every protest because we are seeing more aggressive behaviour,” he said. “I don’t think the members should be target practice for people who want to hurt us.”

Officers attempted to make further arrests on Sunday, but fights broke out as eggs and glass containers filled with water were thrown at the police line. Capsicum spray was used, and a police officer was knocked over in the fracas.

As crowds gathered carrying Australian flags for the anti-immigration March for Australia rally at the steps of Parliament House, an organiser shouted into a megaphone “we will not be replaced” in an apparent reference to a racist conspiracy theory.

After this masthead revealed last month that Australia’s most prominent neo-Nazi group, the National Socialist Network (NSN), had been secretly running the March for Australia rallies around the country, the group backed out of the second rally on Sunday.

But key organisers running Sunday’s marches, which were again held around Australia, are known associates of the neo-Nazi group.

That includes Melbourne organiser white supremacist Matt Trihey, who has hosted events attended by the NSN, and headlined speeches at Sunday’s march.

The Age identified in the crowd a number of NSN members in plain clothes, who at times led chants of “white man fight back” or rushed to confront counter-protesters.

As Trihey spoke to the crowd, blaming immigration for soaring crime rates, chants in the distance from the anti-racism counter-protest grew louder.

When counter-protesters took up a chant of “no Nazis ever again” on the other side of the police line, the anti-immigration crowd all but abandoned speeches on Parliament House’s steps to shout back at the approaching demonstration.

The counter-protest was initially led by a procession of First Nations people who marched from Camp Sovereignty on Sunday morning.

Hundreds of people chanted, “No hate, no fear, refugees are welcome here” and “Nazi scum off our streets” as they walked along Swanston Street towards parliament.

Palestinian man Basil earlier told a crowd at the State Library that anti-immigration talking points were created to divide people and the counter-protest movement needed to stick together.

“Those sick ideologies that see another human as less equal and with less rights simply because of their colour, their ethnicity – that ideology is the most destructive that you can face,” he said.

A breakaway group of counter-protesters exchanged jeers with four people holding Australian flags near the corner of Bourke and Elizabeth streets at 2.15pm.

Counter-protesters advanced on the four men and stole one of their large Australian flags, sparking a fistfight on the steps of an office building. Three police cars arrived at the scene to disperse the men. No one was arrested.

Rallies under the banner of “Unite Against Racism: Migrants and Refugees Are Welcome” were also held in Sydney, Canberra, Brisbane, Perth and Adelaide on Sunday.

In August, Melbourne’s CBD roiled with protests as anti-immigration and counter-demonstrators clashed repeatedly and a large number of police were stretched to cope with a shifting struggle that raged through the city’s streets for more than four hours.

In the Melbourne crowd on Sunday, One Nation’s Victorian director Warren Pickering told this masthead that “politicians hadn’t been allowed to speak” at the marches.

Pickering said he had not attended the first march in August because neo-Nazis were involved, but this time he was hopeful the rally was more about the key issue of immigration.

“We do need to be careful, the stigma associated with One Nation has been a problem for a lot of years,” Pickering said.

Asked about the known neo-Nazis calling out racist chants on Sunday, Pickering said: “I don’t actually know any of these guys.”

On August 31, after the March for Australia rally, a group of men, including neo-Nazis, attacked the standing First Nations protest site, Camp Sovereignty, in Melbourne’s Kings Domain.

Two days later, Premier Jacinta Allan was forced to cut a press conference short when neo-Nazi leader Thomas Sewell gatecrashed the event.

Chatlogs and footage obtained by this masthead has previously revealed how neo-Nazi leadership used far-right influencers to sell the March for Australia rallies as a “spontaneous” groundswell of “everyday Australians”, while they stacked crowds with plain-clothed neo-Nazis and sent key members to other states to headline rallies.

Start the day with a summary of the day’s most important and interesting stories, analysis and insights. Sign up for our Morning Edition newsletter.