Police are prepared to arrest protesters who humiliated mainland shoppers in Tsim Sha Tsui on Sunday.
"It's extremely regrettable," Secretary for Security Lai Tung-kwok said yesterday.
"The behavior of the protesters not only caused a disturbance for tourists but also shocked and insulted [them]."
About 100 people who responded to an online call to rally in support of controls on the number of mainland tourists entering the SAR had gathered at the Star Ferry on Sunday afternoon.
But troubles followed as they ran into several dozen members of pro-Beijing groups while heading to shops along Canton Road.
Mainland shoppers kept their heads down and stayed inside stores as the rival groups traded abuse and scuffled.
Still, the police mostly kept the groups apart by setting up barriers.
Lai did not mention the pro-Beijing groups as he pointed out that all citizens have a right to express their opinions while staying within the law and respecting the rights of others.
But Sunday's action, he said, "caused scenes of chaos, which damaged public order and social stability. The police are reviewing the case and will follow it up."
Members of the protest group could have breached the law with their improper actions in a public place, he added, and that could mean up to 12 months in prison and a fine of HK$5,000.
But Ronald Leung Kam-shing, who organized the protest, was unrepentant and asked what could follow in the push for curbs on the number of visitors - accused of crowding locals out of shops and off public transport.
"The government has neglected our demands," he said. "Does it mean citizens who voice their concerns in future will face criminal investigation?"
Senior counsel and lawmaker Ronny Tong Ka-wah said the police would have to prove the protesters had used threatening or humiliating words to prosecute people.
Meanwhile, other top officials joined Lai yesterday in condemning Sunday's action.
Some of the strongest condemnation came from Secretary for Constitutional and Mainland Affairs Raymond Tam Chi-yuen.
The scenes he saw on television "shocked me," he said.
"I believe such behavior, which is barbaric and uncivilized from my point of view, is against our core values."
Chief Secretary for Administration Carrie Lam Cheng Yuet-ngor said such behavior should not be tolerated, but stressed it does not represent the views of the majority of citizens.
Secretary for Commerce and Economic Development Greg So Kam-leung said anti-tourist acts are intolerable and the scenes "should not be repeated."
Instead, So said, "we hope citizens can maintain the hospitable spirit, continue to welcome tourists and make them feel at home."
Across the border, the state-run Global Times quoted He Maochun, director of the Research Center for Economic Diplomacy Studies at Tsinghua University, as saying the protest action showed free speech in Hong Kong can be anything but dignified.