The violence could reignite tension between the Philippines and Malaysia, threaten growing economic interests in the resource-rich region and even lead to a delay in a Malaysian general election, Reuters reported.
The shootings late on Saturday followed the killings on Friday of two policemen and 12 members of the faction, who are followers of the sultan of Sulu, a south Philippine region, who occupied a Sabah village in February to press their claim.
The killing of the five policemen in an ambush about 150 km (93 miles) away from the main standoff, adds to fears that insecurity is spreading in a region that has been of increasing interest to investors.
Malaysia's inspector general of police, Ismail Omar, tried to ease any worries on Sunday, saying things were under control.
"I don't want speculation that Sabah is in crisis," Ismail told a news conference in the town of Lahad Datu. "We have our security forces at three places to respond."
Sabah is a crucial state in a general election that Malaysian Prime Minister Najib Razak must call by the end of April and which could be the closest in the country's history.
If security in Sabah worsens, he could be forced to delay the election and he would be vulnerable to criticism over the government's handling of the problem. Najib has promised "drastic action" if the group does not leave.
The confrontation had threatened to damage ties between the Philippines and Malaysia. The neighbors have periodically been at odds over security and migration problems along their sea border.
A surge in recent decades of Philippine immigrants to Sabah, many of whom work in palm oil plantations, has sparked resentment and promised to be a hot election issue even before the Sulu sultanate supporters arrived.
Investors will also be concerned about the bloodshed.
Oil majors like ConocoPhillips and Shell have poured in large sums to develop oil and gas fields in Sabah. Chinese companies have been investing in hydro-power and coal mining.
