Australians appear likely to reject the creation of an advocate for the Indigenous population in a referendum outcome some see as a victory for racism.

Two opinion polls published in newspapers on Monday are the latest to show that most respondents oppose enshrining an Indigenous Voice to Parliament in Australia’s constitution.

Creating the Voice would aim to give Australia’s most disadvantaged ethnic minority more say on government policies that affect their lives.

Prime Minister Anthony Albanese would not concede defeat before voting on the referendum ends Saturday.

He told Australian Broadcasting Corporation: “We’ll wait and see when they cast their vote. I’m not getting ahead of the Australian people.

“I know there’s some arrogance has crept into the no-side campaign, but it’s a campaign based upon fear.”

The prime minister has said the world will judge Australians on how they vote at their first referendum since 1999.

Indigenous Australians account for 3.8% of the population and have worse outcomes on average than other Australians in various measures including health, employment, education, incarceration and suicide rates.

Indigenous Australians die around eight years younger than the wider community.

The Yes campaign argues that a Voice, a representative body elected by Indigenous people, would lead to better outcomes.

Human rights lawyer Geoffrey Robertson has warned that the referendum’s failure would be “interpreted by outsiders” as the “vote of an ignorant and racist populace.”

The opposition conservative parties argue that the Voice would be risky because the courts could interpret its powers unpredictably. They also argue that the Voice would divide Australian people racially.

Deputy leader of the conservative opposition Liberal Party Sussan Ley said either result would have a negative impact on Australia.

“It’s a lose-lose, whatever the result is on Saturday,” she told Sky News.

“It will be bad, divisive and unhappy for Australians the next day, so we do need to bring the country together.”

A poll published in The Australian newspaper on Monday showed 58% of respondents opposed the Voice and only 34% supported it.

A poll published in The Sydney Morning Herald on Monday found that 56% of respondents rejected the Voice and only 29% supported it.

More than 2.2 million people had already cast their ballots in early voting by Monday, while a further 1.9 million intended to make postal votes.