At least eight civilians died in the ambush in south-west Ivory Coast on Friday which also left seven UN peacekeepers dead, the UN says.
The attack near Liberia's border has sparked an "immediate" exodus, Anouk Desgroseilliers told AFP news agency.
The government says it will launch a hunt on Monday to find the perpetrators of the ambush - believed to be the first to target UN troops directly.
UN Secretary-General Ban Ki-Moon said he was "outraged" by the deaths.
UN peacekeepers were deployed to Ivory Coast in 2004 to help end the country's civil war and have stayed through the country's recent political crisis.
Mr Ban offered his "deepest condolences" to the government of Niger and urged the Ivorian government to "identify the perpetrators and hold them accountable".
The peacekeepers were patrolling in an area between the villages of Tai and Para after hearing rumours of an imminent attack on communities in the region, UN officials said.
The Ivorian government says Ivorian soldiers were also killed, but there are no details.
Anouk Desgroseilliers, a spokeswoman for the UN Office for the Co-ordination of Humanitarian Affairs, told AFP that a woman was among at least eight civilians killed in the raids.
"Hundreds of people have arrived in Tai and one can imagine that thousands of others are on the road," she said. Thirty-five families had already crossed into Liberia.
The area has seen a number of cross-border attacks over recent months, blamed on Liberian mercenaries and Ivorian militia fighters who fled Ivory Coast following the arrest of the former president Laurent Gbagbo last year, the BBC's John James in Abidjan reports.
Just last week, UN reinforcements were sent to the region following rumours of a planned attack, our correspondent adds.
Earlier this week, campaign group Human Rights Watch published a report saying militants loyal to Mr Gbagbo based in Liberia had killed at least 40 people in cross-border raids since last July.