Hundreds of people attended both demonstrations which became heated but passed off without violent incident.
The radical Dutch politician Geert Wilders addressed one demonstration, calling for an end to the plans.
The demonstrations were held after ceremonies honouring those killed in the World Trade Center nine years ago.
New York authorities blocked off the street passing the site of the proposed Islamic cultural centre, a short walk away from Ground Zero.
Mounted police and dog units patrolled the streets, keeping the protests separated in two pens a distance away from the site of the former World Trade Center.
The question of building a mosque and cultural centre so close to the scene of the devastation of the 2001 attacks has inflamed passions across US society.
The competing protests attracted people from many different groups, from anti-war activists to Hell's Angels, former US Marines to Buddhists.
Mr Wilders, a right-wing politician from the Netherlands who believes that Islam is comparable with Fascism, told the crowd that the planned cultural centre should not be allowed to go ahead.
"We must never give a free hand to those who want to subjugate us, draw this line so that New York will never become New Mecca," he said.
The rally was also addressed by the former US ambassador to the UN John Bolton and other Republican commentators.
But others said campaigners against the mosque were part of a hate campaign against Muslims.
"I'm really fearful of all of the hate that's going on in our country," Elizabeth Meehan, 51, told the Associated Press.
"People in one brand of Christianity are coming out against other faiths, and I find that so sad, Muslims are fellow Americans; they should have the right to worship in America just like anyone else."
But anti-mosque campaigners, some holding plaques that read "never forget", said the plans were an insult.
"This is hallowed ground. It's something like Gettysburg or Pearl Harbour. Why did they have to do it here? Be a little sensitive," said Theresa Angelo, 57.
At the earlier ceremony relatives read out the names of those who died when hijacked airliners hit the World Trade Center.
Some of the families said the argument between both sides was disrespectful of their families' loss.
But others said that "now was the time to speak out" against the planned Islamic centre.
Earlier, the pastor behind the threat to burn Korans in Florida said the event had been cancelled permanently.
"We will definitely not burn the Koran, no," the Reverend Terry Jones told NBC's Today show. "Not today, not ever."
Earlier, New York Mayor Michael Bloomberg addressed the mourners.
"No other public tragedy has cut our city so deeply, no other place is as filled with our compassion, our love and our solidarity," he said.
Speaking at a memorial event at the Pentagon, also hit by a hijacked plane on 9/11, President Obama paid tribute to those who died.
He said that while it was tempting to dwell on their final moments, the memorial events were taking place "to remember the fullness of their time on Earth".
Mr Obama also repeated his recent calls for unity, saying: "It was not a religion that attacked us that September day. It was al-Qaeda."
"We will not sacrifice the liberties we cherish or hunker down behind walls of suspicion and mistrust."
Prominent New York Imam Feisal Abdul Rauf is at the head of a group who plan to turn an abandoned factory building into a community centre and prayer space.
They say the centre will include facilities for all religions and be a place for reconciliation between faiths.