The marchers were nearly all young adults, and many were draped in EU flags while others waved banners bearing slogans such as "I'm with EU" or simply "Wrexit."
They chanted "what do we want to do? Stay in the EU," as they headed towards the Westminster political district to a soundtrack of songs including Rick Astley's "Never Gonna Give You Up" and Whitney Houston's "I Will always Love You."
Buses full of tourists cheered and cars tooted their horns in support - although one van driver, invited to honk by the crowd, shouted "I'm for out."
Cries of "shame on you" rang out as the march halted briefly outside the Downing Street office of Prime Minister David Cameron, who had himself announced the referendum in 2013.
Many taking part said they had been taken aback by the result of the vote.
"I was genuinely stunned on the morning after the vote," said Nathaniel Samson, 25, from Hertfordshire, north of London.
"I feel deeply uncertain about my future," he added. "I'm on the march to voice my discontentment. I am accepting the result, but it's to show that we won't accept it quietly."
London voted 60 percent in favor of remaining in the EU in last Thursday's referendum, with younger voters widely in favor of staying in the bloc, but the overall result was 51.9 percent in favor of leaving.
The vote to leave has prompted a battle within the ruling Conservative party to succeed Prime Minister David Cameron who said he would resign after the result.
The main opposition Labour Party has also turned on itself, with most of its lawmakers in parliament having voted to withdraw support for party leader Jeremy Corbyn after what they saw as his lackluster contribution to the 'Remain' referendum campaign.
Tom Clarke, 29, from London said: "I'm struggling to find the words to describe how I felt the morning after the vote. I was in mourning."
Italian Pamela Zoni, 34, who has lived in Britain for 6 years said she was very upset and having second thoughts about taking British citizenship.
"I would like a second referendum," she said. "The first campaign was based on lies, and the margin was so tight: it was not a fair result."
The demonstration ended peacefully with a rally outside parliament. No police estimate of the turnout was immediately available but one organizer put it at 50,000.