Obama, who is in town for international talks to curb climate change, was joined at the site by French President Francois Hollande not long after Air Force One touched down around midnight local time.
After solemnly placing the rose among the flowers and candles at the makeshift memorial, Obama stood silently in tribute, his head bowed and his hands clasped in front of him.
Afterwards he walked away with his arms around Hollande and Paris Mayor Anne Hidalgo, who was also present.
He did not make remarks.
Obama has made a concerted effort to show U.S. solidarity with France after the Nov. 13 attacks, which killed 130 people. He and Hollande agreed during a meeting in Washington last week to ramp up military operations against Islamic State in Syria and Iraq and coordinate intelligence on domestic threats.
By visiting the memorial so soon after his arrival in Paris, Obama signaled the importance he placed on continuing to show support to the people of France.
His motorcade snaked through the city past several of its key landmarks before arriving at the memorial.
The surprise nighttime visit did not appear on Obama's public schedule.
Obama is scheduled to stay in Paris until Tuesday for talks to hash out an agreement with world leaders to fight global warming.