Obama was speaking to reporters at the White House after talks with German Chancellor Angela Merkel, who also supported a move to wider sanctions and said the European Union and the United States would continue to work in concert on the issue.
"The next step is going to be a broader-based sectoral sanctions regime," Obama said. Merkel agreed, saying that May 25 was a critical date and "we will see to it that elections can take place."
The United States and the EU have imposed several rounds of sanctions on individuals and some companies to try to persuade Russian President Vladimir Putin to halt any interference in Russian-speaking areas of eastern Ukraine.
Energy and banking sectors are two of the most likely areas if those sanctions are widened.
"In Europe, we have taken a decision that (should) further destabilization happen, we will move to a third stage of sanctions. I would like to underline this is not necessarily what we want, but we are ready and prepared to go to such a step," Merkel said.
Obama said the purpose of sanctions was not to punish Russia, but to give it an incentive "to choose a better path."
Obama called on Russia to persuade pro-Russian military groups in Ukraine to stand down and said it was disgraceful that the militias were holding international observers.
The two leaders said they were united in their desire to impose costs on Russia for its actions in Ukraine and for backing Ukraine, including financially.
"We're united in our support for Ukraine, including the very important IMF program approved this week," Obama said, referring to the approval of a $17 billion two-year aid program for Ukraine.
He said the Russian account of events in eastern Ukraine that there was a spontaneous uprising by pro-Russian activists was belied by the use on Friday of surface-to-air missiles that brought down two of Ukraine's military helicopters.
"It is obvious to the world that these Russian-backed groups are not peaceful protesters. They are heavily armed militants," Obama said.