The vote pits former anti-Taliban fighter Abdullah Abdullah against ex-World Bank economist Ashraf Ghani after neither secured the 50 percent majority needed to win outright in the first round on April 5.
As most foreign troops leave by the end of 2014, whoever takes over from Karzai will inherit a troubled country with an increasingly violent Taliban insurgency and an economy crippled by corruption and the weak rule of law.
The process has been fraught with accusations of fraud by both candidates and many fear a close outcome will make it less likely the loser will accept defeat, possibly dragging Afghanistan into a risky and protracted stand-off over the vote.
From windswept deserts on the Iranian border to the remote, rugged Hindu Kush mountains, 12 million eligible voters will start casting ballots at 7 a.m. (0230 GMT) amid tight security at 6,365 polling centers.
"The country is in a crisis ... Only a strong leader can rescue it," said Shukria Barakzai, a female member of parliament.
"Everyone - young, old, rich and poor - came out in unpleasant weather, despite threats, to vote in April and we hope it will be the same this time. This is Afghanistan's spirit," she said.
The Taliban may prove a formidable obstacle. The insurgents, now at the height of their summer offensive, have warned people not to vote in an election they have condemned as a U.S.-sponsored charade.
"This time the Taliban will try to compensate for what they couldn't achieve in the first round of the election," said Defence Ministry spokesman General Zahir Azimi.
The high turnout of nearly 60 percent in the first round was a major defeat for the Taliban. Observers expect fewer than 5 million voters this time, partly because of concern about security.
Officials in Kabul are haunted by the prospect of a close outcome that could furnish the losing candidate and his supporters with an excuse to reject defeat, and, in the worst scenario, propel the country back into war along ethnic lines.
Both candidates set the stage for complaints with repeated attacks on electoral organizers, accusing them of incompetence and bias.
"Some of the teams openly played ethnic politics and that is not good for the country," said Habibi Aminullah, former campaign manager to Qayum Karzai, the president's brother, who was a candidate in the first round.
"I hope the election ends at a point in which no violence takes place. I hope the international community helps the country."
The United Nations has appealed to candidates to refrain from attacking the organizers to safeguard the process.
"There's a short-term gain only in trying to undermine or bully the institutions at the expense of their legitimacy," said United Nations deputy chief Nicholas Haysom.
"It's going to be the legitimacy of the elections which will give legitimacy to the new head."
Abdullah polled 14 percentage points ahead of Ghani in the first round with 45 percent of the vote, but Ghani, who is ethnic Pashtun, stands to gain a portion of the Pashtun vote that was splintered in the first round.
Pashtuns are Afghanistan's biggest ethnic group, making up about 45 percent of the population.
Abdullah is partly Pashtun but is identified more with the ethnic Tajik minority.
The chances of an equal split between candidates are hard to gauge because there are few reliable polls. ACSOR research center, asking respondents to choose between Abdullah and Ghani, predicted a 50:50 split shortly before the first round.
A more recent survey by Glevum Associates indicates that Ghani may have overtaken Abdullah, predicting 49:42 in Ghani's favor.