The attacker entered the Baqir-ul-Olum mosque shortly after midday as worshippers had gathered for a ceremony, the interior ministry said in a statement.
Fraidoon Obaidi, chief of the Kabul police Criminal Investigation Department, said at least 27 people were killed and 35 wounded but that total may rise significantly.
One survivor told Afghanistan's Ariana Television that around 40 dead and 80 wounded had been taken from the building before rescue services arrived at the scene. "I saw people screaming and covered in blood," he said.
Another witness, Sayed Ali, said he had helped carry 30-35 bodies from the mosque and believed another 100 people may be wounded.
Bloody sectarian rivalry between Sunni and Shi'ite Muslims has been relatively rare in Afghanistan, a majority Sunni country, but the attack underlines the deadly new dimension that growing ethnic tension could bring to its decades-long conflict.
Already there had been two major recent attacks on Shi'ite targets in Kabul, both claimed by Islamic State.
Last month, at least 14 people were killed when a gunman in police uniform opened fire on worshippers gathered at a Shi'ite shrine for the Ashura festival. In July, more than 80 people were killed in an attack on a demonstration by the mainly Shi'ite Hazara minority.
There was no immediate claim of responsibility for Monday's attack but the Taliban, seeking to reimpose Islamic law after they were toppled from power in 2001, denied any involvement.
"We have never attacked mosques as it's not our agenda," said the movement's main spokesman, Zabihullah Mujahid.
Any resurgence of sectarian or ethnic violence could threaten the fragile stability of the government headed by President Ashraf Ghani, who described the "vicious attack" as an attempt "to sow seeds of discord".
Government Chief Executive Abdullah Abdullah said Afghanistan should not fall victim to "enemy plots that divide us by titles".
"This attack targeted innocent civilians - including children - in a holy place. It is a war crime & an act against Islam & humanity," he said in a message on his Twitter account.
Thousands of civilians have been killed in Afghanistan in the 15 years since the Taliban regime was brought down in the U.S.-led campaign of 2001.
In July, the U.N. Assistance Mission in Afghanistan reported that 1,601 civilians had been killed in the first half of the year alone, a record since it began collating figures in 2009.