At least 238 people were wounded, Saudi Arabia's Civil Defence body said.
"All those who were wounded and the dead have been taken to hospital. There are no casualties left at the location," General Suleiman al-Amr, director general of the Civil Defence Authority, told al-Ikhbariya television. Strong wind and rains had uprooted trees and rocked cranes in the area, he said.
A statement by a spokesman for the administration of the mosques in Mecca and Medina said the crane smashed into the part of the Grand Mosque where worshippers circle the Kaaba - the black-clad cube towards which the world's 1.6 billion Muslims face to pray.
Pictures circulating on social media showed pilgrims in bloodied robes and debris from a part of the crane that appeared to have crashed through a ceiling.
Saudi authorities go to great lengths to prepare for the millions of Muslims who converge on Mecca to perform the sacred pilgrimage. Last year, they reduced the numbers permitted to make the haj pilgrimage on safety grounds because of construction work to enlarge the Grand Mosque.
The haj, one of the largest religious gatherings in the world, has been prone to disasters in the past, mainly from stampedes as pilgrims rush to complete rituals and return home. Hundreds of pilgrims died in such a crush in 2006.
Saudi authorities have since spent vast sums to expand the main haj sites and improve Mecca's transport system, in an effort to prevent more disasters.
Security services often ring Islam's sacred city with checkpoints and other measures to prevent people arriving for the pilgrimage without authorisation.
Those procedures, aimed at reducing crowd pressure which can lead to stampedes, fires and other hazards, have been intensified in recent years as security threats grow throughout the Middle East.