The court upheld the decision of a lower court, which included commuting the death penalty of the sixth man to life imprisonment.
The victim, Mukhataran Mai, hit world headlines after speaking out about her ordeal in 2002. She has since become an icon for women's rights in Pakistan.
She said she now feared for her life.
Mukhtaran Mai was her clear and unambiguous self when she spoke minutes after the verdict, the BBC's Shoaib Hasan in Pakistan said.
"The police never even recorded my own statements correctly," she said.
"I don't have any more faith in the courts. I have put my faith in God's judgement now. I don't know what the legal procedure is, but my faith [in the system] is gone.
"Yes, there is a threat to me and my family. There is a threat of death, and even of the same thing happening again. Anything can happen."
Ali Dayan Hassan of the US-based Human Rights Watch said the verdict sent a "very bad signal" across Pakistani society.
"It suggests women can be abused and even raped with impunity and those perpetrating such crimes can walk," he told the BBC.
The Supreme Court ordered the five men's immediate release - but it is not clear if they have been freed yet.
The court has yet to issue a detailed judgement. But the Lahore High Court - whose decision was upheld - had put the blame on a lack of evidence.
Our correspondent says many people say another review of the case is needed as it has had such a key impact on the rights of women in Pakistan.
Mukhtaran Mai was attacked on the orders of the powerful Mastoi clan as a punishment because her brother - 12 at the time - had allegedly been having an affair with a Mastoi woman. This, they said, had brought shame to the entire clan.
An illiterate villager in the eastern province of Punjab, Mukhtaran Mai could have gone the way of many other Pakistani women who are raped, committing suicide.
Instead, she battled against her initial suicidal feelings and began a lengthy legal battle, which has since won her human rights accolades and an iconic status for women's rights in the country.
She runs a school for girls in her village, and has vowed that Thursday's ruling will not force her to leave her home.
"Life and death are in the hands of Allah... I will not shut my school and other projects," she told Reuters news agency.