"I'm not telling you they don't have some ability at some point to go back and get involved," Kerry said in an interview with CNN's "State of the Union" program. "But they also have the ability to get killed doing that.
"These guys pick a fight with us, in the future or now or at any time, at enormous risk."
Concern that the five former inmates might return to fight has been a factor in controversy in the United States over the deal, under which Army Sergeant Bowe Bergdahl was handed over to U.S. troops in Afghanistan on May 31 after nearly five years in captivity and the five Taliban members were flown from Guantanamo to Qatar.
Kerry said the United States has proven its ability to target al Qaeda fighters in western Pakistan and Afghanistan in the war on terrorism and that nobody should doubt the capacity of the United States to protect Americans.
U.S. officials have referred to release of the Islamist militants in exchange for Bergdahl as a transfer. They said the men would be subject to certain restrictions in Qatar, which brokered the deal. One official said that would include a minimum one-year ban on travel outside of Qatar as well as monitoring of their activities.
"They're not the only ones keeping an eye on them," Kerry told CNN.
The five had been held at the U.S. Guantanamo Bay prison in Cuba since 2002. Their release drew criticism from members of the U.S. Congress, who complained they were not notified in advance by the Obama administration.
Adding to the controversy, some of Bergdahl's former comrades have alleged he was captured by the Taliban after deserting his post.
Kerry said there was time to sort through exactly what happened in Afghanistan but he fiercely defended the rescue of Bergdahl.
"It would have been offensive and incomprehensible to consciously leave an American behind. No matter what," said Kerry, a Vietnam War veteran.