In a solemn, yet joyful ceremony before a large crowd, an honor guard of white and black state troopers lowered the flag shortly after 10 a.m. (1400 GMT) before a large crowd and live TV cameras.
The relocation of the flag came a little over three weeks after the racially motivated massacre of nine black worshippers during a Bible study session on June 17 at a landmark black church in Charleston.
The banner's new home will be the "relic room" of the state military museum in Columbia, South Carolina's capital, where the flag will reside with other artifacts carried by Southern Confederate soldiers 150 years ago in the Civil War.
South Carolina Republican Governor Nikki Haley, who pushed for the state legislature to enact a law making it possible to remove the flag, was among those watching its departure from its place just yards from the State House entrance.
Earlier, she called it a great day for the state in an interview with NBC's "Today" television show.
"I'm thinking of those nine people today," Haley said, referring to the nine men and women gunned down at Charleston's African Methodist Episcopal church.
Also among those watching the flag's lowering was Charleston Mayor Joe Riley, who in 2000 led a 120-mile (190-km) march from his city to Columbia to protest the flag.
"Today, at long last, this has been done," he said in a statement.
The white man charged in the killings, 21-year-old Dylann Roof, appeared in photographs posing with a Confederate flag that surfaced on a website bearing a racist manifesto. The image spurred politicians and leading national retailers to pull the flag from display.
In South Carolina, the first state to secede during the 1861-1865 U.S. Civil War, this week's debate in the state legislature brought an emotional closure to a symbol long divisive in the state.
The Confederate flag waved atop the state capitol from 1961 to 2000, when it was moved to a Confederate war memorial near the State House entrance.
"My heart is overjoyed. I can feel the togetherness," said Tenetha Hall, of Newberry, South Carolina, who said she took a day off work to drive an hour to Columbia to watch the flag come down. "I'm so glad my children and six grandchildren will get to see this history."
Civil War re-enactor Bobby Dawson stood quietly in gray wool pants and a long-sleeved shirt with the wool coat of the Confederate Army's 101st South Carolina Regiment. He said he had an ancestor who fought on both sides of the Civil War.
"This brings joy to some people, a solemn occasion to others," Dawson said