The San Francisco-based bank said on Wednesday that Stumpf was retiring with immediate effect. It is splitting the role of chairman and CEO with Stephen Sanger, its lead director, becoming chairman. Elizabeth Duke, an independent director, will be vice chair.
The changes come weeks after pressure on the bank when it emerged that its employees had potentially opened two million unauthorized deposit and credit card accounts to meet sales targets. Wells Fargo agreed to a $185 million settlement with regulators.
The scandal marked a stunning reversal of fortune for Stumpf, who had built Wells Fargo into the world's most valuable bank with a focus on Main Street-style lending that was the envy of Wall Street.
Politicians were calling on him to resign and a week after a roasting in front of a Senate committee, Stumpf agreed to forgo $41 million in unvested stock awards.
However, that was not enough and at a second hearing, some lawmakers called for the bank to be broken up.
"I have decided it is best for the company that I step aside," Stumpf said in a statement.
The bank's shares rose 2 percent in after-hours trading.
On Monday, the bank had reshuffled its top management to give more responsibility to Sloan.
Wells Fargo is due to report third-quarter earnings on Friday.