Pearce's team lost all three of their group matches in Israel and scored just once, from the penalty spot.
FA chairman David Bernstein said it was time for a change.
"He has done extremely well in qualifying for finals tournaments, but after the disappointment of the past two tournaments we believe it is time to change coach.
"It is important the organisation now considers the best approach for the role of Under-21 coach and how this works best within the overall England structure before making any new appointment."
Pearce, the 51-year-old former England defender, has held the position since 2007, and also led Great Britain's men's soccer side at the London Olympics last year.