* Gulbis hits back to beat Federer
* Djokovic makes light work of Tsonga
* Bouchard flying
* Murray survives epic fifth set in third round
The Swiss fourth seed, who won the title in 2009, seemed to have the match in his grasp against the unpredictable Gulbis but faded badly to lose 6-7(5) 7-6(3) 6-2 4-6 6-3.
World No.2 Djokovic destroyed local favourite Tsonga 6-1 6-4 6-1 to set up a meeting with eighth seed Milos Raonic of Canada, who advanced with an emphatic 6-3 6-3 6-3 victory over Spain's Marcel Granollers.
Last year's runner-up Maria Sharapova, the seventh seed, survived a shaky start to beat Australian Sam Stosur 3-6 6-4 6-0, winning the last nine games to set up a quarter-final against Spanish sensation Garbine Muguruza, who rounded off a bad day for the host nation with a 6-4 6-2 win against France's Pauline Parmentier.
Another fast-rising talent, Canadian Eugenie Bouchard thrashed German eighth seed Angelique Kerber 6-1 6-2 to claim a quarter-final spot, crunching 30 winners in 52 dazzling minutes to stretch her winning run on clay to nine matches.
Andy Murray lived to fight another day after a five-set thriller against Philipp Kohlschreiber in the third round.
While Djokovic made it to his 20th consecutive grand slam quarter-final, it was the first time since 2004 that 32-year-old father of four Federer failed to reach the last eight in Paris. He has now fallen before that stage in three of his last four grand slam tournaments.
"Mentally I have already switched to the grass, to be quite honest," said Federer. "For me, it's like, okay, claycourt season was fun, but we are moving on. Clay doesn't need me anymore, I got flushed out here."
Former world No.1 Federer, who became father to a second set of twins recently, looked a slightly forlorn figure as Gulbis turned around the match and the days when he was a near certainty to reach semi-finals appear to be over.
The 17-times grand slam champion was beaten in the second round at Wimbledon last year and the last 16 at the U.S. Open although he did enjoy a semi-final run in Australia.
On yet another gloomy day in Paris after some welcome sunshine on Saturday, Federer never looked comfortable against Gulbis, who said earlier this week he was hoping to catch the "last-chance train" after under-achieving since reaching the quarter-finals of the Frecnh Open in 2008.
"It is the biggest win of my career. I had to win, I know how all of you love Roger but that's sport," world No.17 Gulbis said on the court as the stunned fans looked on.
"I won tournaments in Marseille and Nice so hopefully Paris is the next one. I win, I win over some fans," he added, looking ahead to his next match against sixth seeded Czech Tomas Berdych who ended U.S. interest in the men's singles with a 6-4 6-4 6-4 trouncing of towering John Isner.
Federer, who squandered two set points for a two-set lead, left court Philippe Chatrier head bowed after allowing Gulbis to come back and dominate with some relentless attacking tennis.
At 5-2 in the fourth set Gulbis took a medical timeout and he came back with all guns blazing to break Federer's serve immediately, on the way to winning 10 of the next 11 points against a visibly rattled opponent.
Federer still managed to take the set but he was never the same again and Gulbis controlled his own suspect temperament to claim victory when a Federer backhand went wide.
The French crowd were hoping that 13th seed Tsonga would cheer them up, but Djokovic had other ideas.
It took him 89 minutes to tear Tsonga apart, opening a 5-0 lead and never looking back, ending the ordeal on the first match point.
Seventh seed Murray reached the fourth round, winning 3-6 6-3 6-3 4-6 12-10 against Germany's Philipp Kohlschreiber having re-started the match on Sunday locked at 7-7 in the fifth after bad light stopped play the previous evening.
He will play Spain's Fernando Verdasco next after he also returned to finish off Frenchman Richard Gasquet with ease.
With so many top seeds gone from the women's draw, 20-year-old Bouchard is now looking like a serious title contender, but so is crowd favourite Sharapova, who once again showed her resilience on court Suzanne Lenglen.
The 18th seeded Bouchard, who reached the Australian Open semi-final this year, completely outclassed Kerber and after her first WTA title last week in Nuremberg she is flying high.
"I feel like since the beginning of the year I have been improving my game, since Australia I'm at a different level from there," Bouchard said.
"I have confidence in myself. I can play like this and play even better."
She will face 14th seed Carla Suarez Navarro after the Spaniard used her one-handed backhand to great effect to end the run of up-and-coming Croatian Ajla Tomljanovic with a 6-3 6-3 win.
Sharapova was outfought by Stosur in the first set but the Russian began to build a remorseless momentum to grind down Stosur with her accuracy from the baseline.
Sharapova broke for a 5-4 lead to take the second set and raced to a 3-0 lead in the third after winning 17 of 20 points. Stosur never recovered.