Nicole Scherzinger, Steve Jones, Paula Abdul

Steve Jones and Nicole Scherzinger are leaving The X Factor after one season, and Paula Abdul may not be far behind them.

Jones, 34, tweeted Monday afternoon that he would not be returning to the show. Hours later, Fox confirmed to TVGuide.com that Scherzinger would also be exiting. And still later that night, Deadline reported that Abdul had also been cut. Fox did not respond to an e-mail asking about Abdul's status.

Deadline cited series insiders who said that Jones, Scherzinger and Abdul were all axed as part of a major house cleaning at Fox's other singing competition. Earlier in the day, E! sources had said Scherzinger made the decision not to continue because the series took too much time away from her recording career, and that Simon Cowell had given her his blessing.

Steve Jones weighs in on the ups and downs of hosting The X Factor

The shake-up come after months of speculation about whether or not Jones and Scherzinger, specifically, would be back. Fans criticized Jones for his heavy accent, abruptly cutting off the judges and for his stiff on-air presence. Despite early reports prior to the Season 1 finale that he would not be asked back, Jones remained optimistic. "Hopefully I'm involved in the second season! I've had a whale of a time doing the first season. But I'm sure we'll look at the first season as a gigantic pilot that'll become more refined," he told Vulture in December. "But as I said, I've gotta get confirmation of my involvement in it first. We'll see! Fingers crossed!"

Scherzinger, 33, also received negative feedback because of her limited professional experience in comparison to fellow judges Cowell, Abdul and L.A. Reid, as well as her excessively soft and sympathetic critiques. U.K.'s Now Magazine reported in November that Cowell was considering replacing Scherzinger with X Factor U.K. judge, former Destiny's Child singer Kelly Rowland. Scherzinger herself admitted to having a hard time criticizing the contestants. "It just took a lot out of me," Scherzinger told reporters in December. "I'm an artist first, not a judge, so I do wear my heart on my sleeve."

Abdul, 49, who served as a judge on American Idol for eight seasons, was someone Cowell championed for The X Factor from the start. However, she did not sign her contract until the day auditions kicked off in Los Angeles. Abdul was also the first judge to see all of her final acts eliminated from the competition.

The X Factor's Astro and Chris Rene score record deals

It is unclear who the show may tap to fill the empty spots in Season 2. Scherzinger was originally slated to co-host The X Factor with Jones before she was called up to the judges table to fill in for ousted judge Cheryl Cole.

The X Factor will return for Season 2 this fall on Fox.

Do you think producers made the right call? Who do you want to see join The X Factor?