Jimmy Iovine Jimmy Iovine

It's taken five weeks, but the studio audience is finally starting to drop the "S" bomb: Simon. He was gone like a puff of black smoke. And in the hoopla surrounding the two new judges — glamour goddess Jennifer Lopez and rock god Steven Tyler — he was almost forgotten.

But that was before Pia Toscano was booted off the show, making it only to ninth place. And the fans, still touchy on the subject, are now trying to place blame. Eavesdrop on their conversations and here's how the thinking goes: If it wasn't the fault of Toscano herself, with her sometimes stiff performance style, or the fact that girls tend to vote for boys, then it had to be the fault of the super nice, super supportive judges. "Pia was so much better than anyone else out there," said one audience member Wednesday night. "But the judges say everyone is great. To Steven [Tyler], everyone is 'beautiful.'"

And that's why Simon Cowell's name has surfaced again. Where is that scathing criticism, that unapologetic boredom when a contestant fails to set the room on fire? When all the critiques are favorable, how is a viewer supposed to edit his or her choices? "I miss him," sighed another audience member. "These judges are so nice that they're giving absolutely no direction."

Given what happened on the show Wednesday night, that doesn't seem likely to change. "You know, everyone wants us to be tough with you guys," said Lopez to country singer Scotty McCreery, apparently aware of the fingers pointed in her direction. "But the truth is, you're all so damn good."

Then, later on, Lopez completely pulled her punches with Haley Reinhart. "I'm so afraid to say anything because I don't want any of the (two remaining) girls to go home."

So it falls to record producer Jimmy Iovine to play the new heavy. "He totally is," says Idol vocal coach Debra Byrd after the show. "I never thought of it that way, but he is." Trouble is, Iovine isn't sitting at the judges' table, and we don't hear what he really thinks of a performance until a week later.

Subscribe to TV Guide Magazine now!