American Idol castoff Scott Savol is not just a singer of questionable talent who's sparked heated debate among Idol addicts. He's also a man of few words. Very few words.

Asked how he feels about his love/hate relationship with the audience, Savol simply tells "That remains true in life. There were people that hated me and loved me before the show, and now there are just more of them."

Sure, but the knowledge that scores of Americans were rooting for his downfall must have added stress to his performances. "I knew people didn't like me," he sighs. "There are a lot of people that didn't like all of us. If we went with that and performed with that in mind, we all would have had bad performances and just canceled the show [last Wednesday] night."

Much of viewers' distaste for Savol has stemmed from the publicity surrounding his violent past. (He'd previously been arrested for throwing a phone at his newborn baby's mama.) However, he feels the miniscandal was not significant in his ouster.

"I doubt it [is related] because I am not the only one who had any of that [kind of thing] come out," he says, undoubtedly referring to fellow contestant Bo Bice's drug past. "Mine was just blown up more. I think it was because somebody really wanted me to get kicked off the show. About a month ago is when it came out, and it didn't really affect me."

Furthermore, Savol is unapologetic about sassing Simon when the Brit bluntly shot him negative reviews. "In order for me to be who Scott is," he says, "I have to be who Scott is all the time and not just when it looks good."

While Fox publicists prefer to keep the Corey Clark scandal off limits, Savol addresses it without naming names. "Whatever came out in the press as far as any controversy," he says, "the person knows that we are part of the No. 1 show. So for them to come out with allegations of something that has happened between them and a judge, [it's] just so they can get [leverage] to put out a mediocre product, like a book or an album. The best way to get into the limelight is to stir up controversy."

Lastly, we noticed the contestants handing flowers to Paula Abdul on Wednesday night. Could that have had something to do with the aforementioned "controversy"? "I dunno," Savol shrugs. "I think they had some extra flowers in the back. They were a couple of days old."

The sassy hostesses of TV Guide Channel's Idol Chat dish all things American Idol every Thursday at 8 pm/ET. Click here to watch an exclusive video clip.