Keegan Allen and Troian Bellisario

For Pretty Little Liars' A-Team member Toby, it's only a matter of episodes before the jig is up.

"The clock is ticking at a fairly quick pace in terms of the revelation of Toby, and the girls are getting closer to uncovering the truth about it," executive producer Marlene King tells TVGuide.com.

On the winter premiere of the ABC Family series, Toby (Keegan Allen) was still deeply involved with head "A" member Mona (Janel Parrish), while keeping up the ruse for his girlfriend Spencer (Troian Bellisario). For viewers, it's seems hard to believe that Toby could be living a double life without having fallen for Spencer in the process, but Allen maintains that, for now, he can understand the reasons behind his character's horrible actions.

Returning shows: Where we left off

"Toby started out as an outcast and took a huge burden on by being a scapegoat for Alison and the girls when they blinded his stepsister," Allen tells us. "He had to go off to school in another county and was excommunicated, while these girls got away with it just flipping their hair and wearing their heels like it's fine. He thought, 'I'll get you back,' and he does in the most extraordinary way."

Viewers will get even more of an understanding of that in an upcoming episode. "We're doing a great flashback with Toby where Alison [Sasha Pieterse] visits Toby in juvie this season," executive producer Oliver Goldstick says. "You'll see that ... he hated those girls and really wanted to make them pay. He's always hovered these feelings, this real disdain for these girls."

Pretty Little Liars executive producer on suspicious Byron and Toby's axe to grind

Since it's only a matter of time before Spencer finds out about Toby, how will she handle the betrayal? "If this really happened to me, I wouldn't move and I couldn't just go back to school, but that's what the girls always have to do," Bellisario says. "But we'll take the time to peel her open and see what she would actually go through and because she takes this journey, she's able to get to a new place and become a new person."

So when she finds out, does she try to slap him like in this Season 2 scene? "There are definitely some physical moments between them," Bellisario adds.

Watch a clip from Tuesday's episode where the girls run into an A-Team-dressed Toby.

Pretty Little Liars airs on Tuesdays at 8/7c on ABC Family.