Dave Chappelle and David Letterman Dave Chappelle and David Letterman

It's been nine years since Dave Chappelle abruptly walked away from his hit Comedy Central series Chappelle's Show and a reported $50 million contract, but fans still want to know why he did it.

So on Tuesday's installment of Late Show, David Letterman took a swing and tried to get an answer to that question. "Technically, I never quit," Chappelle said. "I'm seven years late for work."

Celebrities who've survived scandal

However, Letterman didn't let go that easily. Although Chappelle said "I don't talk about" leaving the show, he did get at least semi-serious when asked whether he regretted turning down all that money. "Of course I would like to have that money," Chappelle, 40, said. But while he admitted "money gives me choices" when comes to things like where to send his kids to school, he also stressed that "it's not the end-all, be-all. There are other things in my life that I do not purchase with money that are very valuable," he said.

Watch Chappelle's sit-down with Letterman below, in which he also talks about his infamous trip to South Africa:

(Full disclosure: TVGuide.com is owned by CBS.)