"The persistent media speculation regarding allegations dating back to my years at USC has been both painful and distracting," Bush said in a statement on his website. "For the rest of my days, I will continue to strive to demonstrate through my actions and words that I was deserving of the confidence placed in me by the Heisman Trophy Trust."
Bush said he did not want the media coverage or the allegations against him to affect the integrity of the award.
"While this decision is heart-breaking, I find solace in knowing that the award was made possible by the support and love of so many," he said.
The 25-year-old two-time All-American received the Heisman trophy in 2005 after he helped lead his college team, the University of Southern California, to a national championship in 2004.
After receiving the Heisman, he was the No. 2 overall draft pick in 2006, when he was signed to the New Orleans Saints. Bush helped the Saints clinch the Super Bowl championship last February.
Besides his time on the gridiron, Bush is known for going out with Kardashian. The two dated from mid-2007 to July 2009, before reuniting two months later. They split again in March.
Bush's announcement comes months after the NCAA deemed Bush ineligible for the 2005 season after it was determined that Bush had received improper benefits during his time at USC. The university was stripped of all 14 wins during the 2004-2005 season, and 30 football scholarships.
USC has already returned its copy of Bush's Heisman trophy. It's unknown whether the trophy will be re-awarded to another player from that year.