(CNN) - New York Gov. David Paterson defended himself Monday from rumors of womanizing and drug use, describing these unfounded reports as "outrageous," during an interview with the Associated Press.

"For the last couple of weeks I have been the subject of what, even by Albany standards, has been a spate of outrageous rumors about me," Paterson told the AP.

Despite rumors of drug use and gallivanting, Paterson told the AP he hasn't used an illegal substance since his early 20's and rarely eats dinner outside of his home. He also said he hasn't had sexual relations with another woman in more than 10 years. Both Paterson and his wife have previously admitted to extramarital affairs in their past.

The embattled Democratic governor, who is serving the remainder of former Gov. Elliot Spitzer's term, also criticized some reports in the media about his life as "callous and sleazy."

Paterson specifically pointed his finger at the New York Post, which reported late last month that a state police officer caught the governor involved with a woman who wasn't his wife. Paterson said the room he was supposedly caught in doesn't even exist. The Post stood by its reporting in a statement to CNN.

Paterson also blamed the spate of rumors on an investigation that The New York Times is reportedly working on about Paterson's private life. The governor said that investigation has "spawned a bunch of speculations that are so way out that it's shocking."

"There is an accountability that should exist in the media," he said. "How do I get my reputation back? Because I don't believe I have done anything to deserve this kind of bashing."

Paterson added he is determined to press on with his election campaign despite the rumors, a flood of negative media attention, and polls that show he trails likely Democratic primary opponent and the state Attorney Gen. Andrew Cuomo by as much as 40 points.