The Battle Creek, Mich.-based company said Thursday that Phelps's behavior - caught on camera and published Sunday - is "not consistent with the image of Kellogg."
The company put Phelps on boxes of its Frosted Flakes and Corn Flakes.
Separately Thursday, Phelps was suspended from competition for three months by USA Swimming, the sport's national governing body.
USA Swimming also cut off its financial support to Phelps for the same three-month period, effective from Thursday.
"This is not a situation where any anti-doping rule was violated, but we decided to send a strong message to Michael because he disappointed so many people, particularly the hundreds of thousands of USA Swimming member kids who look up to him as a role model and a hero," the federation said in a statement.
"Michael has voluntarily accepted this reprimand and has committed to earn back our trust."
Phelps has kept the backing of many sponsors since the photos surfaced that showed him inhaling from a marijuana pipe.
Kellogg says its contract with Phelps expires at the end of the month.
Phelps said Wednesday that he is going to have to live with the fallout from the photo. He spoke to The Associated Press about the ordeal outside the pool where he trains. The public comments were his first since the photo surfaced in a British tabloid over the weekend.
"It's something I am going to have to live with and something I'll have to grow from," Phelps said. "I know with all of the mistakes I made, I learned from them and that is what I expect to do from this. By no means it is fun for me, by no means is it easy."