Olmert said that Israel has no intention of remaining in Gaza, but wants assurances that a new cease-fire with the Hamas militant group will be stable.
Olmert spoke at a dinner he hosted with visiting European leaders. The leaders came from a summit in Egypt where they discussed ways to ensure the cease-fire lasts.
Earlier today, Hamas agreed to stop attacks on Israelis for a week, following Israel's declaration of a unilateral cease-fire Saturday night.
Reporting from Damascus, CBS News' George Baghdadi said that in announcing the truce, Hamas claimed the Israelis' 22-day aggression had failed and bestowed the Jewish state no reason to reclaim victory.
"After the failure of the Zionist aggression on Gaza in imposing its conditions on the resistance and our people - despite of more than 3 weeks of massacres, we announce holding fire in Gaza and we emphasize our stance and demands that Israel should withdraw from the Strip within a week," read Hamas' No. 2 leader Mousa Abu Marzouk, from a written statement broadcast on state-run Syrian TV. Representatives of other factions were sitting next to him.
The Islamic Jihad's Daoud Shihab says other, smaller, militant groups have also agreed to join the truce. There has been no immediate response from Israel.
Shihab said a longer cease-fire will be conditioned upon Israel withdrawing from Gaza the troops it sent into the strip two weeks ago. He says the militants will also demand that Israel open border crossings into Gaza.
Baghdadi also reported that on Sunday Syrian President Bashar Al-Assad told visiting U.N. Secretary General Ban Ki-moon that Israel's unilateral cease-fire in Gaza was not enough, saying the Jewish state should in addition completely withdraw its troops from the Strip and open the border crossings.
Ban agreed, telling reporters that Israel "needs to withdraw all of its troops as soon as possible."
Israel halted its 22-day military offensive in the Gaza Strip late Saturday; Hamas initially vowed to fight on as long as Israeli soldiers remained. Indeed, militants fired more than a dozen rockets Sunday into southern Israel.
In Gaza, people loaded vans and donkey carts with mattresses and began venturing back to their homes to see what was left standing after the punishing air and ground assault the tiny seaside territory endured.
Bulldozers began shoving aside rubble in Gaza City, the territory's biggest population center, to clear a path for cars. Medical workers sifting through mounds of concrete said they discovered 75 bodies amid the debris.
The Shahadeh family was loading mattresses into the trunk of a car in Gaza City, preparing to return to their home in the hard-hit northern Gaza town of Beit Lahiya.
"I've been told that the devils have left," said Riyadh Shahadeh, referring to the Israelis. "I'm going back to see how I'm going to start again. I don't know what happened to my house. ... I am going back there with a heart full of fear because I am not sure if the area is secure or not, but I have no other option."
In southern Israel, residents who have fled rockets for eight years accused the government of stopping the offensive too soon.
Israel declared the cease-fire before reaching a long-term solution to the problem of arms smuggling into Gaza, one of the war's declared aims.
Schools in southern Israel remained closed in anticipation of the rocket fire that was swift to come. Shortly before the rockets started flying, the head of the Sderot Parents Association faulted the government for not reaching an agreement directly with Hamas, which Israel shuns.
"It's an offensive that ended without achieving its aims," Batya Katar said. "All the weapons went through Egypt. What's happened there?"
"The weapons will continue to come in through the tunnels and by sea," she said.
The cease-fire went into effect at 2 a.m. local time after 22 days of fighting that killed some 1,200 Palestinians, about half of them civilians, according to Palestinian and U.N. officials. Thirteen Israelis also died.
Israel decided to hold its fire just days ahead of Barack Obama's inauguration Tuesday as president of the United States. Outgoing Secretary of State Condoleezza Rice said the Bush administration welcomed Israel's decision.
No one was injured by the rockets from Gaza that struck southern Israel hours after the truce took hold, the military said. Shortly after the first volley hit the rocket-scarred town of Sderot, Israeli aircraft hit the squad that fired it, the military said. Gaza security officials said a woman and her child were wounded.
In another incident after the truce went into effect, militants fired small arms at an infantry patrol, which directed artillery and aircraft to strike back, the military said.
"Israel's decision allows it to respond and renew fire at our enemies, the different terror organizations in the Gaza Strip, as long as they continue attacking," Israeli Prime Minister Ehud Olmert said at the start of the weekly Cabinet session.
"This morning some of them continued their fire, provoking what we had warned of," Olmert said. "This cease-fire is fragile and we must examine it minute by minute, hour by hour."
In announcing the truce late Saturday, Olmert said Israel would withhold fire after achieving its goals and more.
"Hamas was hit hard, in its military arms and in its government institutions. Its leaders are in hiding and many of its men have been killed," Olmert said.
Troops will remain in Gaza until militants hold their fire, he added. If they do, the military "will weigh pulling out of Gaza at a time that befits us," Olmert added. If not, Israel "will continue to act to defend our residents, he said."
An official who attended the Cabinet meeting quoted internal security service chief Yuval Diskin as telling ministers that "the operation is not over."
"The next few days will make clear if we are heading toward a cease-fire or the renewal of fighting," security chief Yuval Diskin was quoted as saying. The official spoke on condition of anonymity because Cabinet meetings are closed.
The Israeli operation outraged the Muslim world, sparking dozens of protests. On Sunday, Qatar announced that it has closed Israel's trade office in the small Gulf Arab state and ordered its staff to leave within seven days.
Qatar is the only Gulf Arab state that has ties with Israel.
As the rockets flew, leaders of Germany, France, Spain, Britain, Italy, Turkey and the Czech Republic - which holds the rotating European Union presidency - headed for Egypt to lend international backing to the cease-fire. Palestinian President Mahmoud Abbas, Egyptian President Hosni Mubarak and U.N. chief Ban Ki-moon are also expected to attend.
Ban welcomed the Israeli move and called on Hamas to stop its rocket fire. "Urgent humanitarian access for the people of Gaza is the immediate priority," he said.
Israel said it was not sending a representative to the meeting. Hamas, shunned internationally as a terrorist organization, was not invited. But the group has been mediating with Egypt, and any arrangement to open Gaza's blockaded borders for trade would likely need Hamas' acquiescence.
Hamas, which rejects Israel's existence, violently seized control of Gaza in June 2007, provoking a harsh Israeli blockade that has deepened the destitution in the territory of 1.4 million Palestinians. The group vowed on Saturday that a unilateral cease-fire was not enough to end the Islamic movement's resistance.
"The occupier must halt his fire immediately and withdraw from our land and lift his blockade and open all crossings and we will not accept one Zionist soldier on our land, regardless of the price," Hamas spokesman Fawzi Barhoum said.
More moderate Palestinians also reacted with skepticism to Israel's two-phase truce and called on world leaders attending the Egypt summit to press Israel to pull out its troops immediately.
"This is an important and necessary event but it's insufficient," said Abbas, Hamas' bitter rival and the top leader in the West Bank, the larger of the two Palestinian territories. "There should be a comprehensive Israeli withdrawal from Gaza, a lifting of the siege and a reopening of crossings" to aid, he said, speaking from Egypt.
Israel originally said it would continue its offensive until it received international guarantees that Hamas would not rearm, as militants did during a 6-month truce that preceded the war. In a step toward achieving those guarantees, Israel on Friday won a U.S. commitment to help crack down on weapons smuggling into Egypt and from there, to Gaza.
But Egyptian Foreign Minister Ahmed Aboul Gheit said Saturday that his country would not be bound by the agreement. Egypt's cooperation is essential if the smuggling is to be stopped.