(CNN) -- Workers at the site of the multi-story building collapse near the Bangladeshi capital on Monday kept up their grim task of removing severely decomposed bodies from the rubble as the death toll climbed above 650.
The building, which housed five factories full of garment workers, caved in on April 24, trapping hundreds of people in a tangled heap of concrete slabs and twisted metal in Savar, a suburb of the capital, Dhaka.
Rescue workers managed to save more than 2,400 people from the mountain of debris during days of dangerous, painstaking work in high temperatures. But they gave up hope of finding any more survivors a week ago and are now using heavy machinery to pick apart the ruins and uncover the bodies buried inside.
The number of people confirmed dead has now reached 657, said Col. Sheikh Zaman, a military official overseeing the recovery operation in Savar.
Missing loved ones
Calls for action after factory collapse
The high cost of cheap clothing
Bangladesh's PM: 'Accidents happen' Amid the stench of death that permeates the surrounding streets, a large number of people continue to wait near the site of the collapse for news of missing relatives. Their gathering point is a school playing field where bodies retrieved from the ruins are taken for initial identification attempts.
The task of identifying is made all the more difficult by the decomposed state of many of the bodies. If authorities cannot identify them through ID they might be carrying or other means, the bodies are taken to a morgue in Dhaka.
Some of the people camped out in the area have also been holding up trucks transporting rubble removed from the disaster to check if they're mistakenly carrying the remains of people, recovery operation officials have said.
Zaman said authorities still don't know exactly how many bodies remain encased in the wreckage because the factory owners still haven't provided a full account of how many people inside when the building gave way.
The recovery effort is expected to continue for several more days.
The owners of the building and the factories are under police investigation over accusations they ordered workers to enter the premises on the day of the collapse despite cracks appearing in the structure the day before.
Heavy machinery blamed
Preliminary results of a government inquiry into the building collapse found that "heavy machinery and high-capacity generators" were "largely responsible," the state-run Bangladesh Sangbad Sangstha News Agency reported last week.
"During the inquiry, we have found that use of substandard materials during the construction also contributed to the building collapse," the leader of the inquiry, Main Uddin Khandaker, told BSS.
Bangladesh's $20 billion garment industry accounts for 77% of the country's exports.
Among those caught up in the finger-pointing after the building collapse disaster are Western retailers and clothing brands that Bangladeshi suppliers say put heavy pressure on prices, resulting in bad pay and conditions for workers.
On Saturday representatives of Bangladesh's government, industry and workers issued a joint statement laying out an "action plan" to improve worker safety in the wake of the disaster.
The European Union has said it is considering trade action against Bangladesh if it doesn't take clear steps to improve the safety conditions of its millions of garment workers.
READ MORE: Prime minister says Bangladesh is reforming its garment industry
Bangladesh vs. the U.S.: How much does it cost to make a denim shirt?
OPINION: The bloodshed behind our cheap clothes
CNN's Saeed Ahmed and journalist Farid Ahmed contributed to this report.