DHAKA, Bangladesh--Factory owners accused of forcing staff back to work in a building shortly before it collapsed and killed about 400 people last week were under financial pressure because political unrest had scared off investors, company executives said.

The companies lost orders to rival garment makers in other countries after protests in Bangladesh since February led to strikes and port blockades ahead of elections next year.

The $20 billion-a-year garment industry had lost $500 million in export orders due to the earlier turmoil, the Bangladesh Garment Manufacturers and Exporters Association said this month. Western buyers had shifted many of those orders to India, it said, warning that some of the three million jobs in the sector were at risk if instability continued.

Workers for all five factories in the building said in interviews they were urged by factory managers Wednesday morning to return to work at Rana Plaza, in Savar, a commercial hub north of Dhaka, despite warnings from engineers that an exterior crack on the building made it unsafe. It collapsed hours later.

Just before last week's disaster, two of the factories--Phantom Apparels Ltd. and Phantom Tac Ltd.--were rushing to complete an order from Spanish fast-fashion retailer Mango, said factory executives who survived the building collapse.

Those two factories, owned by Aminul Islam, a Dhaka-based businessman, and the other factories had been unable to fill orders due to the chaos, said the factory executives and trade association members. When Mr. Islam won a contract with Mango, he was eager to make it work, the executives said.

"The owner told us that the company had lost $121,000 worth of orders and Mango, being such a large buyer, was crucial for us," said one Phantom executive. "Workers were told not to miss a single day of work since we were losing so much time due to political unrest."

Mr. Islam and the owners of the other factories turned themselves in to police over the weekend after authorities ordered their arrest for criminal negligence. The building owner is also in police custody. The men in custody haven't yet been charged. Lawyers for the men couldn't be reached or declined to comment.

Sayed Shah Jamal, Phantom Apparels' quality-assurance manager, who was rescued from the wreckage, said of the work for the Mango contract, "The fabric had just come in, and we were all working very hard to ship the order before the June deadline."

Documents recovered from the factory site by labor activists and reviewed by The Wall Street Journal show one order for 35,000 units of boys' polo shirts and girls' leggings for Mango's autumn-winter collection.

Phantom Apparels and Phantom Tac, with a total of 10 production lines between them with 60 seamstresses on each line, were rushing to complete the order, the executives said.

The factories, which also supplied clothes to other European retailers, had been audited by Mango executives in October to determine safety standards and work conditions, one factory executive said.

"The audit team found some problems and were due to arrive this month for a follow-up audit," the executive said. "They canceled the visit due to general strikes in the country."

Mango contradicted the Phantom executives, saying it was in talks with the factory to place a test order but that it hadn't yet authorized it to do the work and hadn't yet begun an audit. On Monday, Mango stood by its earlier statements.

"Even so, notwithstanding the social audit, it would have been impossible to detect the structural defects of the collapsed building," the company said.

Other documents recovered from the building site showed the factory received an order from Italy's Benetton Group SpA through an Indian agent in August 2012. The order was for 45,000 sleeveless shirts and 27,000 full-arm shirts.

Benetton initially denied any connection to the factory. But Monday, after labor groups found Benetton-labeled clothing and production documents in the rubble, the company acknowledged a "one-time, subcontracting order that was completed several weeks ago," a Benetton spokeswoman said.

She said the company had removed the manufacturer from its supplier list after completing the one-time order "as we found that the manufacturer no longer met certain of our standards."

Another factory in the building, New Wave Style Ltd., also had problems with delayed shipments and was facing onerous debt repayments, said Annisul Huq, a former president of the Bangladesh garment manufacturers' association.

The factory, owned by Mahbubur Rahman Tapas and Bazlus Samad Adnan, decided to keep operations going after the warning from building engineers.

"They were under tremendous pressure to meet shipment deadlines," Mr. Huq said.

New Wave couldn't be reached for comment.

Other companies that contracted work in the building included Canadian retailer Loblaw Cos. and Primark, a European budget fashion chain owned by Associated British Foods.

Both companies said Monday they would pay compensation to victims of the building collapse and urged other retailers to follow suit. Primark said payments will be made to injured garment workers, children who have lost parents and other family members of the deceased.

"We are fully aware of our responsibility," Primark said. "We urge these other retailers to come forward and offer assistance."

Authorities said they detained building owner Sohel Rana on Sunday as he was allegedly attempting to flee to India on allegations he built Rana Plaza without acquiring mandatory safety permits.

The death toll from the accident, after rising rapidly in recent days, reached about 400 people as of Monday.

Rescuers fear the toll could still rise as the salvage operation shifts from search and rescue to recovering corpses buried in the wreckage

Tens of thousands of workers have joined the fray in recent days, clashing with police, ransacking garment factories and vandalizing cars. Police have responded with tear gas and rubber bullets.

Subscribe to WSJ: http://online.wsj.com?mod=djnwires