Tripoli, Libya (CNN) -- Libyan Prime Minister Ali Zeidan appealed for calm Thursday in his first public comments since he was freed after being abducted by militia gunmen for several hours.
In remarks to a Cabinet meeting broadcast on Libyan state TV, Zeidan said he did not want to see the situation escalate and urged Libyans to show "wisdom."
Zeidan's abduction from a luxury hotel early Thursday highlighted the security threat posed by militias that have run rampant in Libya since the revolution that ousted Moammar Gadhafi two years ago.
"These types of groups exploit these types of gaps that exist in the country at the moment, given the fact that we are in the process of building our criminal justice system," Libyan Foreign Minister Mohammed Abdelaziz told CNN's Christiane Amanpour.
But the prime minister appeared to play down his kidnapping, characterizing it as an internal political problem.
Libya: Birth pains of a new nation
Libya: Terror arrest an 'abduction'
U.S. captures Al Qaeda suspect in Libya
"I want to reassure the foreigners inside Libya that this issue happened within the context of political Libyan disagreements and the foreigners are not being targeted," he said.
Immediately after Zeidan's release, Nouri Abusahmain, president of the Libyan General National Congress, said the prime minister was "in good shape" and "in good spirits" as he headed to his office.
His abduction prompted a slew of confusing reports.
Gunmen move in before dawn
The gunmen took Zeidan before dawn from the Corinthia Hotel in Tripoli and put him into a convoy of waiting cars, according to a hotel clerk who was not authorized to speak to the media.
The five-star hotel that Zeidan calls home is popular among government officials, some of whom reside there, including the justice minister.
The witness reported no gunfire during the incident and said the gunmen were respectful and "caused no trouble."
Zeidan's office initially called the abduction a "rumor" on its official Facebook page, but later posted an update that it was "coerced by kidnappers to deny the report."
His spokesman told CNN that the prime minister was kidnapped.
But the Operations Room of Libya's Revolutionaries, the militia that took him, said it merely detained him over financial and administrative corruption charges.
However, the Justice Ministry said there was no arrest warrant for Zeidan, calling the move a kidnapping. Abusahmain said the government was not aware of the charges and the prime minister was prepared to answer any questions.
The militia works with the Interior Ministry -- a not altogether uncommon practice in Libya, which has tried unsuccessfully to rein in the many militia groups. Instead, various ministries have teamed up with them for their own needs, including providing security services.
U.S. Secretary of State John Kerry, on a visit to Malaysia, said he had spoken with U.S. Ambassador to Libya Deborah Jones and Washington as the situation in Libya evolved, and would stay in close touch.
"Our embassy personnel are secure. We're confident about our abilities to keep them in that security," Kerry said.
The situation underscores "something that we've been really focused on in these last months, which is building capacity in Libya," he added. It has been just over a year since U.S. envoy Christopher Stevens was killed in an attack on the U.S. Consulate in Benghazi, Libya.
British Foreign Secretary William Hague welcomed Zeidan's release. "We will work with Libyan government on ensuring the transition remains on track and insecurity is addressed," he tweeted.
Libya questions U.S. ambassador over al Libi capture
Militias demand more autonomy
In the eastern part of the country, militias are demanding more autonomy from the central government and have severely constrained Libya's oil output, which is central to its export revenue.
Last month, Zeidan said the nation is trying to rebuild after decades under Gadhafi's rule. He shot down reports that Libya is a failed state.
"We are trying to create a state, and we are not ashamed of that," he told CNN's Amanpour. "The outside world believes that Libya is failing, but Libya was destroyed by Gadhafi for 42 years, and was destroyed by a full year of civil war. And that's why we are trying to rebuild it."
Part of Libya's rebuilding involves reconciliation and accountability, Zeidan said last month. Libya has an interim president, but the prime minister holds all executive powers.
Security an issue
Armed militias have roamed the country largely unchecked since the 2011 ouster of Gadhafi.
"We are starting from scratch, and I think it will take time," said Abdelaziz, the foreign minister. "As long as we have a sustained support on the part of the international community, I think Libya will make it."
Rights groups have said security remains a principal concern in Libya.
"The main problem affecting both justice and security is that armed militias still maintain the upper hand," Human Rights Watch said in a report last month. "They have various agendas -- financial, territorial, political, religious -- and operate with impunity two years after the Gadhafi regime ended. Successive interim governments have failed to assert control over these militias, preferring to contract them as parallel forces to the army and police."
In a statement issued Thursday, the rights group said it was "deeply troubled" by Zeidan's detention, which it described as a clear violation of Libyan law.
"The incident highlights the grave security conditions in Libya today. Hopefully it will refocus attention on the urgent need to strengthen both the country's security forces and its judicial systems," it said.
Recent attacks have added to the uncertainty in Libya.
Gangs of armed men have surrounded key ministries, including the Justice Ministry, trying to force out members of the democratically elected government.
Libyan Justice Minister Salah Marghani was forced to evacuate after armed militias surrounded his ministry in April.
Libyan intelligence services have warned that the country is becoming a haven for al Qaeda to regroup and regenerate itself.
Numerous weapons left over after Gadhafi's downfall are providing groups with different motivations to form their own militias, government officials said.
On Saturday, U.S. forces swooped into Tripoli and seized a Libyan national indicted in the 1998 American Embassy bombings in East Africa.
Abu Anas al-Libi is a suspect in the embassy attacks in Kenya and Tanzania, and American officials have described him as "one of the world's most wanted terrorists."
The militia group said the prime minister's capture had nothing to do with the arrest of al-Libi. However, three days ago, it issued a warning that whoever collaborated with "foreign intelligence services" in the arrest of the terror suspect would be punished.
It's unclear if Zeidan played a role in the arrest of al-Libi by U.S. forces.
But the Libyan government has decried the arrest of the terror suspect, and its national congress Tuesday demanded that the United States hand him over.
The foreign minister, Abdelaziz, denied that Zeidan's abduction had anything to do with the American operation.
"I don't really see the relevance of this abduction to that what happened," he said, "because the debate in relation to the disagreements between some of the military groups and the government officials -- it was there for some time."
CNN's Nic Robertson reported from Tripoli; Saad Abedine and Faith Karimi reported from Atlanta. CNN's Mick Krever, Victoria Eastwood, Yousuf Basil and Jomana Karadsheh also contributed to this report from Tripoli.