Also overnight, suspected Taliban militants attacked several police posts in eastern Paktia Province, leaving at least three police dead, according to Gul Agha Roohani, the province's police chief.
The fall of Khawaja Omari to the Taliban extends their territory closer to the provincial capital, also called Ghazni, a city of 150,000 people 150 km (95 miles) southwest of the Afghan capital, Kabul.
He said the Taliban forces set buildings on fire but left the area by daylight.
Taliban guerrillas attacked a district headquarters in Afghanistan's Ghazni province on Thursday, killing its administrator along with at least 14 others, officials said.
Dozens of people, many of them children, were killed or wounded after Afghan Air Force helicopters struck a madrassa in Dashte Archi district, security sources and witnesses told AFP.
President Ashraf Ghani condemned the attack and said it was carried out by "Afghanistan's enemy", a reference to the Taliban. Only three rebels were killed and four others wounded, he added.
"All parties to the conflict in Afghanistan must do everything in their power to protect civilians from harm", said Ingrid Hayden, the secretary-general's deputy special representative for Afghanistan.
Mohammad Arif Noori, a spokesman for the governor of Ghazni Province, said that Taliban insurgents attacked the government compound of Khwaja Umari district.
He said eight government security forces were wounded in the fighting.
The district governor, Shukrullah Shaker, said a roadside mine and a rocket hit a auto carrying police reinforcements sent to the scene, killing the police commander.