CNN: TOKYO, Japan -- Japanese authorities have released an American man who was jailed for allegedly trying to snatch back his children from his estranged wife.

Police in the rural southern town of Yanagawa let Christopher Savoie go Thursday without indicting him on charges of child kidnapping.

Officials said the indictment was "on hold," but did not elaborate.

The prosecutor's office in nearby Fukuoka said Savoie was released after he promised not to take his children back to the United States "in this manner," implying he could not have any contact with the children.

Savoie, 38, a Tennessee native and naturalized Japanese citizen, allegedly grabbed his children -- 8-year-old Isaac and 6-year-old Rebecca -- as his estranged wife walked them to school on September 28 in Yanagawa.

With the children, Savoie headed for the nearest U.S. consulate, in the city of Fukuoka, to try to obtain passports for them. Screaming at guards to let him in the compound, Savoie was steps from the front gate but still standing on Japanese soil when he was arrested.

Consulate spokeswoman Tracy Taylor said her office had been notified that Savoie would be released Thursday afternoon.

"We are pleased to learn he was to be released," she said. "The U.S. government, together with the Japanese government, will try to find a long-term solution to the joint custody and parenting issues."

Savoie and his first wife, Noriko Savoie, were married for 14 years before a bitter divorce in January. The couple lived in Japan but had moved to the United States before the divorce.

Christopher Savoie remarried in February.

Noriko Savoie was given custody of the children and agreed to remain in the United States. Christopher Savoie had visitation rights.

On the day that the children were to start school in August, Savoie learned that Noriko Savoie had fled with them to Japan.

After they went to Japan, Christopher Savoie filed for and received full custody of the children. Police in Franklin, Tennessee, issued an arrest warrant for Noriko Savoie. However, Japan is not a party to a 1980 Hague Convention law on international child abduction.

Japanese law also follows a tradition of sole-custody divorces. When a couple splits, one parent typically makes a complete and lifelong break from the children.

Complicating the matter is the fact that the couple is still considered married in Japan because they never divorced here, police said. In addition, Japanese authorities say, the children hold Japanese passports.

Foreign parents have had little luck in regaining custody, the U.S. State Department said.

Noriko Savoie's mother did not want to talk about Christopher Savoie's release on Thursday.

"I cannot make any comment on this, for the sake of my grandchildren," she said.

Upon hearing news of her husband's release, Amy Savoie told CNN, "The elation lasted for a few minutes and now we're back to square one, where there's no closure. We don't have Isaac and Rebecca in a situation from which they were taken. Isaac and Rebecca had a very, very happy situation here in Tennessee. ... They have people who love them here."

She said she did not believe the parties could work out an agreeable arrangement regarding the children. "There are two parents who love these children and one of them has just been -- just cast aside."

She said other parents in similar situations have contacted her. "I think the focus for us will be, you know, we'll become spokespersons, I suppose, for so many of these families."

Yasuhisa Kawamura, a spokesman with the Foreign Ministry, said the government may reconsider signing on to the Hague treaty.

"Japanese government is also considering seriously to conclude this treaty on the grounds that this treaty would provide one of the most effective measures to protect the children after their parents divorced," he said.

Read more stories moms are talking about.