SALT LAKE CITY (AP) — A federal appeals court has upheld Utah's ban on polygamy, dealing a legal blow to TV's "Sister Wives" in a ruling that sides with prosecutors who said they need the ban to go after polygamists like imprisoned leader Warren Jeffs.

While some said Monday's decision could help in the prosecution of crimes like underage marriage and sexual assault, others worried it could send otherwise law-abiding plural families back into hiding and make such investigations harder.

The 10th U.S. Circuit Court of Appeals dismissed a 2013 ruling that struck down key parts of Utah's law against bigamy, or holding multiple marriage licenses. U.S. District Judge Clark Waddoups had found the state violated polygamists' right to privacy and religious freedom.

The three-judge panel ruled that Waddoups should not have considered a lawsuit from Kody Brown and his four wives because they were never charged with a crime and there was little chance they would be.

The decision brings back a rule forbidding married people from living with a second purported "spouse," making Utah's law stricter than those in every other state and creating a threat of arrest for plural families. Like most polygamous families in Utah, Brown is legally married to one wife and "spiritually married" to the others.

State authorities have long said they don't go after polygamists who otherwise follow the law, and they reiterated that stance after the ruling. Officials wanted the clause kept on the books to help in criminal cases related to polygamy.

"We want them to come out of any shadows to report crimes," Utah Attorney General Sean Reyes said.

Prosecutors pointed to Jeffs, the polygamous sect leader convicted of assaulting girls he considered wives. Authorities have recently cracked down on his sect on the Utah-Arizona border, charging leaders in a multimillion-dollar food stamp fraud scheme. A jury in Phoenix also found the towns denied nonbelievers basic services such as police protection.

The Browns have never belonged to Jeffs' group, and lumping in good, honest polygamist families with him isn't fair, said Brady Williams, who has five wives.

"We're only guilty of trying to love a different way than the norm," he said. "They are marginalizing a minority class in the United States. That's unconstitutional."

The Browns will appeal the ruling, either asking the 10th Circuit to reconsider or taking the case to the U.S. Supreme Court, said their lawyer, Jonathan Turley.

"The underlying rights of religious freedom and free speech are certainly too great to abandon," Turley said in a statement.

The family has said other laws exist to target crimes linked to plural marriages and that banning the practice can sow distrust of authority. They argue their show is evidence that polygamous marriages can be just as healthy and monogamous ones.

Kristyn Decker, a former polygamous wife, disagrees. She says that people don't report problems amid pressure to avoid casting the practice in a bad light.

"We have all been told over and over again, 'We need to protect the principle of plural marriage at any cost,'" said Decker, who argued that the ban lets authorities get a foot in the door.

The state said the appeals court made the right call.

"I don't think the Browns had a legitimate fear of prosecution," said Parker Douglas, who argued the state's case.

There are about 30,000 polygamists in Utah, according to court documents. They believe polygamy brings exaltation in heaven — a legacy of the early Mormon church. The mainstream Church of Jesus Christ of Latter-day Saints abandoned the practice in 1890 and strictly prohibits it today.

___

Associated Press writer Hallie Golden contributed to this story.