Student Well-Being

Teen Girls Are Being Victimized by AI-Generated Nude Images

By The Associated Press — December 04, 2023 5 min read
Dorota Mani sits for an interview in her office in Jersey City, N.J., on Nov. 8, 2023. Mani is the parent of a 14-year-old New Jersey student victimized by an AI-generated deepfake image.
  • Save to favorites
  • Print

A mother and her 14-year-old daughter are advocating for better protections for victims after AI-generated nude images of the teen and other female classmates were circulated at a high school in New Jersey.

Meanwhile, on the other side of the country, officials are investigating an incident involving a teenage boy who allegedly used artificial intelligence to create and distribute similar images of other students – also teen girls - that attend a high school in suburban Seattle, Washington.

The disturbing cases have put a spotlight yet again on explicit AI-generated material that overwhelmingly harms women and children and is booming online at an unprecedented rate. According to an analysis by independent researcher Genevieve Oh that was shared with The Associated Press, more than 143,000 new deepfake videos were posted online this year, which surpasses every other year combined.

Desperate for solutions, affected families are pushing lawmakers to implement robust safeguards for victims whose images are manipulated using new AI models, or the plethora of apps and websites that openly advertise their services. Advocates and some legal experts are also calling for federal regulation that can provide uniform protections across the country and send a strong message to current and would-be perpetrators.

“We’re fighting for our children,” said Dorota Mani, whose daughter was one of the victims in Westfield, a New Jersey suburb outside of New York City. “They are not Republicans, and they are not Democrats. They don’t care. They just want to be loved, and they want to be safe.”

The problem with deepfakes isn’t new, but experts say it’s getting worse as the technology to produce it becomes more available and easier to use. Researchers have been sounding the alarm this year on the explosion of AI-generated child sexual abuse material using depictions of real victims or virtual characters. In June, the FBI warned it was continuing to receive reports from victims, both minors and adults, whose photos or videos were used to create explicit content that was shared online.

State laws target deepfakes

Several states have passed their own laws over the years to try to combat the problem, but they vary in scope. Texas, Minnesota and New York passed legislation this year criminalizing nonconsensual deepfake porn, joining Virginia, Georgia and Hawaii who already had laws on the books. Some states, like California and Illinois, have only given victims the ability to sue perpetrators for damages in civil court, which New York and Minnesota also allow.

See Also

Kids Coding In School

A few other states are considering their own legislation, including New Jersey, where a bill is currently in the works to ban deepfake porn and impose penalties — either jail time, a fine or both — on those who spread it.

State Sen. Kristin Corrado, a Republican who introduced the legislation earlier this year, said she decided to get involved after reading an article about people trying to evade revenge porn laws by using their former partner’s image to generate deepfake porn.

“We just had a feeling that an incident was going to happen,” Corrado said.

The bill has languished for a few months, but there’s a good chance it might pass, she said, especially with the spotlight that’s been put on the issue because of Westfield.

The Westfield event took place this summer and was brought to the attention of the high school on Oct. 20, Westfield High School spokesperson Mary Ann McGann said in a statement. McGann did not provide details on how the AI-generated images were spread, but Mani, the mother of one of the girls, said she received a call from the school informing her nude pictures were created using the faces of some female students and then circulated among a group of friends on the social media app Snapchat.

The school hasn’t confirmed any disciplinary actions, citing confidentiality on matters involving students. Westfield police and the Union County Prosecutor’s office, who were both notified, did not reply to requests for comment.

Details haven’t emerged about the incident in Washington state, which happened in October and is under investigation by police. Paula Schwan, the chief of the Issaquah Police Department, said they have obtained multiple search warrants and noted the information they have might be “subject to change” as the probe continues. When reached for comment, the Issaquah School District said it could not discuss the specifics because of the investigation, but said any form of bullying, harassment, or mistreatment among students is “entirely unacceptable.”

If officials move to prosecute the incident in New Jersey, current state law prohibiting the sexual exploitation of minors might already apply, said Mary Anne Franks, a law professor at George Washington University who leads Cyber Civil Rights Initiative, an organization aiming to combat online abuses. But those protections don’t extend to adults who might find themselves in a similar scenario, she said.

Protecting AI victims

The best fix, Franks said, would come from a federal law that can provide consistent protections nationwide and penalize dubious organizations profiting from products and apps that easily allow anyone to make deepfakes. She said that might also send a strong signal to minors who might create images of other kids impulsively.

President Joe Biden signed an executive order in October that, among other things, called for barring the use of generative AI to produce child sexual abuse material or non-consensual “intimate imagery of real individuals.” The order also directs the federal government to issue guidance to label and watermark AI-generated content to help differentiate between authentic and material made by software.

Citing the Westfield incident, U.S. Rep. Tom Kean, Jr., a Republican who represents the town, introduced a bill on Monday that would require developers to put disclosures on AI-generated content. Among other efforts, another federal bill introduced by U.S. Rep. Joe Morelle, a New York Democrat, would make it illegal to share deepfake porn images online. But it hasn’t advanced for months due to congressional gridlock.

Some argue for caution — including the American Civil Liberties Union, the Electronic Frontier Foundation and The Media Coalition, an organization that works for trade groups representing publishers, movie studios and others — saying that careful consideration is needed to avoid proposals that may run afoul of the First Amendment.

“Some concerns about abusive deepfakes can be addressed under existing cyber harassment” laws, said Joe Johnson, an attorney for ACLU of New Jersey. “Whether federal or state, there must be substantial conversation and stakeholder input to ensure any bill is not overbroad and addresses the stated problem.”

Mani said her daughter has created a website and set up a charity aiming to help AI victims. The two have also been in talks with state lawmakers pushing the New Jersey bill and are planning a trip to Washington to advocate for more protections.

“Not every child, boy or girl, will have the support system to deal with this issue,” Mani said. “And they might not see the light at the end of the tunnel.”

Copyright 2023 The Associated Press. All rights reserved. This material may not be published, broadcast, rewritten or redistributed without permission.


Student Well-Being K-12 Essentials Forum Boosting Student and Staff Mental Health: What Schools Can Do
Join this free virtual event based on recent reporting on student and staff mental health challenges and how schools have responded.
This content is provided by our sponsor. It is not written by and does not necessarily reflect the views of Education Week's editorial staff.
Curriculum Webinar
Practical Methods for Integrating Computer Science into Core Curriculum
Dive into insights on integrating computer science into core curricula with expert tips and practical strategies to empower students at every grade level.
Content provided by

EdWeek Top School Jobs

Teacher Jobs
Search over ten thousand teaching jobs nationwide — elementary, middle, high school and more.
View Jobs
Principal Jobs
Find hundreds of jobs for principals, assistant principals, and other school leadership roles.
View Jobs
Administrator Jobs
Over a thousand district-level jobs: superintendents, directors, more.
View Jobs
Support Staff Jobs
Search thousands of jobs, from paraprofessionals to counselors and more.
View Jobs

Read Next

Student Well-Being Kids Are Getting Priced Out of Youth Sports. How Schools Can Help
The rising costs of participating in organized youth sports is making these activities unaffordable for many families.
3 min read
Graham Bacigalupi, of Team Louisiana, watches from the dugout during the DYB, formerly Dixie Youth Baseball, Little League tournament in Ruston, La., on Aug. 8, 2023.
Graham Bacigalupi, of Team Louisiana, watches from the dugout during a Little League tournament in Ruston, La., on Aug. 8, 2023.
Gerald Herbert/AP
Student Well-Being All Public Schools Can Now Get Free COVID-19 Tests. Here's How
The Education Department is encouraging schools to share the COVID-19 tests with staff, families, and the broader community.
2 min read
COVID-19 antigen home tests indicating a positive result are photographed in New York on April 5, 2023.
COVID-19 antigen home tests indicating a positive result are photographed in New York on April 5, 2023.
Patrick Sison/AP
Student Well-Being Teletherapy Challenges: Schools Share How to Navigate Mental Health Support
Schools need to invest in the right ingredients to make their teletherapy sessions useful for students.
7 min read
A tele-therapy session at Kershaw County School District in Kershaw County, South Carolina.
A teletherapy session at the Kershaw County school district in South Carolina.
Courtesy of Presence
Student Well-Being Q&A What Is Driving Youth Mental Health Problems? It's Not Just About Social Media
Focusing too much on one potential cause of the youth mental health crisis could backfire.
3 min read
Teenage girl looking at smart phone