This blog post is part 1 of a 2-part series. The second part will set out recommendations for companies and policymakers.

Six years ago—one month into a global pandemic—we argued that the automated moderation processes many platforms were rapidly adopting should be highly transparent, easily appealable, and temporary. We warned that "protocols adopted in times of crisis often persist when the crisis is over."

That warning proved prescient. The use of automation and artificial intelligence (AI) to identify, flag, and moderate content has become the new norm—a permanent feature of how platforms govern speech online. In this two part series, we’re take stock of this new norm, and considering what platforms can and should do to ensure that AI serves online expression rather than stifling it.

A brief history of automated content moderation

From spam filtering and keyword blacklists to the hash-matching technologies used to identify child sexual abuse material and terrorist content, automated technologies have been used in commercial content moderation for many years. While these tools have long posed risks to freedom of expression, their use was, for quite some time, relatively limited in scope.

Then, in 2017, a blog post published by Facebook (now Meta) described the company's "fairly recent" use of artificial intelligence to identify, classify, and remove violent extremist content. At the same time, Facebook emphasized caution, noting that it did not want to suggest there was "any easy technical fix."

Just one year later, Mark Zuckerberg appeared before the U.S. Senate's Commerce and Judiciary Committees and disclosed that "99 percent of the ISIS and Al Qaida content" removed by Facebook was flagged by AI "before any human sees it." He also stated that Facebook was "developing A.I. tools that can identify certain classes of bad activity proactively and flag it for our team at Facebook." At the time, we raised concerns about the ethical implications of using AI in this manner.

Then came 2020. The sudden reduction of the human moderation workforce, combined with a dramatic increase in social media use—and with it, a surge in misinformation—created the perfect conditions for platforms to expand their reliance on AI-driven moderation. It quickly became apparent that companies'—and particularly Meta's—approach to moderation during the pandemic represented a backslide in transparency, freedom of expression, and access to remedy. The increased reliance on automation was a significant factor.

The costs and benefits of AI content moderation

We knew in 2020 that the use of AI to moderate content would present problems for online freedom of expression. Today, those problems are well-documented. A 2025 joint declaration by special rapporteurs and representatives of the United Nations (UN), Organization for Security and Co-operation in Europe (OSCE), Organization of American States (OAS), and African Commission on Human and Peoples’ Rights (ACHPR) states:

“The use of AI content moderation can lead to over-removal, discrimination and censorship. Reliance on inherently biased datasets and opaque training processes can amplify pre-existing inequalities, risking homogenisation of expression, and erasure of linguistic and cultural diversity.”

EFF and many of our allies have documented these impacts. For example, our 2019 paper co-authored with Witness and Syrian Archive examined the impact of extremist content regulations—and their implementation through automation and AI—on human rights documentation. A 2020 report from Human Rights Watch highlighted the consequences of these removals, noting: "There is no way of knowing how much potential evidence of serious crimes is disappearing without anyone's knowledge."

The Center for Democracy and Technology's recent series on content moderation in the Global South demonstrates persistent inequities in content moderation of four “low-resource” languages—so-called because the relative scarcity of training data makes it more difficult to develop equitable and accurate AI models for them. 

Content moderation often disproportionately impacts vulnerable and historically marginalized groups, and AI content moderation is no different. GLAAD recognizes the role AI plays in scaling content moderation but notes that “when moderation systems lack nuance, transparency, and human oversight, they can fail to curb harassment and wrongly suppress legitimate LGBTQ content.”

These failures are not incidental. They are a predictable consequence of deploying automated systems to make complex judgments about language, culture, context, and identity at scale.

All of that said, automated content moderation can offer important benefits. The primary one: helping to spare human content moderators who must review content that varies from whimsical to horrific, often for little pay and with devastating mental health consequences. Outsourcing this work to the bots can offer some relief—though it’s worth noting that the humans hired to train the AI models face a similar dynamic.

In addition, AI models could potentially be trained over time to be more precise, accurate, and dynamic, helping to mitigate over-censorship and disinformation. The jury is still out on whether this potential will be realized; what we do know is that new approaches to the persistent problem of over and under-enforcement are desperately needed.

Automated moderation is no longer an experiment

Getting the balance between real costs and potential benefits depends a lot on the details: how automated systems are designed, trained, implemented, and audited.  

Despite advances in the sophistication and scale of automated moderation systems, many of the transparency, accountability, and due process safeguards advocated by civil society, researchers, and human rights experts have yet to be fully realized. At the same time, automated systems have become increasingly central to how platforms enforce their rules and govern online speech.

The question today is not whether companies will use AI to moderate content, but under what conditions they should do so. And now as ever, the answer is not that the public should just trust that platforms’ deployment of increasingly powerful systems will serve, rather than inhibit online expression. In fact, as automated systems become more sophisticated and more deeply embedded in platform governance, the need for transparency and accountability becomes more urgent. 

In the global race to build and dominate the AI industry, it can sure seem like the interests of ordinary people sit last on the agenda. It's just the opposite for EFF. While companies furiously jam AI tools into their veins and your eyeballs, EFF’s technologists, activists, and attorneys have been meticulously cutting through the hype to ensure AI can serve your privacy and free expression. Technology has leaned into a new era, and this summer you can help EFF fight for the people.

JOIN EFF

Over the next two weeks, we’re encouraging you to support the cause as an EFF member for as little as $10 each month. You can get great member swag every year like our privacy puffy stickers, Claw Back t-shirt, and Privacy Badger Crewneck.

Fight mass surveillance! Pictured: Claw Back member t-shirt and Privacy Badger Crewneck.

AI tools—beyond their marketing fluff—demonstrate both incredible potential and real danger. With the support of members around the world, EFF detangles the possibilities from the anxieties and threats with the care and nuance it deserves. In recent months, EFF:

• Mobilized people against the GUARD Act, which would require problematic age verifications systems for AI companions.
• Joined civil society partners to call out a General Services Administration proposal that would make AI tools less safe and less useful.
• Sued for answers under the Freedom of Information Act to uncover how the government is using AI to evaluate requests for medical care.
• Testified before the U.S. Congress Homeland Security Subcommittee on Cybersecurity and Infrastructure Protection.

The scope of AI, both the good and the bad, multiplies every day. If we want the AI-powered benefits of efficiency, scientific discovery, and greater accessibility to knowledge, then we also need strong protections against surveillance, harms to creativity and innovation online, perpetuating systemic bias, and privacy violations now.

With AI taking over the public consciousness, you can be assured that EFF will never stop advocating for you. Together, we can ensure that technology supports freedom, justice, and innovation for all people.

Join EFF

____________________

EFF is a member-supported U.S. 501(c)(3) organization. We've received top ratings from the nonprofit watchdog Charity Navigator since 2013! Your donation is tax-deductible as allowed by law.

This Pride, we’re answering all your digital rights questions in season two of our initiative, LGBT Q&A. 

You Asked: Is there a way for me to wipe data about me online that could point to my queer identity?

EFF’s Answer: You cannot protect everything all the time, but there are ways to wipe information about yourself online. 

Most information available about you online will typically be found in two places:

1. The site where you voluntarily posted the data, such as your pictures and videos on social media, comments in user reviews and forums, and even classified postings for items you’ve sold.
2. A data broker. These companies collect personal information, repackage it, and sell it to the highest bidders. This information often includes your address, phone number, details about your family members, and more. 

So you might not want this information out there, especially if it points to your queer identity. 

The best time to take steps to protect yourself is before anything bad happens, because once this information is in the hands of bad actors you have fewer options.

To see what information people might find about you online, you can look for it for yourself. This is as simple as opening up a search engine and entering your name, nickname, handle, avatar and seeing what comes up. It can also be worth searching for your address, phone number, and email addresses to check what's out there.

Do this in a private browsing window or a separate browser than the one you normally use to ensure you’re not logged into any accounts that might skew the results, like a Google account. 

It’s also best to try to make a lot of your information hard to find in the first place—and we’ve got you covered on how to do this. 

1. Establish a strong security baseline: use unique passwords (a password manager helps simplify this) and set up two-factor authentication for your online accounts to add an extra layer of protection when logging into your accounts.
2. Add our install-and-forget tracker blocking tool, Privacy Badger, which lets you browse in peace and stops the sorts of web trackers that compile information about your habits for advertising purposes and for data brokers.
3. Remove your advertising ID on your phone to help prevent some tracking there, too (directions for Android or iPhone). This way less information about you is available for purchase, making it harder for corporations to profit from your online activities.
4. Ask data brokers to delete your personal data. You might spend the time doing it yourself. If you’re in California, you can use the Privacy Protection Agency’s tool for this. You also might use professional services like EasyOptOuts and Optery to help minimize the information available about you online from data brokers and similar sources.
5. You can remove yourself from Google results by heading to the “Results about you” page, then entering your information. Once set up, you’ll get notifications if some new types of information about you appear in Google Search. Just remember that this will not remove the information from the internet, it just won’t show up in Google’s search.

You also should consider auditing your digital footprint on public-facing social media and forums. Different people have different tolerance for risk when it comes to announcing who we are and what we are doing in these online spaces. You can make a list of every social media or forum account you’ve had over the years, and review the public-facing content about you, including your name, contact information like email addresses or phone numbers, and pictures that might show your home or workplace. You can also review the account settings to ensure you’re comfortable with the privacy options and that you’ve got strong login credentials.

For more in depth advice check out our Surveillance Self Defense guide on managing your digital footprint.

X Corp. should not be able to escape privacy compliance because it changed its name. 

On May 15, X Corp. filed a petition before the Federal Trade Commission (FTC) to set aside or modify an order issued in 2022 requiring the company to report regularly to the FTC for its violations of user data. The order or “consent decree” is a result of misleading the platforms’ 140 million users by using private information given to secure accounts, like phone numbers and email addresses, for targeted advertising. It also fined the company $150 million for the infraction. As part of an open comments period, EFF and allies including Demand Progress Education Fund (DPEF), National Consumers League (NCL) and Electronic Privacy Information Center (EPIC) call on the FTC to reject this petition.

The 2022 order was a renewal of an order stemming from a previous violation. Back in 2011, Twitter (now X) reached a settlement with the FTC after the regulator found Twitter had failed to secure users’ personal information, resulting in exposure of that data to hackers. The settlement banned the company from misrepresenting its data protection measures, required it to set up safeguards on user data, and regularly report its security posture for twenty years. The renewal updated the expiration of X’s obligations to 2042, but if the FTC accepts X's petition, it would end much sooner.

In arguing to set aside the order, X remarks that since the order in 2011 it has “built an entirely new privacy and information security program staffed by new personnel operating under new leadership with a … philosophy grounded on the importance of privacy and information security.” 

These sweeping assurances that corporate restructuring led to a fundamental change in X’s policy and practices around user data should be met with a healthy dose of skepticism, given evidence to the contrary. For example, the company’s quiet rollout integrated its AI model Grok with the platform in 2024, trained (without meaningful consent) on X user data. The company was also subject to a massive data breach in 2025. Even if a rotation of leadership led to prioritizing privacy and information security, our letter highlights that this would not be sufficient grounds to remove the order, “because the FTC orders bind the corporate entity. Those obligations do not dissolve when the employees who negotiated or administered it depart.”

X argues that its entry into the AI space should be reason not to continue the oversight, claiming that “terminating the Order is critical to advancing American leadership in artificial intelligence.” Here again, broad-stroke claims that the guardrails in place “[diverts] engineering resources from innovation to compliance paperwork” ignores the dangers that AI introduces to user data. Far from being a reason to waive the order, clever attacks on models trained on user data has the ability to supercharge the types of secondary use violations that led to the 2022 order renewal. After all, an entire art has been developed around engineering LLM prompts to reveal the data a model was originally trained on.

Our response to X’s petition debunks many claims the company uses in its arguments. For example, there’s little evidence the order placed an undue financial burden on X. In our letter, we note that the compliance cost is merely “a rounding error against the $200 billion valuation of X Corp. following the xAI merger.”

Strong safeguards on our information require eagle-eyed oversight when that data is abused and misused for profiteering ventures. X’s actions not only showed us this in the past, but continue to do so in the present day. We and our civil society partners urge the FTC to take the clear, sensible path and reject X’s petition.