As deepfake technology enters the mainstream, trust and safety teams are grappling with a new scale of operational challenge. Artificial intelligence now enables anyone to generate highly convincing fake videos, images, or audio that can be weaponized to spread misinformation or impersonate others. Nearly half of internet users today say they’ve encountered at least one deepfake in the past six months, a clear indicator that these synthetic media threats are no longer rare novelties but a regular moderation concern. At the same time, new digital regulations like Europe’s Digital Services Act (DSA) are dramatically raising the bar for content moderation speed, transparency, and accountability. In 2026, trust & safety leaders find themselves squeezed between the rising tide of deepfakes and the tightening grip of compliance mandates. This post examines how deepfakes are straining moderation teams’ operations and psyche, how laws such as the DSA demand stronger responses, and why safeguarding moderator wellbeing is becoming as critical as enforcing platform rules.
Deepfakes and the rising strain on moderation teams
The proliferation of deepfakes has quickly escalated the pressure on content moderation operations. Thanks to advanced AI tools, today’s deepfakes can look and sound real, making it difficult for moderators and detection systems to tell authentic content from forgery. Social networks have already been flooded with AI-manipulated videos and photos, leading to widespread public skepticism about what’s real online. A Deloitte survey found 59% of users familiar with generative AI have a hard time distinguishing AI-generated media from human-made content which is a troubling statistic that underscores how even trained eyes can be fooled. Moderators face a moving target: new deepfake techniques (including bogus audio voices that perfectly mimic real people) continually emerge, often outpacing the development of detection tools. Indeed, existing moderation algorithms show degraded performance on AI-generated misinformation compared to human-created content, forcing companies to invest heavily in improved deepfake detection (global spending on detection is projected to triple from $5.5B in 2023 to $15.7B in 2026).
Beyond sheer volume and sophistication, deepfakes raise the stakes of moderation errors. A realistic fake video can go viral and erode public trust within minutes, potentially inciting real-world harm or fraud before it’s caught. For example, AI-generated “revenge porn” or non-consensual fake sexual images has become disturbingly common, causing severe emotional harm to victims and reputational damage before platforms can remove it. Moderators now must treat every suspicious video or account as high-risk, knowing that missing a single deepfake could mean major legal or safety consequences. Conversely, falsely flagging legitimate content as a deepfake is also costly as it undermines user trust and could violate the new transparency rules. This narrow margin for error puts moderators under intense decision pressure. They are expected to make split-second judgments about nuanced, borderline content, often within 10–30 seconds per item, thousands of times each shift, even as deepfakes make those judgments far more complex. In short, deepfakes are forcing trust & safety teams to work faster and smarter at the same time, a combination that is stretching operational capacity to its limits.
New laws demand faster, more transparent moderation
Overlaying this challenge is a wave of new digital legislation that compels platforms to step up their content moderation practices. The EU’s Digital Services Act (DSA), which came into force for large platforms by 2024, exemplifies this regulatory push. The DSA mandates that online platforms swiftly remove illegal content once notified and provide clear explanations to users when their posts are taken down. Users must have easy appeal options for moderation decisions, and “trusted flagger” organizations’ reports of illicit content receive priority handling. In effect, laws like the DSA are introducing service-level standards for moderation speed and due process. A platform can no longer afford to leave a harmful deepfake circulating for days or quietly delete it without notification, such lapses could mean non-compliance. In fact, regulators are already flexing their muscles: in late 2025 the European Commission fined Twitter (X) €120 million for breaching DSA transparency rules, signaling that accountability will be enforced with real penalties.
Very large online platforms are also required by the DSA to proactively assess and mitigate “systemic risks” such as the spread of disinformation and manipulation. That means trust & safety teams must not only react faster, but also anticipate how emerging threats like deepfakes could impact public discourse, elections, or users’ mental wellbeing. Internally, this translates to more rigorous reporting, documentation, and oversight of moderation processes which is essentially, additional layers of work for teams already under strain. Moderators might have to log detailed justifications for borderline deepfake removals to demonstrate consistency and fairness, and companies must compile comprehensive transparency reports. “The law’s transparency and safety requirements raise the floor for everyone” as one industry insider put it, forcing even reluctant platforms to invest in more robust content policing. In practice, however, many trust & safety professionals feel their work becoming “cold, compliance-driven…ticking legal checkboxes and racing to avoid fines” rather than focusing on the spirit of user protection.
Regulatory pressure is not confined to Europe. Governments around the world are introducing rules aimed at deepfakes and AI-manipulated media. China’s new guidelines, for instance, require AI content to be clearly labeled and even hold platforms responsible for scanning every uploaded file for AI-generated traces, a task that one expert noted would be a “huge burden” given the hundreds of millions of uploads per day. Australia recently passed laws criminalizing the sharing of explicit deepfake images without consent. In the U.S., proposed measures like the DEFIANCE Act seek to penalize malicious deepfake creators. The common thread is that moderation teams are expected to catch more violations faster, and to document their actions in greater detail, under threat of legal repercussion. By 2026, trust & safety teams operate in an environment of heightened scrutiny and higher expectations from both the public and policymakers. The result is that frontline moderators and policy specialists often feel they are racing the clock and the algorithms to stay ahead of both deepfake purveyors and the regulators looking over their shoulder.
Deepfake escalation takes a psychological toll on teams
All of this is happening against the backdrop of an already-challenging job for content moderators. Long before deepfakes, moderation teams were known to face severe psychological pressures from constant exposure to graphic violence, hate, child abuse, and other traumatic material. Now the rise of deepfakes adds new layers to that burden. Ironically, the fact that a piece of media is fake does not necessarily lessen its emotional impact on a reviewer, a hyper-realistic video of violence or abuse, even if AI-generated, can be just as disturbing to watch. Moderators also grapple with the moral implications of synthetic media. They may experience moral injury when tasked with enforcing policies that conflict with their personal ethics or sense of justice. For example, a moderator might feel ethical strain removing what appears to be satirical or consensual synthetic media in an environment of strict zero-tolerance rules, or conversely, feel guilt after leaving up a deepfake that later causes harm. Research on content moderators notes they often face ambiguous guidelines and high ambiguity, which “can leave moderators unsure if their decisions align with ethical standards…leading to stress and moral injury”.
Compounding these ethical dilemmas is simple decision fatigue. The volume and velocity of decisions required in trust & safety work are tremendous, moderators may process thousands of pieces of content per day, spending mere seconds on each under constant time pressure. Deepfakes heighten the mental toll because each decision demands extra vigilance (to discern subtle signs of manipulation) and carries higher stakes. This relentless cognitive load quickly leads to exhaustion. Moderators describe feeling like they are on a “pressure cooker” assembly line where speed and accuracy are both paramount and never-ending. There is little room for mental rest, yet a single lapse could have serious consequences. Over time, this contributes to burnout and error rates creeping up as fatigue sets in.
A related burden is the emotional labour required. Trust & safety staff must maintain a calm, professional demeanor while continually immersing themselves in often vile or distressing content. They often cannot show or even fully acknowledge their natural emotional reactions during the workday. This suppression takes a toll: studies have likened content moderators’ levels of psychological distress to those of first responders dealing with trauma. Many moderators report symptoms of secondary traumatic stress, anxiety, depression, or PTSD after months or years on the job. The constant need to “stay neutral” and not let disgust, fear, or sadness surface is itself exhausting. As one analysis noted, moderators operate in an environment where they must make split-second judgments on complex content while suppressing any emotional response which is a recipe for emotional exhaustion and detachment. Indeed, clinical research finds over a third of content moderators show moderate to severe psychological distress requiring intervention, and nearly all moderators exhibit some level of distress symptoms over time. Key forms of psychological strain include:
-
Moral injury: Psychological distress from actions that violate one’s moral or ethical code. In content moderation, this can occur when moderators feel forced to enact decisions at odds with their values or witness harms they are powerless to prevent. The clash between personal ethics and corporate policy (for instance, leaving up a harmful political deepfake in the name of free expression, or removing content they feel shouldn’t be censored) can accumulate into profound guilt or inner conflict.
-
Decision fatigue: Deterioration of decision quality and mental energy after making many decisions in succession. Moderators face this daily as they must rapidly adjudicate an endless queue of content. Deepfakes intensify decision fatigue by requiring extra analysis of each frame or audio clip for subtle falseness. As mental reserves deplete, the risk of mistakes grows which is a dangerous situation given new laws’ expectation of near-flawless, rapid moderation.
-
Emotional labor: The strain of managing one’s emotional expressions as part of the job. Trust & safety workers must continuously project impartiality and resilience, even while reviewing graphic or manipulative content. They often cannot react naturally to disturbing imagery, which can lead to suppressed emotions “leaking” in other ways such as burnout, numbness, or anxiety. Research highlights that this constant suppression can push moderators toward emotional exhaustion and detachment from their feelings, classic signs of burnout. Over time, moderators may become desensitized (a coping mechanism), which can itself be distressing and impact their empathy in personal life.
It’s clear that the wellbeing of moderation teams is being tested like never before. As one industry expert observed, these staff are the “first responders” of the internet’s worst content, and “unless we take care of the people, the machines won’t work and we won’t achieve what we’re supposed to”. In other words, a moderation team under extreme stress or trauma cannot effectively protect users or enforce policies, human factors directly affect operational performance.
Managing performance and wellbeing together through systemic support
Given the dual pressures of higher performance demands and mounting psychological strain, leading organizations are recognizing that moderator wellbeing and moderation quality are two sides of the same coin. Achieving faster takedowns and greater accuracy in detecting deepfakes is futile if the very people implementing those measures are burning out or faltering under stress. The strategic response is to bake support structures into the fabric of trust & safety operations like moving from ad-hoc wellness perks to systemic, embedded support for teams. In practice, this means treating moderator mental health as an operational priority, not just an HR issue.
Crucially, traditional one-size-fits-all wellness programs have proven inadequate for high-pressure roles like content moderation. Generic offerings that sit outside daily workflows (a meditation app subscription, occasional webinars, or an EAP hotline) often go unused by moderators dealing with acute stress. What works better is a holistic approach co-designed with the teams, addressing needs at every level from frontline moderators to policy managers. This “whole ecosystem” mindset acknowledges that the pressures come from all directions, the relentless content stream, the external regulatory demands, internal targets from leadership, and so support must be multi-layered as well.
Key elements of a systemic support approach include proactive and ongoing interventions rather than reactive care. For example, some trust & safety organizations now build in regular psychological skill-building and check-ins as part of the work routine. Evidence shows that structured peer support programs combined with basic psychoeducation can significantly improve moderators’ wellbeing and resilience, boosting their self-efficacy and reducing depression levels. Embedding trained peer supporters within teams and offering short, evidence-based mental health modules during work hours encourages moderators to actually utilize these resources. Another proven practice is scheduling resilience workshops followed by one-on-one counselling sessions for moderators on a periodic basis. In one recent study, 28% of content moderators were found to have moderate-to-severe distress, but those who developed stronger problem-focused coping skills had noticeably lower trauma symptoms. By delivering coping strategy workshops (teaching, say, how to actively reframe stress or seek social support) and then immediately giving staff access to confidential counseling, companies saw reductions in trauma symptoms and improved overall wellbeing.
Crucially, these supports must be integrated into daily operations, not bolted on as afterthoughts. Forward-looking teams are experimenting with measures like protected wellbeing time, short breaks or rotation periods specifically designated for decompression, to interrupt the cycle of continuous high-alert monitoring. Neuroscience research indicates that even brief downtime to reset can “deactivate” the physiological stress response and help moderators return to a calmer baseline. Many companies are also implementing task rotation policies: for instance, no moderator should spend an entire day on the most extreme content queues (like graphic violence or child abuse material) without rotation to lighter tasks. Simply spreading out exposure and giving recovery time can prevent the worst cumulative trauma. Additionally, leadership training is being introduced so that managers in trust & safety can recognize burnout or trauma signs early and foster a team culture where it’s normal to talk about mental health challenges. The goal is to catch issues before they escalate, as one study notes, early intervention programs can significantly cut sick leave and reduce stigma around seeking help.
Not only do these embedded support strategies protect employees, they also directly bolster performance metrics. Organizations have found that when they invest in moderator wellbeing as a core strategy, it pays off in more consistent, higher-quality moderation outcomes. Prioritizing moderators’ mental health is “not just an ethical necessity but a strategic investment,” as one report put it, it helps improve accuracy and reduces costly staff turnover in these critical roles. In fact, cases show that after rolling out comprehensive support frameworks, teams experienced fewer moderation errors and better decision quality under pressure. By keeping moderators within a healthier “window of tolerance”, their optimal zone of cognitive functioning, for longer, these support systems prevent performance from breaking down during crisis spikes. In essence, well-supported moderators make better moderators. They can sustain focus, exercise sound judgment, and bounce back from stressful incidents more readily, which in turn means platforms stay in compliance with those demanding new laws and maintain user trust.
As we look towards 2026 and beyond, it’s evident that tackling deepfakes under strict digital laws will require more than just better algorithms or bigger moderation teams. It demands a human-centered strategy that elevates the wellbeing of the people on the frontlines to the same priority as technical and policy solutions. The organizations leading the way are those treating trust & safety not merely as a compliance function or cost center, but as a mission that depends on human resilience. They recognize, as one safety veteran noted, that regulation has “raised the floor” on what’s expected, and that to meet these expectations sustainably, they must raise their support for the workforce in tandem. By embedding systemic wellness support, providing psychological tools and protections, and cultivating a culture of care, platforms can empower their teams to meet the deepfake challenge without sacrificing their mental health. The result is a virtuous cycle: moderators who are supported to thrive will deliver the faster, more accountable content governance that regulators and communities now demand. In an era of AI-driven deception and heightened oversight, keeping our trust & safety professionals healthy, focused, and resilient is not just about taking care of them, it’s about safeguarding the integrity of our digital spaces for everyone.
