Algorithmic Amplification of Hate Speech: How Platform Design Shapes Online Extremism

Content recommendation algorithms are among the most consequential but least visible forces shaping what people see online. While these systems do not create hateful content, they determine which content gets distributed to large audiences — and research consistently finds that content triggering strong emotional reactions, including outrage, fear, and moral disgust, tends to perform well under the engagement metrics these algorithms are optimised to maximise. Understanding how recommendation systems work is essential for anyone seeking to address the relationship between platform design and the spread of online extremism.

How Recommendation Algorithms Work

Most major social media platforms use algorithmic recommendation systems to decide what content to show each user. These systems are typically optimised for engagement — broadly defined as time spent on the platform, clicks, shares, likes, and comments. Machine learning models analyse which content each user has previously engaged with and predict what will generate further engagement from that particular user.

The core problem is that engagement-maximising systems are not neutral with respect to content type. Content that provokes strong emotional reactions — anger, fear, moral outrage, disgust — reliably generates more engagement signals than content that is merely informative or that requires careful consideration. This creates a structural tendency for recommendation algorithms to surface emotionally provocative content, regardless of its accuracy or its effects on the people who see it.

The Radicalisation Pipeline

Researchers studying algorithmic recommendation in the context of online extremism have documented a pattern sometimes described as the radicalisation pipeline. Users who engage with moderately partisan or provocative content may find that algorithmic recommendations gradually shift toward more extreme versions of similar material — not through any deliberate design intent, but because progressively more extreme content tends to generate stronger engagement signals from users already interested in a topic.

A user who watches mainstream political content may be recommended increasingly inflammatory commentary. A user who engages with concerns about food safety may receive recommendations escalating toward elaborate health conspiracy theories. This dynamic is probabilistic rather than deterministic — most users who encounter extreme content do not become radicalised. But at the scale at which major platforms operate, even marginal effects on small percentages of users translate to large absolute numbers of people receiving consistent exposure to extreme material.

Research on Algorithmic Amplification

The academic study of algorithmic amplification has expanded substantially in recent years, partly because researchers have gained intermittent access to platform data through regulatory proceedings, litigation, and voluntary disclosure agreements. Studies have found that on several major platforms, a disproportionate share of engagement with extremist or false content arrives through algorithmic recommendation rather than through users actively seeking that content out. The implication is that changes to recommendation systems — not just content removal — could meaningfully reduce the spread of harmful material even without deleting individual posts.

Platform-commissioned research has also produced findings relevant to algorithmic amplification. Internal studies at some platforms have found that certain design choices — including particular approaches to ranking engagement signals — contributed to increases in the volume of divisive and false content that users were shown. The publication of some of this internal research through regulatory processes has been important for public understanding of how platform design choices affect information environments.

What the DSA Requires

The EU's Digital Services Act (DSA), fully applicable to very large online platforms since early 2024, creates specific obligations relating to algorithmic recommendation systems. Very large online platforms — those with more than 45 million monthly active users in the EU — must conduct annual systemic risk assessments that explicitly consider how their recommendation algorithms contribute to the amplification of illegal content and content with significant negative societal effects, including content that contributes to violence or hatred. Identified risks must be addressed with proportionate mitigation measures.

The DSA also requires very large platforms to provide users with at least one option for content recommendation that does not rely on profiling of the user's behaviour — meaning at minimum a feed that shows content without algorithmic personalisation, such as a chronological timeline. Platforms must make these options genuinely accessible rather than burying them in settings menus. Non-compliance with DSA obligations can result in fines of up to 6% of global annual turnover.

Researcher Access and Transparency

The DSA additionally requires very large platforms to provide vetted researchers with access to their data for the purpose of studying systemic risks, including algorithmic amplification. Researchers affiliated with universities or civil society organisations can apply for vetted researcher status through the European Commission, giving them the right to request data that platforms would not ordinarily disclose. This provision is intended to create an ongoing evidence base for regulation rather than relying solely on what platforms choose to publish in their voluntary transparency reports.

Civil society organisations have argued that the vetted researcher provisions should be interpreted expansively, and that transparency reports — which platforms must also publish under the DSA — should include sufficiently granular data on how recommendation systems interact with content policy violations to be useful for independent analysis.

What Civil Society Can Demand

Beyond regulatory compliance, advocates have identified several design changes that would reduce harmful amplification. Chronological feeds — showing users content in the order it was posted rather than in engagement-optimised order — remove the recommendation layer for users who prefer them. Friction mechanisms, such as prompts asking users to read an article before sharing it, have shown modest effects on the spread of false information in some research contexts. User controls that allow people to exclude specific content categories, reduce exposure to content from accounts with policy violation histories, or limit recommendations to accounts they actively follow all represent less disruptive alternatives to chronological feeds.

The DSA framework provides the legal basis for demanding these changes from very large platforms operating in the EU. The detailed regulation of how these requirements are implemented is developed through the Digital Services Coordinator system and guidance from the European Commission. For further background on DSA obligations and what the regulation requires of platforms on hate speech, as well as the national legal frameworks across EU member states, see the related articles on this site.

Frequently Asked Questions

Do platforms deliberately design their algorithms to spread hate speech?

Not deliberately. Recommendation algorithms are generally designed to maximise user engagement — time on platform, clicks, and reactions. The problem is that hateful, outrage-generating, and extreme content often performs well under engagement metrics. Platforms optimising for engagement without specific constraints therefore tend to amplify harmful content as a side effect of pursuing their commercial objective.

What is the DSA's systemic risk assessment requirement?

Very large online platforms under the DSA must annually assess the systemic risks their services pose, including risks arising from algorithmic amplification of illegal or harmful content. These assessments must be transmitted to the European Commission and are subject to independent audit. Risks identified in the assessment must be addressed with proportionate mitigation measures, and failure to do so can result in significant financial penalties.

What is a chronological feed, and does it reduce harmful content?

A chronological feed shows posts in the order they were published rather than in an algorithmically determined order optimised for engagement. Because the engagement-maximisation dynamic that tends to reward extreme content does not apply to chronological feeds, users who switch to them typically see less extreme content in their timelines. However, they still see content from accounts they follow and can still encounter harmful material if the accounts they follow share it.

Can users currently opt out of algorithmic recommendation on major platforms?

Some platforms offer limited options to adjust recommendation settings or view chronological timelines. The DSA's requirements — which require very large platforms to offer at least one non-profiling-based recommendation option — have strengthened these obligations for platforms operating in the EU. The accessibility and completeness of these options varies significantly between platforms and is an active area of regulatory attention.