This week we ask how social media platforms should handle harmful content on their sites — and what human rights law can tell us about when to remove it and when to simply reduce its visibility.
Social media has a content problem. Some posts incite violence or hatred; others spread misinformation or promote self-harm. The instinctive response is removal — but is that always the right tool?
A new paper argues that platforms should often demote rather than remove problematic content: reducing its visibility rather than taking it down entirely. Drawing on international human rights law, it sets out a framework for when demotion is justified, when removal goes too far, and what transparency obligations platforms owe their users. The argument has implications for everything from climate misinformation to eating disorder content to the regulation of use of social media by under 16s.
Joining host Emily McTernan is Jeff Howard, Professor of Political Philosophy and Public Policy at the UCL Department of Political Science and the founding Director of the Digital Speech Lab.
Mentioned in this episode:
[00:00:05] Emily McTernan: Hello, this is UCL Uncovering Politics, and this week we ask how should social media platforms handle the problematic content on their sites?
Hello, my name is Emily McTernan, and welcome to UCL Uncovering Politics, the podcast of the School of Public Policy and Department of Political Science at University College London.
Social Media has a content problem. Some of its posts incite violence and hatred. Others cause harm in more diffuse ways or promote misinformation. So what should social media platforms do to clean up their sites? Some weeks back, we had Rob Simpson on discussing whether we should just quit social media altogether, and we're gonna link back to that episode for those interested in the show notes.
But this week I'm joined by a professor who has, I think it's fair to say, a more positive view of social media and what it might do for us. And so a careful answer about what we should do about that bad stuff on social media sites. And that's Jeff Howard, professor of our department and a director of the Digital Speech Lab.
And he's gonna give us an answer to this content problem of social media. In a nutshell, it's that social media platforms should often reduce rather than remove that bad stuff. So welcome back to the podcast, Jeff.
[00:01:20] Jeff Howard: Thank you, Emily. It's great to be with you.
[00:01:23] Emily McTernan: Let's start with your overall approach to the social media space.
Is it fair to describe you as someone who's positive about the value of social media, even in the face of growing doubts in society about social media, what it's done to our politics, culture, and even our brains?
[00:01:37] Jeff Howard: I think about social media in much the same way I think about automotive transit. So I think automotive transit has done a lot of good for the world. It allows people to go visit relatives and friends who live far away. It allows them to travel and experience new scenery and places, visit new cities, try new foods and cultures.
But obviously, there can be huge harms that arise when it isn't regulated properly, when people don't wear seat belts, when there aren't guardrails on the side of the roads when there aren't speed limits. And of course, when it results in various harms to our planet, but provided all the relevant regulations, both in terms of safety and environment are in place, then I tend to think that automotive transit is a net good.
And I can imagine a version of social media that works like that where it's appropriately regulated, it's used in a reasonably moderate way, it has certain safety guardrails in place that it does constitute a positive good for society.
And that's pretty much because social media serves as a kind of public forum. It is the place where people go to express themselves, to ask questions to one another, to reconnect with old friends, to receive information.
Increasingly, social media platforms like TikTok are really a new form of entertainment that's replacing television where people go to be amused. And I think all of the standard reasons why we value these multifarious forms of communication help explain what's valuable about social media when it is valuable.
[00:03:02] Emily McTernan: And so the paper we're discussing today is about one part of the regulation of this social media set of platforms that you're describing as having all of these goods. And so I want to talk a little bit about bad content before we get to your answer. What's the kind of problematic, bad content you want to talk about in this paper?
[00:03:18] Jeff Howard: I think that there are lots of categories of harmful speech that people often post on social media. It can include bullying and harassment, it can include varieties of death threats and incitements to violence. It can include varieties of hate speech that denigrates members of vulnerable groups. It of course can include content like child sexual abuse, material content that tries to coordinate terrorist attacks or recruit people into terrorist organisations.
If you go to the websites of the main social media platforms, you will see a very long list of the different categories of harmful speech that they prescribe. And I tend to think about these categories as embodying normative or moral expectations that we appropriately place on users of these platforms.
So if you're gonna go and be part of these communicative spaces, you have certain obligations as a user not to post certain kinds of harmful content. And so I see these policies as enforcing rules against this content.
And what's interesting is the most common way that rules are enforced against harmful content is of course, by removing it. So if you post a piece of hate speech or you post a death threat, then it's gonna be taken down.
And what's really interesting though, is that that's not the only way that platforms can go about the activity of reducing harmful content. And that's really the insight that inspired me to work on this next topic.
[00:04:33] Emily McTernan: And we're gonna talk about your alternative to removing in just a second. But before we get there, what's interesting about some of the examples you've just given us is a lot of it we have existing law to deal with that would apply offline as well, right?
So if I go and I say some hate speech in a public forum offline, then there are gonna be consequences for me, potentially legal consequences, likewise, child sex abuse, and all the rest of it.
So what you are saying then is you're saying something like you are interested in what social media platforms should do to help enforce those laws, or are you interested in a category of speech that falls short of the kinds of stuff that is banned by the law?
[00:05:06] Jeff Howard: So I think that there will be overlap between the categories of harmful content that social media platforms ought to restrict, and the categories of speech that are already appropriately criminalised. So certainly examples like incitement to imminent violence or posting of a death threat or posting child sexual abuse material, posting revenge pornography will all fall afoul of various criminal statutes.
And so in those cases, it isn't just appropriate for platforms to remove the speech in question. It may even be appropriate for the platforms to report those speech to the relevant authorities.
In other cases, you'll have content that is unlawful but actually doesn't necessarily violate the platform's policy. So if you think about defamation for example. So suppose I post a nasty rumour about you on Facebook, you could sue me in court for posting that nasty rumour about you and therefore defaming your reputation. But we don't actually expect the social media platforms to engage in some kind of offline investigation about whether the rumour is true or not.
So there'll be some cases in which something is unlawful, but actually isn't gonna be covered by the platform's explicit rules. Although it's worth noting that if platforms get a court order to take down defamatory content, they'll be required to do so.
But I think the criminal law is a pretty extreme tool that's used only for the most harmful categories of speech. And I think there's gonna be lots of categories of speech that platforms ought to moderate, even though they don't fall afoul of appropriately specified criminal laws.
And so I think for example, misinformation will fall into this category. I think platforms have responsibilities to maintain the health of our information environment, the health of our information ecosystem. And you can hold that view while also thinking that it would be a mistake for like the police to start arresting people for posting misinformation.
[00:06:47] Emily McTernan: Okay, so we've got misinformation up there. And then I guess the other thing that comes up in your paper is things like encouraging self-harm or encouraging eating disorders and giving extreme weight loss advice that the kind of content that we wouldn't see being legally banned but we might think actual social media sites should be doing something about that.
[00:07:03] Jeff Howard: That's right. So you could imagine speech that explicitly encourages someone to take their own life or explicitly encourages them to engage in physical self harm. And it seems plausible to me that these kinds of communications are not protected by the right to free speech and could appropriately be criminalised by the state and also removed by the social media platforms.
But then there's gonna be these really difficult borderline cases in which people aren't explicitly encouraging these forms of self-harm, but are posting content that we know is correlated with eventual self-harm. So you've just used the example of extreme body weight loss advice or just informational content of 'Hey, here's this new diet you can do to try to lose weight.'
And in these kinds of cases, I think it would be disproportionate to try to ban the speech in question. But I still think that social media platforms have a responsibility to think about the harms that this kind of content can cause, especially when particular vulnerable people are bombarded with streams of it on their news feeds.
[00:08:04] Emily McTernan: Great. So now we've got the problematic content in view and these harder cases to deal with that we can't just deal with through the law. Let's talk about your positive strategy that you propose and discuss the ethics of. So what is demotion as a strategy and how's it different to the ordinary ways social media companies manipulate what we see through their algorithms?
[00:08:21] Jeff Howard: Great. So demotion refers to the deliberate practise of reducing the visibility of content so that fewer people see it and people see it less frequently. So when content is demoted, people are allowed to post it. But the number of people who see it will be lower and the frequency of which people see it will be less.
Now what's really interesting is that platforms have been doing this for forever. And there was a real spate of debate a few years ago in connection with this idea of shadow banning. So people found that suddenly they were used to getting hundreds of thousands of likes on their posts on Instagram, but then they were only getting thousands of likes.
And they were like, what's going on here? Clearly the system is manipulating who can see this in some way? So I'm just getting less exposure and this provoked a real outburst because people were like, this is really, really powerful, this tool of demoting content. And yet the platforms are not being transparent about what they're doing.
And that public outcry has had, I think, a salutary impact, which is that platforms are now much more transparent about what kind of content receives this treatment. So if you go to Google right now and you type Meta types of content we demote, you'll see the list of content that Meta demotes on its Instagram and Facebook platforms.
TikTok has a section outlining what it calls its eligibility standards for its For You feed, which is basically the same thing. A list of the kind of content that is excluded from recommendations. So it isn't gonna be piped up and put in the prime real estate of social media, which is the top of people's feed.
And so all the platforms now are engaging in this practise whereby they say, look, here's some content that we think is problematic in some way. We don't think it's so problematic that we're gonna remove it, but we aren't going to actively amplify it to users. I think this practice is often confused with a very closely related phenomenon that we might discuss.
And this is the phenomenon by which platforms are just regularly amplifying or not amplifying content based on individualised predictions about what people will find engaging. So think about this outside the social media context.
So like, I like horror movies, so Netflix knows this from my past behaviour, knows in scare quotes. And so Netflix will amplify, that is increase the visibility, of horror movies and its recommendations for me based on an accurate, in this case prediction that I'll find that content engaging. I don't know if you like horror movies, Emily, but if you don't, then Netflix will probably learn that from your past behaviour and it will be reluctant to recommend those horror movies to you. And this doesn't reflect any judgement that horror movies are bad or harmful. It just reflects a individually tailored algorithmic prediction about what is to our taste.
And so, here's an example of content that regularly doesn't get amplified on social media: boring content. The system predicts that people won't find it, that engaging or it predicts that certain groups of people won't find it engaging, and so that content is not amplified. And that phenomenon is really, really interesting and important because how the platforms decide what's in people's feeds can really have huge impacts on how public discourse is shaped.
I'm talking about something a bit more specific, which is about a deliberate policy choice to say, here's a certain kind of content that we think is problematic and we're gonna reduce its amplification, not just for people who are gonna find it boring. We're gonna reduce its amplification for everybody, and we're gonna do it because we've made a judgement that this content is bad or harmful or problematic in some way. That's what demotion policy is about.
[00:11:38] Emily McTernan: And you might ask, why do we even need to justify this demotion policy? So surely a right to free speech doesn't mean we have a right to an audience or a platform, so what do you think the moral problem with just demoting stuff is?
[00:11:48] Jeff Howard: Yeah. Renée DiResta, who's a wonderful activist and scholar in this space, coined a fantastic slogan a few years back: free speech is not free reach. And this is something that even Elon Musk has adopted explicitly in explaining why certain kinds of hate speech, for example, might technically be allowed on X, his platform, but in theory anyway, the speech should not be amplified.
And so you might think, well gosh, why is it a problem to demote content? Why is this the sort of thing that the platforms need to justify? And I think we can get a grip on this situation by imagining that the state were to force platforms to engage in certain kinds of demotion and then asking us, does that raise any moral questions?
So imagine the state were to pass a law telling the platforms that they had to demote all content promoting the Muslim faith, or promoting the Jewish faith or promoting Catholicism, or imagine the state were to pass a law that said all pro-Labour Party or pro-Reform Party or anti-Labour Party content should get demoted.
I think that kind of compelled demotion would trigger a public outcry on the grounds that it's incompatible with freedom of expression. And it would be no adequate reply for the state to say, but hang on, we're not removing the content, we're just demoting it. And to my mind, the point here is that the fact that these hypothetical policies I just imagined involve a less severe sanction than removal does not obviate the free speech issue.
And that's because these policies involve what's called viewpoint discrimination. They involve suppressing certain disfavoured viewpoints in order to limit people's exposure to them and suppressing viewpoints on the grounds that we want fewer people to see them because we're worried about what people will do if they'll see them, gosh, that is the fundamental way in which free speech is violated according to traditional theories of freedom of expression.
[00:13:38] Emily McTernan: When it comes to the state.
[00:13:39] Jeff Howard: So that's the.
[00:13:40] Emily McTernan: They've smuggled in the state here, right?
[00:13:41] Jeff Howard: Not smuggled. Not smuggled. Explicit.
[00:13:43] Emily McTernan: Explicitly. Explicitly explicit in the state.
[00:13:45] Jeff Howard: Explicit brought in the state.
[00:13:46] Emily McTernan: And used it to motivate a worry about a private company doing this.
[00:13:49] Jeff Howard: That's right.
[00:13:49] Emily McTernan: And lots of people might think those are just really different cases, so you should get extremely worried about states.
[00:13:55] Jeff Howard: Yeah.
[00:13:55] Emily McTernan: Heavy handedly trying to produce some kind of viewpoint discrimination amongst their population. But it's much less worrying if, I don't know, Twitter decides it's not gonna be a site of hate speech and it's not gonna be a kind of right wing cesspit. And so it's going to change slightly what you see on your algorithms or someone else takes it over and decides, actually it is gonna be a right wing cesspit. And so you are gonna see all of that content all the time, even if you've shown no interest in it.
And that's just for commercial companies to make their own decisions about what we'll sell and what will produce the kind of engagement that they can make money off. That's very different to introducing the state, right? So some free speech, people think that that's centrally a right, that citizens have against the state. We're much just worried about what happens at a more horizontal level amongst citizens.
[00:14:36] Jeff Howard: That's exactly right. And I think there are good reasons to be particularly concerned about restrictions on our expression by the state. Given the immense amount of power that the state has, there's a huge difference between a social media platform deciding to take down my post and a police officer arresting me for something I've said as part of public discourse.
And so I think these traditional reasons to worry, in particular about state restrictions on speech are completely valid. The question then is, is it possible for other entities other than the state to violate our free speech rights? And there is a strong tradition of saying the answer is no, freedom of expression is just a right we hold against the state.
But increasingly I think this traditional way of thinking about free speech is strained. And in the social media context, the insight is that these entities have become such powerful, pervasive, central forums for public discourse and their policies end up having a pervasive effect on the shape and content of our public discourse that for them to simply say, 'ah, we're private companies, we can do whatever we want', just doesn't strike many people as good enough.
It looks like there are, at the very least moral reasons, that bear on what these platforms decide to do. And in some cases, the platforms themselves have voluntarily signed up to this normative scrutiny by saying, we see ourselves as spaces for the exercise of free speech, we see ourselves as, to use a phrase from the free speech literature, a public forum, a space that's expressly dedicated for the exercise of people's communicative freedoms.
And so at the very least, for those platforms that have explicitly said, we want you to think of us as a space for the expression of free speech, well, when they then engage in demotion, then I think they've invited the question, okay, can the demotion policies at issue be justified? And there is a certain kind of free speech absolutist that'll be like, no, they can never be justified. I think many demotion policies can be justified, but I do think they do need to be justified.
[00:16:32] Emily McTernan: Let's turn to your normative framework though. So then to think about when they are justified and when we should do this rather than remove the content. So you take normative insights in your paper from international human rights law to think about content moderation. So perhaps we can go through this a little bit. Let's start with the reasons you draw for thinking that we ought to demote speech sometimes.
[00:16:50] Jeff Howard: Sure. So the reason I draw on international human rights law, and I should say I've co-authored this paper with my wonderful colleague, Beatriz Kira. The reason we draw an international human rights law is that international human rights law is increasingly appealed to by the social media platforms, by civil society activists, and by external oversight organisations like Meta's Oversight Board as an appropriate normative basis for critiquing and evaluating content moderation decisions online.
And so there's a strategic reason then to use international human rights law, which is this is the language in which so many crucial actors are articulating their views about how content moderation should or shouldn't be done. But at a deeper philosophical level, there's nothing particularly controversial going on here because international human rights law embodies longstanding ideas about how to justify infringements or restrictions on people's rights that are very commonplace in the liberal tradition.
So if we could just go through some of these criteria for how to justify a restriction on speech. So the first is legitimacy. The restriction on speech needs to serve some recognised goal. It can't just be for any reason whatsoever. So in the case of restricting speech, inciting violence, the goal in question is of course, to protect people's physical security. So that is not some niche idea in international human rights law. That's like a foundational idea in the history of liberal political philosophy. That's legitimacy.
Second legality is the idea that restrictions just need to be set out clearly in advance so that people know what the rules are. The rules aren't applied retroactively, they're applied even handedly, so that like cases are treated alike. Again, not a niche idea. Foundational insight in the liberal legal tradition, otherwise known as the rule of law.
Third requirement, I'll go through these quickly, is suitability. That just means that restrictions are suited to achieve their goal so that a restriction that clearly would have no chance of accomplishing its putative aim can't be justified because it has no chance of accomplishing its aim and so it's gratuitously infringing someone's liberty for no purpose.
And then fourth and fifth, which maybe we'll talk about more, involve these ideas of necessity and proportionality. And these are longstanding normative principles that govern when you can restrict people's rights.
So in a nutshell, the reason we apply to international human rights law is that it is the vocabulary that scholars and activists and decision makers are appealing to. But there's a deep continuity between these insights and what political philosophers and moral philosophers already think are the right principles for thinking about interferences with people's rights.
[00:19:27] Emily McTernan: Even if you're a social media platform rather than a state.
[00:19:30] Jeff Howard: Even if you're a social media platform rather than a state. What's interesting here is that the analysis will just shake out differently, especially at the final stage. So I appealed to this idea of proportionality a moment ago to refer to the notion that the benefits produced by a particular restriction need to be sufficient to offset the costs and how that cost benefit analysis should be structured and how it will shake out given the different areas at issue, will look differently.
I think depending whether we're thinking about a company implementing a particular policy versus a state passing, say a criminal law. And that connects to our discussion earlier because it could be that certain restrictions, say restrictions on certain forms of harmful misinformation will be proportionate when implemented by platforms, but maybe won't be proportionate when enacted by states.
Why? Because we might really worry about giving government the power to manipulate public discourse right before an election, say in a way that we're less bothered in terms of social media platforms. So one would be justified and one wouldn't be, at least in principle, that's possible.
[00:20:28] Emily McTernan: So let's use your framework to think about a particular case of problematic social media content. So suppose I want to post something and have it seen by lots of people that says that climate change isn't manmade. What happens on your framework?
So kind of climate change denial, but not sort of dramatic, just saying, oh, there are natural fluctuations in the warmness of the earth. It's not because we've been polluting the atmosphere, it's just gonna happen anyway. What happens to my speech? How do, how does a social media company decide whether to demote it or remove it?
[00:21:00] Jeff Howard: It's a great example because it's not even a hypothetical example. So Meta for a long time had a policy for its Facebook and Instagram platforms whereby climate misinformation got a fact-check label appended to it, which would tell people that the content at issue was false or misleading.
[00:21:16] Emily McTernan: Which is different to the demotion strategy you're talking about, that?
[00:21:18] Jeff Howard: Yeah, that's right. That's right. So that's a separate issue. But any content that then got a fact check label received the demotion treatment as well. And so you had this fact check misinformation as a category of content that received the demotion treatment so that fewer people would were to see this content.
And the plausible argument for that kind of policy is that, look, it's not that you're breaching some weighty moral obligation you owe to others by airing some dumb view about climate change. However, when lots of people are posting that content and the system is amplifying and aggregating that content, it can have a corrosive effect on the rationality of our public discourse about climate change.
And so demoting that content is a way of preserving the speaker's ability to air dumb questions or dumb perspectives, or in this case, misguided perspectives, rather, that's a kinder of way of putting it, about climate change, but not sabotaging or hijacking the public discourse. So it's interesting because Meta, after the reelection of President Trump rolled back this policy in the United States and no longer demotes fact-check misinformation, there is a separate system in place called Community Notes, which has interesting potential advantages, which is about crowdsource fact-checking.
But that's just about the fact-checking bit stuff that gets a CommunityNote doesn't get demoted. So it's not quite the same system, but it's worth noting that Meta still has a policy on the books of demoting fact-check misinformation outside the United States. And so they can be held to that.
[00:22:40] Emily McTernan: Fascinating. So just talk me through the case using your framework. So you've got this kind of complicated framework from international human rights law. So here's my climate change denial post. The social media platform is meant to use your framework to decide whether it would be suitable, necessary, proportionate to demote it.
How does it do that? Like, does it have to look at the number of these kinds of posts it's having to deal with? Does it have to look at how irrational public discourse is? So right, so don't do it in the states anymore because it's not gonna make any difference. What's going on here in your framework?
[00:23:12] Jeff Howard: So I think we need to work through all the tests. So I think the legitimacy argument will involve appealing to the positive benefit that you're trying to achieve with this measure. So you're trying to preserve the information ecosystem and preserve the possibility of rational dialogue about climate change by removing baseless falsehoods from that conversation, or at least limiting their presence in the conversation.
The second is legality. This just means the policy needs to be announced clearly in advance and applied even handedly so that people know that their content is gonna receive this treatment. I think the outbursts or the problems that people had with so-called shadow banning wasn't even necessarily that their content was being demoted, it's that they weren't told that the content was being demoted. And of course, if you're not told the content is being demoted, then you can't think about whether the policy itself can be justified, right? So accountability requires legality.
In terms of suitability, that's a pretty easy one in this case, like the objective in question is to reduce the visibility of this content. By definition, demotion will do that. So suitability will be satisfied in this case.
The trickier tests come when we go to the fourth and fifth criteria, which are necessity and proportionality.
[00:24:11] Emily McTernan: Which is where a lot of the work is gonna have to go.
[00:24:12] Jeff Howard: So I think in a lot of cases the normative difficulty will arise at these stages, and I think a lot of proposed demotion policies might fail at these stages.
I'm inclined to think that removing climate misinformation would be an unnecessary step because insofar as the objective is just to preserve. Rational dialogue about climate change. You don't need to remove falsehoods in order to do that. You just need to prevent them from overtaking the discourse. That's my speculative conjecture about how this would play out in the real world.
And so I think what's striking is that a demotion policy will be able to satisfy that necessity test because I think it is almost by definition necessary to use demotion to reduce the visibility of the content. But it isn't necessary to remove, so removals will fail at that hurdle, but demotion will be satisfied at that hurdle.
[00:25:02] Emily McTernan: And, and presumably that will often happen with the kinds of problematic content you're talking about that demotion will pass, but removal will fail.
[00:25:10] Jeff Howard: Exactly. So take a post that promotes a certain form of aggressive intermittent fasting. So do we think that the speaker violates such a weighty duty that he or she owes to others that this content should be removed? I don't think so.
Moreover, it doesn't seem like it's necessary to remove the content in this case because the harm of this content plausibly is caused by inundating particular individuals with lots of this kind of content. If people just occasionally see a video encouraging intermittent fasting plausibly that doesn't lead to that much harm.
Now I say plausibly because all of this involves lots of empirical conjectures about the effects of different kinds of speech. And so really what I'm trying to do is lay out the normative criteria that platforms can use to help themselves structure their reasoning about these cases. But there's gonna be no substitute for social sciences' role in helping us understand the actual effects of different kinds of speech.
And so what that shows is that philosophy and law alone are insufficient to give guidance in this space. We do need empirical evidence about the actual costs and benefits of different measures and different harms caused by different kinds of harmful speech.
[00:26:12] Emily McTernan: Great. And proportionality, how does that play in? So we've talked a lot about necessity. That seems to be really, really important, right? It has to be necessary to do this. What's going on with your proportionality constraint?
[00:26:21] Jeff Howard: So proportionality comes in when we're thinking about the costs and benefits. So you could imagine a policy satisfying all the tests, including necessity, but you might think it's just too dangerous to give the decision making this power.
And so that insight or concern, I think, arises at the proportionality level of the analysis because that's gonna be a cost of the policy, that it's gonna entrench a certain decision making power that perhaps could be abused in the future. And so I think when we're thinking about why it's a bad idea in general for the state to have overly sweeping laws against hate speech, in my view that's because even if the law you write today is very narrowly tailored, it's gonna give the state powers that it could abuse in the future.
And so I think in some cases it will be a mistake to give decision makers the power to reduce certain content because it will be disproportionate. Now, as I was saying, I think this will happen much more when we're thinking about state policies than at the level of social media policies.
I think there will also be a set of cases in which the moral infraction at issue is a quite small one and so removing the content is just a disproportionately severe sanction. So you, Emily, are one of the leading scholars having written about microaggressions. Microaggressions and other forms of small offensive acts are by their nature, the kinds of acts that don't, in a one-off case constitute like a severe breach of moral obligation that the speaker owes to others.
It's a moral wrong, we can stipulate, but it's a quite small moral wrong. And in these cases. It may be that removing the content, which by the way for most social media platforms then means that there's a strike put on the user's account that could lead to their whole account being suspended so that they're not allowed to post anything. It's just a disproportionately severe punishment to such a small moral infraction.
However, if you're able to reduce the visibility of this content so that the kinds of people who are affected by it are less likely to see it, that will be more likely to pass the proportionality test. And what's striking about that example is it's possible that removals would also be unnecessary in that case as well, not just disproportionate, because as you and others have argued, the problem with microaggressions isn't the one-off exposure to it, it's the exposure to a stream of this content, especially over time.
And so insofar as demotion operates by arresting this pattern of content where people are constantly subjected to an endless litany of racist jokes, say, it can accomplish the relevant aim. Because I take it that the exposure to the occasional racially insensitive joke doesn't affect the kind of damage to someone's sense of equal standing as being bombarded with an endless concatenation of it and it's that bombardment that demotion seeks to arrest.
[00:28:58] Emily McTernan: Let's talk for a moment about the proportionality worry and what ends up happening in social media. Companies are making lots of decisions about misinformation. I'm wondering how much power this is giving to social media companies and whether you can accommodate that within your framework.
So thinking about your misinformation cases, you and I are both happy with the climate change one, but of course sometimes social media companies are gonna call it wrong about what is misinformation and what is accurate. So some people have claimed that happened during COVID with the case of the lab conspiracy case, right? That stuff gets demoted. COVID didn't come from a lab, and now some people think maybe it did come from a lab after all.
So, are you at all worried or can you accommodate in your framework the worries that people have about whether social media companies should be acting this paternalistic way, deciding what's right thinking for us and what's wrong, thinking for us, and then shaping our social media environment to make sure that we have right think not wrong think.
[00:29:47] Jeff Howard: I think it's an incredibly important point, and I think some of that concern is encompassed at the very initial stage of legitimacy. So it is not a legitimate aim to intervene in a ongoing public controversy about what is true and what is false. Announce the winner and then set out to manipulate public discourse so that disfavoured views receive less circulation.
[00:30:15] Emily McTernan: And just to be clear, some people who are on, I'm not, but some people are on the climate change denial side. They might say that is what it's doing when it demotes the climate change denial context.
[00:30:23] Jeff Howard: Totally. And I think everything will turn on what counts as a reasonable disagreement, I'm afraid. So it will turn on this question of what are the legitimate life controversies that we think it's incredibly important that citizens be free to carry out for themselves, and what counts as the kind of disputes over falsehood that we don't need to have a public debate about?
So if we go back to electoral misinformation. Electoral misinformation can get quite tricky, because of course, allegations of voter fraud are sometimes fabricated and baseless, but sometimes they're justified. And so actually maybe we want to err on the side of allowing people to make allegations of voter fraud, even though we know that allowance will be abused by certain actors.
But straightforward misstatements about the time and dates of elections, or who's eligible and not eligible to vote, it's not clear we need to have a live ongoing discussion about whether the vote is on Tuesday or not when we know the vote really is on Tuesday.
So I think there are going to be a range of cases, both in the electoral misinformation context, in the public health information context, and in some other context as well, we can think of financial fraud in the space as well, where I think the decision maker can confidently assert that the claims at issue really are false, and they also need to show that they're harmful in some way. It's not enough that they're false, right? Flat-earther content is pretty innocuous, I don't think platforms should be banning that or even demoting it necessarily, although that's an interesting case.
And so I think that we need to be really careful in the misinformation space for exactly the reasons you lay out. We have powerful free speech interests in sorting out our disagreements about what's true and what is false, rather than having to have decision makers, like social media companies intervene.
But in cases where the decision makers have a very high level of confidence, say, because all of the relevant qualified experts or nearly all of the relevant qualified experts agree that a certain thing is true or a certain thing is false, I think it can be justified to act.
But that's why legality is so important. It all needs to be announced in advance. It all needs to be provisional. It needs to be subject to revision because of course we're gonna get things wrong sometimes.
[00:32:21] Emily McTernan: Let's touch on this transparency thought that you've had throughout. So you say it's really, really important that we can tell in advance what kind of content is going to get demoted. It's great that you can go and Google and find out what Meta demotes or what other kinds of sites demote, and I was wondering that might look okay as a content poster.
Do you think it's enough as a content user? So if you are told, oh, this kind of content gets demoted and yours has been demoted, that might look like enough transparency for the poster. But as someone who's just a casual user of the site, is it transparent enough to you what you're going to get shown and what's being hidden from you?
[00:32:55] Jeff Howard: This raises really interesting questions about the way in which excessive notification or excessive transparency can impair users' experience. Imagine that you were notified every time you were not showed a piece of content that you would have shown, but for a demotion, I think it would be technologically feasible, I think, to design an architecture where people were constantly seeing blank bits of their feed where it said 'You would've seen some climate misinformation here, but we demoted it'.
Okay, so you could do that. And the question is, can users reasonably demand that? And I do think platforms do have legitimate commercial prerogatives in this space as well. And it is a little unclear how these commercial prerogatives fit within the international human rights law framework.
And Meta's Oversight Board is, has itself expressed cry for greater guidance on exactly this point, because I don't think it's entirely obvious to people how the fact that these are private companies with their own commercial incentives feeds into this overall framework and it's something I'm trying to think about in other work, but I think that you could imagine the experience of social media being substantially impaired if people got constant notifications, if it even were technologically feasible to do that.
There is an interesting question on the speaker side though, about do we think that speakers are entitled morally to individualise notification every time their speech gets the demotion treatment? And some companies have said to me, they think that goes above and beyond what they owe speakers, that it's simply enough to tell them the general policy.
Now, I don't think we'd accept that in the removals case. I think we would expect people to be notified individually. And so I'm minded to think that at the very least in the settings, you should be able to go in and, you know, click three dots and go under the hood and see did this post of mine receive the demotion treatment or didn't?
And I think because of the free speech issue, we have a right to that information when our content is demoted on the grounds that it's harmful or wrong or problematic, but we're not entitled to it when our post is just ordinarily de-amplified because it's predicted to be boring.
[00:34:59] Emily McTernan: And why is that? Why is there such a difference between boring and wrong?
[00:35:02] Jeff Howard: I think it's because it expresses a substantive judgement in one case, but not the other. I think that when Netflix declines to recommend certain horror movies to you, but recommends them to me, there is no substantive judgement that horror movies are good or bad.
There's rather just an attempt to show people more of the content, that's if you're a cynic, is gonna hold their attention, or if you're charitable, is gonna keep them engaged and show them the kind of content they want to see. And similarly, when I see standup comedy content on TikTok, but not videos of people gardening, there's no accompanying normative judgement about the harmfulness or badness of certain kind of content.
But when, say, pro-Palestine content gets demoted on the grounds that, ooh, this content is problematic, that triggers a normative issue that I think these other cases don't.
[00:35:48] Emily McTernan: One last worry. Someone might worry about demotion as a strategy, still. I mean, so one way of putting this might be isn't demotion either ineffective, the content's still out there and it still does harm even on a smaller scale. So you imagine the person who's prone to an eating disorder, they have to go and search out the extreme fasting material, but it's still there.
Or if it's not ineffective, it's a form of de facto removal near no one sees it. It's hard for that person even to search it out. And what really then is the difference to banning it? So to come back to the state comparison, which you're keen on, you can imagine a totalitarian state saying, Hey, you can write, you can even publish that book. It's just, there'll only be a handful of copies and we'll put them in the bunker. I've demoted your speech. Still looks like a violation of free speech, or at least a culture of free speech.
So is demotion really better than removal? Looks like it's either ineffective or I make it stronger and it is effective now, but it looks very like banning it or removing it.
[00:36:42] Jeff Howard: It's a forceful point, and I think it illustrates that demotion is a scalar notion that comes in degrees: the more aggressive demotion is, the closer it is to removal. And we can imagine a version of demotion that is almost as severe a sanction as removal. And so our analysis needs to be sensitive to the different forms of demotion.
So I've skated over all these differences for the sake of discussion, but content can be down-ranked, so the system is less likely to recommend it to others. And you can have different levels of down-ranking, right? So you can have complete exclusion from the recommendation list, or you can simply have it so the content shows up less frequently in the recommendation list or that it's lowered down so it's not at the top so you have to scroll for a bit if you really wanna see it.
Content can be like delisted, so it's not shown up in a search, or it can be down-listed so it does show up in the search. Or you can just make it so that content can only be seen or will only be recommended to the people who follow that account, but not to people who don't. So there are loads of varieties here and I think what's gonna be the right remedy is gonna depend on the exact objective in.
But I totally take the point that the point of demotion is that people are allowed to post the content. People are allowed to find it if they really want to. It's just that we're not going to actively amplify it in a way that allows it to bombard our discourse.
[00:37:54] Emily McTernan: What's the one thing you'd most like a social media company to do in response to your moral framework that you've offered them? If they had to make one change, what would it be?
[00:38:04] Jeff Howard: I think social media companies have improved a lot over the years in how they protect teenagers from self harm and eating disorder content. But we still have an epidemic on our hands in this space. Suicide and self-harm are still at staggeringly alarming levels, and so I, I think social media companies really need to lean into this space even further. And I think demotion is the right remedy.
And you raise the worry about paternalism earlier. Paternalism is a particularly forceful moral wary when adults are at issue, but when children are at issue, it's pretty uncontroversial that various adults have a duty to be paternalistic toward children. And so I think platforms need to take really seriously their demotion responsibilities in this space.
[00:38:48] Emily McTernan: So you're not as supporter of the thought that the state should just simply ban social media access for the under sixteens.
[00:38:53] Jeff Howard: I think it's an interesting experiment to see what happens in Australia. I like this natural experiment. So we have some countries that allow it and introduce safeguards like the Online Safety Act in the United Kingdom, which I think has a lot of merit to it, and then we see what Australia's doing, and then we can see like, which is better being a kid, being a 15-year-old, say, in the United Kingdom versus a 15-year-old in Australia. And we can compare and contrast these experiences. So I, I don't think we should all rush to ban it. I think we should try to learn something from the experiment.
I mean, I tend to be persuaded by Jonathan Haidt's argument that the biggest cost of social media for children is actually the opportunity costs. People should be outside engaging in, as he puts it, one-to-one or one to a few synchronic real world offline interactions. And that's really how we develop as human beings rather than sitting at home posting a picture of yourself on social media and waiting to see how many people like it, that's no way to spend a childhood.
[00:39:45] Emily McTernan: Thank you, Jeff. Today we've been discussing Jeff Howard and Beatriz Kira's paper Remove or Reduce: Demotion Content, Moderation and Human Rights just out in the journal Law and Philosophy. As ever, you'll find full details in the show notes for this episode.
Remember, to make sure you don't miss out on future episodes of UCL Uncovering Politics, all you need to do is subscribe. You can do so on Apple, Google Podcasts or whatever podcast provider you use. And while you're there, we'd love it if you could take a moment of your time to rate or review us too.
I'm Emily McTernan. This episode was produced by Matthieu Dinh. Our theme music is written and performed by John Mann.
This has been UCL Uncovering Politics. Thank you for listening.