Evelyn and Alex discuss how China’s censors are struggling to contain information about the ongoing protests, how they are going on the offensive, and why that offensive includes an image of Alex with horns.They also, of course, check in on Twitter with the weekly Elon Musk segment, and discuss US-backed information operations.
Stanford’s Evelyn Douek and Alex Stamos weigh in on the latest online trust and safety news and developments:
Join the conversation and connect with Evelyn and Alex on Twitter at @evelyndouek and @alexstamos.
Moderated Content is produced in partnership by Stanford Law School and the Cyber Policy Center. Special thanks to John Perrino for research and editorial assistance.
Like what you heard? Don’t forget to subscribe and share the podcast with friends!
Alex Stamos:
He joked it was Elon Musket. I mean, come on, but like the guy's having a breakdown, because it also has four cans of caffeine-free Diet Coke, which is the greatest call for help.
Evelyn Douek:
Yeah, exactly.
Alex Stamos:
I feel like, if those are four cans of Diet Dr Pepper, you could 10-50 him now, you'd have the orderly show up and check him into a mental institution.
Evelyn Douek:
Welcome to the Weekly News hit episode of Moderated Content with myself, Evelyn Douek and Alex Stamos. And this Thanksgiving week I'll start by saying I'm thankful to Elon Musk for keeping us tech podcasters in business. But before we jump into our regular Musk segment, I think the story dominating the headlines and quite rightly one of the biggest stories in the world right now is the protests in China where very brave protestors are coming out in droves to show rare descent and protest against the Communist Party's zero COVID policy.
This is obviously a huge geopolitical story, huge political story, but it is of course also a content moderation story and a story about how China's censors are struggling to keep up with the content that these protestors are producing in the way in which they're trying to get the word out. Obviously mainstream media hasn't been covering at all, but we know about it because there are videos and footage being posted on social media. And then of course there's also the way that China has been on the offensive to do with these protests. So Alex, what are you seeing and what's your big takeaway here?
Alex Stamos:
Yeah, so a really important time for the Chinese people. We haven't seen unrest in the People's Republic since I think Tiananmen Square 1989 student uprising of the scale triggered by COVID lockdowns. But it's clearly taken on a broader dimension. One of the ways that these protestors are protesting is by holding up blank pieces of paper, which is a pretty clear metaphor for online censorship and that they feel like they can't speak about these things and are finally feeling safe doing so. And some incredible videos coming out of people confronting the police, of yelling at police officers, of the ground level civil disobedience that you pretty much never see out of the people's republic.
And like you said, the censors are having trouble catching up. One of the things this highlights is over the last decade of internet censorship, Chinese people have gotten pretty good at evading that censorship in real time. They come up with different words, different language. Chinese is a very metaphorical rich language, and so they come up with kind of metaphors or terms that to a Chinese speaker clearly is about something that is banned and they move very, very quickly. And so that has been true for a long period of time, but we're seeing that now in overdrive as those kinds of capabilities and skills that the Chinese people have honed are putting the censorship capability of the people's Republic to the test.
Evelyn Douek:
Right and I will link in the show notes to a great thread by Paul Moser on this about the various tactics that the Chinese people are doing to evade censorship. But it's things like video is an especially hard kind of content to track and take down and because of the hashing technology that gets used in order to, if the censors find a video and they want to remove that from the rest of the internet, they give it a digital fingerprint and then any uploads are marked against that. And so what people do and what the Chinese people are doing is taking a video of the video so that the image is different enough that evades those filters, they know all of these kinds of tricks to evade those automatic filters.
And Paul has some great commentary on that. And he's just saying it's also the volume of this kind of thing and the speed of change and the things like they're using parody, like they are having people sing patriotic songs and waving Chinese flags, but doing it in a way that doesn't trigger keywords but is obviously meant to criticize. So it's an amazing moment and obviously has, we'll watch and see what happens in incoming days. And as we said, there's not only this censorship side of how China is battling this out online, but there's an offensive side of how they're trying to make sure that the message about the protest doesn't get out. Alex, do you want to talk about that?
Alex Stamos:
Yeah, so the PRC has propaganda and censorship capabilities both internally and then propaganda capabilities externally. And that external capability has traditionally been in the hands of what's called the 50 Cent Army of patriotic individuals who are motivated and given talking points to go out and spread the word. What we're also seeing is a tactic that has been seen in the past but is being used in a more broader than we've ever seen before, is the creation kind of information shaft of the PRC trying to flood the zone for hashtags and for search terms about the Chinese protests to make it effectively impossible for people to find good information. Now they're doing that on platforms outside of the PRC that are blocked to Chinese citizens. And so it's pretty clearly not about their internal protections, which are going to be focused on WeChat and Webo and such, but more about trying to keep the external media from being able to see these videos that are leaking out from people who are using Tor, using VPNs and are sneaking these videos out and then getting them posted to social media.
Evelyn Douek:
I just want to point out, this is such a good example of how more speech doesn't always necessarily mean better protections for free speech. There's this very simplistic understanding of free speech and first amendment that dominates a lot of these content moderation debates where it's just like leave up all the content and that is the best free speech ecosystem that you can have. Here's a great example of how free speech is being weaponized, that the algorithms, the platforms are being gamed by just posting stuff that isn't really free speech in any meaningful sense of the word, but drowning out important political content.
Alex Stamos:
Well, and most of it seems kind of porny ads. One of the interesting things, there's been some good analysis. One of the analyses I raised up was by an account, a pseudo anonymous account called Air Moving Device on Twitter, who is, despite not having his name on there, is actually a very reliable person. I can speak as to the accuracy of their analysis in which they looked and some of the evidence that they posted really points to these being spam accounts that have probably been active in the past that have been dormant and all of a sudden have woken up, which is a tactic we've seen from the PRC in the past, which is they will buy accounts on the black market or they will enlist Chinese spammers in pushing stuff. The other thing they're doing is attacking anybody who is not towing the line on China, including myself.
I got a fun response where somebody created, it looks like a stable diffusion image of me with horns, like dope horns coming out of my head, sitting in front of a computer. Pretty funny in this case, but lot less funny in other situations where traditionally, especially if you're a woman and you speak out against the PRC, you'll get rape threats, you'll get stories made about you. There's some great examples of Chinese women who have left the PRC and who are critics from the outside, who have really extensive kind of trolling as well as campaigns to defame them around the world and to reduce their voice. So like you said, not a situation you can fix with more speech. If you're going up against an organized adversary that cares nothing for the truth and has the ability to run tens or thousands or hundreds of thousands of fake accounts, then more speech does not fix the problem of them suppressing speech, trying to drive people off the internet using their own speech.
Evelyn Douek:
It's a truly awesome image. We're going to have to use it as the episode art, or at least included in the show notes, just so that I can feel more badass by association with such a cool demon overlord. And so as you predicted, this is I think probably a good segue moment to our regular Musk and Twitter update. That signifies the beginning of our weekly segment. So you predicted this, Alex, you said one of the big risks of Musk getting rid of all of its content moderation and trust and safety people is helping foreign influence operations and lo and behold.
Alex Stamos:
Yeah, this is pretty obvious one. One of the things I said the day he closed was that Musk was buying himself into a huge geopolitical problem in that he has most of his net worth is tied up in a company that has lots of international exposure. The most important country other than the United States for Tesla is China. And so Musk has now put himself in a position where he is responsible for deciding how much the PRC is going to be able to manipulate Twitter, and he's doing so with almost no control. So Casey Newton on a podcast called Musk's kind of amnesty of bad accounts, the joker freeing all the inmates of Arkham Asylum. Well he's done that, but the Gotham PD has basically all quit or been fired, so there's nobody left to put the inmates back into Arkham Asylum. It is just Musk as Batman, except in this case, Bruce Wayne owns most of his money in Joker enterprises, so is quite conflicted about what he's supposed to do.
But in all seriousness, the threat intel team, the people who are like ex State Department, ex Intelligence, ex Mandiant, CrowdStrike kind of people who spent a bunch of time studying how governments manipulate platforms who Twitter built over a five year period. It took a lot of money and a lot of time for them to build this team evaporated in effectively a week. There's nobody there left who understands this kind of stuff. And we don't even have a contact at SIO. Every single person we used to interact with is gone. When we find these accounts we don't even know who to send it to. I guess I'll just put it on Twitter and tag Musk of here's a network of PRC.
Evelyn Douek:
That's everyone's favorite content moderation strategy and remedy these days. So why not?
Alex Stamos:
Right. Right. And so without that team, this is the kind of thing that you're open to. And yeah, that was just an interview I gave last week about it being open season and it did not take long for that to happen. And I think it is a bad sign because it is clear that they don't have the skill set in house anymore to battle these kinds of things. Musk is going to have to invest incredibly quickly to either bring back some of those people or build a completely new team. I can't imagine somebody who does that kind of work wanting to work for Musk with the kind of things that he's saying.
If you are completely mission driven and your goal is I'm not going to let the Chinese Communist Party control Twitter, going and working for Elon Musk is a very kind of personally morally ethical, complicated thing to do. And so I don't see this getting better and Musk I don't think understands because he keeps on bragging about how many new accounts are being created on Twitter. Well, yeah, I mean if you stop shutting down spam accounts and if you're advertising your open season for manipulation, lots of people are going to create new accounts. He clearly, despite being the guy who complained all summer about Twitter accounts being fake, he clearly has very little understanding of what the dynamics are here.
Evelyn Douek:
I mean, he just seems to like everything else, find it funny, posting a bunch of, he appears to have learnt the word psy ops in the last couple of days and is just tweeting a bunch about it and saying the amount of pro psy ops on Twitter is ridiculous. At least with the new verified, they'll pay $8 for the privilege, haha. So I don't think this is as yet a problem that he appears to think is serious or should be taken seriously.
Alex Stamos:
No, he has not said anything about the Chinese op. In fact, he's just tweeting about totally. I mean he looks like he's having a mental breakdown to be honest. He's tweeting Pepe the Frog images, he tweeted a picture of what looks like a replica gun. It's like a sci-fi replica gun, but what looks like a gun and then a real musket, which looks pretty cool actually. I wish I could afford a Revolutionary War musket to have on my bedside and then a bunch of-
Evelyn Douek:
It's just the most American thing ever, but okay, carry on.
Alex Stamos:
Right. Right. Well it's pretty cool. Okay, I was going to say, I wish I had that musket. He joked it was Elon Musket. I mean come on dude. But like the guy's having a breakdown because it also has four cans of caffeine-free Diet Coke, which is the greatest call for help.
Evelyn Douek:
Yeah, exactly.
Alex Stamos:
I mean, I feel like if those are four cans of diet Dr Pepper, you could 10-50 him now. You could have the orderly show up and check him into a mental institution, but this is clearly on the road to the diet Dr. Pepper.
Evelyn Douek:
That's right and so when I get too stressed or overwhelmed, I go and hide under the covers. Musk, meanwhile get himself into a giant battle with Apple because why not? So that is actually happening as we record. So who knows where this will be by the time you're actually listening to this. But yeah, set the stage, Alex, between these, the battle between these two check giants that is playing out right now.
Alex Stamos:
Okay. Yeah. So this is going to be really fascinating because both sides have actually a pretty good point here. As you and I discussed last week, Uli Roth, upon leaving Twitter writes in about one of the last things that might stop Elon, because he apparently doesn't care about advertisers. He's blown through caring about revenue, which means that he is just going to be subsidizing Twitter as a hobby by selling Tesla stock, which he could do effectively indefinitely at this point. So if it's not advertisers, who's left? It's Apple and Google and their app store, and you and I agreed that we did not think that, that's an appropriate way for limits to be placed on Twitter. People can go listen to the last episode for a more discussion of what are the appropriate choke points for content moderation. And neither Evelyn or I thought, I believe that's still your position, right? That's not the right place.
Evelyn Douek:
Right. Yes.
Alex Stamos:
Okay. So Musk is now aware of this. It seems clear that he was not aware that this was a possibility until Roth wrote his op-ed. And now it looks, reading between the lines of Musk's tweets, it looks like Apple actually probably back channeled to Twitter and did say that they need to get things cleaned up to a certain extent. We know nothing about the details of that because now Musk has gone on a total tear against Apple and censorship. Okay. So this is where it's interesting because the truth is, is that Apple has done more for the people's Republic of China and the Chinese Communist Party than any other US tech company has done for any authoritarian state or party. That's just true. Apple sensors iPhones in the country, they have the iCloud backups, which are decryptable going to servers controlled by a joint venture of which is controlled by the Chinese Communist Party.
They keep VPNs from being installed, they keep apps being installed. Apple does horrible things for the PRC. So that is, to say that Apple is censorious is actually true. But the funny thing is they do it on behalf of the PRC who are the people who are manipulating Twitter right now and who Elon has yet to criticize. And so he's walking a really fine line here because if he wants to bring up the legitimate kind of human rights crimes of Apple, he's also going to have to criticize the country in which he is getting a quarter of his revenue and he has built a massive factory relying upon the suppression of local labor unions and such by the Chinese Communist Party.
Evelyn Douek:
Right. And he has legitimate points as well about the app store and the way that Apple runs content moderation too. Like he is now currently running a poll about should Apple publish all censorship actions it has taken that affect its customers. And he is tweeting out people who are complaining about restrictions that Apple has placed on their apps in response to his tweets. And he's also got a point there that Apple is very opaque about the rules that it exercises and the power that it exerts over the apps on its stores. And we saw this in the wake of January 6th when we've talked about this before, kicked off Parlour and things like that.
And so yeah, I'm for more transparency in the app store and the rules that they have and how they apply those apps. Occasionally Elon Musk comes around the other side and we find things that we agree on, and he's also tweeting about the secret 30% tax on everything that you buy through their app store, which we were talking about last week as well. It can be problematic. And so do you have a view of how this is going to play out? I mean, do you think that Apple would take the step as to kick them out?
Alex Stamos:
I think it's a huge step. I mean, Apple's attacks against other tech companies have been very carefully calibrated to win the PR war. The most incredibly monopolistic and economically destructive thing that Apple has done is the changes to advertising rules on their platform that they do not apply to themselves. So they came up with these rules for the quote unquote, privacy of users, that massively hit Facebook and Twitter and Google and a bunch of other companies. Apple does not apply those rules to themselves, therefore they stole a huge amount of revenue from these companies. But they did so in a way over a multi-year period that they were celebrating the press as being pro privacy. They completely hoodwinked kind of the privacy industrial complex of people who write about this kind of stuff and think about this kind of stuff who never really, there's a lot of privacy voices out there that don't think too deeply about these things.
They're just some companies they really hate and they love to hate on those companies and they never actually look at what are the actual harms to individuals and why are people making these decisions? And so they didn't ask very many questions and Apple was able to do that, but that was a carefully calibrated multi-year campaign for them to do so. To get in a fight with Musk right now by de-platforming Twitter is not in their control, is exactly the kind of thing that I think could really hurt them when they're in the middle of defending their app store monopoly and their 30% cut.
And so I think it is highly unlikely Apple drops Twitter just because what they really care about is the ability to get 30% of every purchase made that they want to own all of the economic flow and kicking Twitter out, if that creates all of these enemies on the right where they haven't had it before from a antitrust perspective, and you combine those people with the Warrens and the Klobuchar on the left, you can actually see something come out of Congress or definitely FTC action just as there's EU investigations going on right now. I can't imagine that they want to open that door. If they're going to get back at Elon, it will be Tim Cook is the master at the slow knife.
Evelyn Douek:
That is so disappointing because my favorite part of this whole saga is that Elon has said that if he does get kicked out of the app store, he will build his own phone, which I think is just fantastic. I will carry around my iPhone and oh my Twitter phone just to make sure that I can regularly refresh my feed. I mean, Alex, would you buy an Elon Musk Twitter phone?
Alex Stamos:
No. Look but, I mean he's right. Once again, Musk is right here. This is something that Zuckerberg understood about a decade ago was that when Facebook was making the shift to mobile, that he was putting himself completely at the mercy of Google and Apple. And at the time Google had a direct Facebook competitor. Apple wasn't really competing at the time, wasn't running an ad network, was seen as more neutral. In the end it turned out to be Apple that was the much more monopolistic in their use of the app stores than Google. But in either case, Zuckerberg had the right idea. He spent something like a billion dollars to try to create a Facebook phone and ended up never shipping.
And so I think, I mean Musk could do it, he could definitely get Tesla hardware designers, get some Taiwanese OEM to do the manufacturing for him, fork Android, which is what most of these kind of these special phones are. They're just Android forks that don't run the Google app store, that run their own app store. Nothing stops him from doing that. But being popular and being, figuring out what makes that actually valuable is hard. And the phone market is a brutal market, especially the Android market is very low margin. It is highly competitive. Samsung and Huawei and Google with the pixel phones, they're all killing each other in that market. So it's just another bad business for him to get into and to waste his billions on.
Evelyn Douek:
Well, at the end of the day, if it doesn't pan out, he can just use his Teslas. So you have to buy a Tesla to get your Twitter and just plug in and you get extra retweets for going, for driving accurately around the corner. I don't know. Okay, so let's move to back to government influence operations. We were talking about China, but Facebook Meta released its official report this week of the adversarial threats that it seen over the last quarter, and it had the official update on a campaign that it had taken down that it said had ties to the US government. Now Alex, you worked on this at Stanford Internet Observatory so give us the overview there.
Alex Stamos:
So our team published a couple months ago a report alongside Graphica called Unheard Voice, which was an analysis of a take down between Facebook and Twitter, that Facebook and Twitter did together, but actually touched a bunch of other platforms of an influence operation that was clearly pro Western, that had some technical links to the US Department of Defense and then was confirmed by Ellen Nakashima with her Washington Post reporting to be the work of a defense contractor that was hired by the Pentagon. So big deal, this is a straight up kind of Russian style influence campaign, fake accounts operating around the world, but targeting places like Iran and such, just pretending to be people who they aren't pushing propaganda. Yeah, I'm not going to use the term fake news. Some of the things they say are not true, some of them are true, but just like slanted, it's kind of your standard propaganda campaign and doing so over a multi-year period. One of the things that we found is that they're actual, they're really bad at it.
So it turns out when the US tries to act like Russia, in this case, we suck at it. Whoever exactly ran this, and I have some theories about which contractor, we can't say for sure, but there are some open contracts in the DOD database that are hinting towards, they might be subcontractors of those contracts, but whoever did it did a really crappy job. The first tweet we did about our report ended up getting more engagement than the entire network over five years. Right. And so DOD did something really stupid here, which is they basically justified Russian influence campaigns. They created a report that is now, it's our names on it, and I feel bad about this, but we just have to tell the truth. Our report gets cited now by all the time I see the links of Iranian, Chinese, Russian propagandists saying the US does it too.
Like the people who really manipulate the internet is the US. I don't know if Glen Greenwald's talked about it -yet, but it's exactly the kind of thing that he would blow out a proportion without ever talking about any other SIO research about Russia, for example. And so they created kind of comfort for America's enemies. They did something that was really morally and ethically wrong, and they had no actual upside in doing so. Effectively like the dumbest possible campaign. And Facebook effectively included this in their quarterly report and verified that their attribution goes back to the US government. My understanding is that this caused, our report caused kind of a stir in the Pentagon. I've heard about some meetings that have happened that perhaps now there will be some real controls that the people who did this, it wasn't really understood at the top. A lot of this work started in the first Obama administration and maybe even the Bush administration. And so that there was finally discussion that DOD should have some policies around not allowing this kind of stuff to happen in the future. But we haven't seen anything published yet.
Evelyn Douek:
Right. Yeah, no, I mean, I couldn't agree more that I think this is profoundly damaging. It's also not new. This is not like this is the first information operation run by a democratic state. I mean, America has been running influence operations for as long as there's been information as well. And France was also caught by Graphica a couple of years ago, or maybe the year before last year, doing a similar kind of thing in the Central African Republic. And I think it's just profoundly damaging because as you say the authoritarian states point to it as justification or as retaliation.
And it creates a world that I don't think is the world that we want to live in. If we believe in free speech as transparency, as if we believe in the idea that the public debate should happen and the value of free speech is self governance it shows our hand or lack of faith in that if we give in to the worst kind of instincts and play that game. So yeah, it sucks that you're being held up for those reasons, but I couldn't agree more that it's the right thing to do.
Alex Stamos:
Yep. So we'll continue to do this when we can find these kinds of things. My hope is that we don't because the US stops doing this stupid stuff. And also it's just a great demonstration of US government waste that somebody got paid tens of millions of dollars to do something that's much more amateur than what I could have done with a couple of undergrads.
Evelyn Douek:
Ouch. And with that, anything else that you want to cover before we close out for the week?
Alex Stamos:
Nope. Hope everybody had a happy Thanksgiving. And yeah, David Shaw retired as Stanford football coach. Unfortunately, I know this is not a college football broadcast, but I think this is the end of an era for academically competitive schools being also football competitive, that we're entering a new era of college football. Maybe we can do an entire hour on that, on the NIL and the open transfer portal.
Evelyn Douek:
I mean, I clearly have lots to learn. Am I, I'm happy or I'm sad about this retirement by the sounds of things?
Alex Stamos:
I don't know. I mean, I think you don't care, is the truth.
Evelyn Douek:
Okay. That is true. I just want to know how to fit in. Alex, tell me my feelings. All right and-
Alex Stamos:
Are you watching the World Cup? Do you have a World Cup team?
Evelyn Douek:
I mean, come on, the Socceroos, of course, Aussie, Aussie, Aussie, oy, oy, oy and yourself.
Alex Stamos:
I'd love to see the United States make it to the second round. If they can't I think the English, England's, the UKs having a tough time right now overall, so I think it wouldn't be a bad time for them to get a little bit of a win.
Evelyn Douek:
I love that sort of very realistic expectation setting rather than aiming too high, just sort of middle of the bar. And that concludes our regular sports section of Moderated Content, which is becoming a weekly feature. I hope you all enjoy this high level sports analysis. And that is also the rest of Moderated Content for the week. This show is available in all the usual places, including Apple Podcasts and Spotify. Show notes are available at law.stanford.edu/moderatedcontent. This episode wouldn't have been possible without the research and editorial assistance of John Parrino, Policy Analyst at the Stanford Internet Observatory, and it is produced by Brian Peletier. Special thanks also to Alyssa Ashdown, Justin Fu and Rob Huffman. And see you next week for more Moderated Content and in the spirit of Thanksgiving, thanks for listening.