The 404 Media Podcast (Premium Feed)

Joseph: 00:03

Hello, and welcome to the four zero four media podcast where we bring you unparalleled access to hidden worlds both online and IRL. Four zero four media is a journalist founded company and needs your support. To subscribe, go to 404media.co. As well as bonus content every single week, subscribers also get access to additional episodes where we respond to the best comments. Gain access to content at 404media.co.

Joseph: 00:29

I'm your host, Joseph. And with me are two of the four zero four media cofounders, the first being Sam Cole

Sam: 00:35

Hi.

Joseph: 00:36

And Emmanuel Mayberg. Hey.

Emanuel: 00:38

I'm back. I feel like I haven't been here in many years.

Joseph: 00:41

You come back. Jason's now gone. Who knows next week. Speaking of Jason though, he published what is, you know, our first short documentary. It's called how artists are keeping the lost art of neon signs alive.

Joseph: 01:01

I've put a link to the YouTube. There's the video on YouTube in the show notes. There is an accompanying article as well. But this is exciting stuff. Right?

Joseph: 01:11

We obviously, before we launched four zero four Media, we all worked at Vice that was, you know, infamous, I would say, for its documentaries. We had the pleasure of of working on some of them. I definitely didn't do a lot on it. I think all of you did more than me. But now that we run the company, we can start to experiment and branch out and and dabble into doing documentaries ourselves.

Joseph: 01:45

And I don't know. That that's I think that's a very, very exciting time to be in. Emmanuel, is it I know obviously you haven't done a documentary with four zero four yet, but like, it's gonna be pretty different moving from a big company like Vice all those years ago, where there were teams upon teams of people to do this to doing it ourselves. Like and I feel like you touched it maybe more than me or maybe I'm misremembering. What do you do you make of the us doing it now?

Emanuel: 02:13

I think Sam actually has done the most both in helping the video teams make their own videos, but also she hosted a video, which I hope she can talk about. I'm really excited, you know, Vice did really good videos, we supported them, Motherboard made really amazing series. One that I really liked that Jason worked on very closely was about right to repair, super viral video about how farmers fix their tractors and such. I hope that we're able to match that level of of quality. It'll be hard.

Emanuel: 02:52

We're we're it's gonna take a while to to hit that, but we we have enough support support now from our audience to at least try to to do that, and we we obviously have plenty of interesting worthwhile subjects to to make videos about. So I'm I'm stoked.

Joseph: 03:10

Sam, what did you work on video wise back at Vice slash Motherboard?

Sam: 03:16

I did an episode of Cryptoland, which was my episode was about crypto being used for sex work. So I went out to Vegas and went to, like, a content house, basically, for crypto, and it was really fun. It was cool. Yeah. Obviously, video documentaries are really hard.

Sam: 03:39

It's a complicated process. So I'll be interested to see how it goes now that we're dipping our toe into doing our own. So but, yeah, Jason's I mean, the neon documentary is really good. You should go watch it. It's really interesting.

Sam: 03:53

The main character in it, Gio, is such a character. So I wanna go hang out in his shop in LA.

Joseph: 04:00

Yeah. I mean, it's great. And when we say we're doing it ourselves, it is very much that Jason took the pod car sorry. The camera he usually uses for this podcast and went and filmed this, you know, short film. So that that's literally what we're doing.

Joseph: 04:18

And of course, as we get further into it, I'm sure maybe we'll we'll bring help on and outside contributors and that sort of thing. And that's all very much TBD down the line. But hey, it's just an exciting time that a couple of years after launch, as you say, we have the support where we can diversify and start to explore different stuff. And I definitely have an ideas of what I want to try to do. Alright.

Joseph: 04:42

Well, for this week, Emmanuel, do you wanna lead us through this first story?

Emanuel: 04:48

Yeah. So our first story is from Joe. The headline is hackers dox hundreds of DHS, ICE, FBI and DOJ officials. So Joe, what is this data exactly and what is the scale of the hack or or or docs or whatever you wanna call it? Like, how big of a problem is it, would you say?

Joseph: 05:12

Yeah. So the data itself, this first story we're talking about, it's mostly DHS. And, of course, that includes ICE well. And then there's some FBI and DOJ as well. But it's something like nearly 700 DHS officials in total and that includes their official email address, which, you know, helps verify that they're a government employee, their name, a phone number.

Joseph: 05:38

Sometimes that looks like a personal number. Sometimes it looks like an office number but there's definitely some personal ones in there. Then addresses. And again, this is also sometimes some sort of government facility, but the ones I looked at, they sure look like residential addresses when you put them into Google and a Zillow listing comes up and it's like, oh, I I don't think that's an official DHS facility or something like that. So it's straight up docs.

Joseph: 06:03

And, obviously, that is a lot of personal data that people could do various things with. And in this case, it landed in the hands of a group of hackers who are financially motivated, I would say. And I'm sure we'll get into that as well.

Emanuel: 06:23

So the first thing we do when we find out about a leak or docs like this is we try to verify that it's accurate. Otherwise, we wouldn't report on it. How did you do that for this story?

Joseph: 06:35

Yeah. When a hacker reaches out and they say, have this data, it could be absolute BS. You have no idea if they've fabricated it and they're trying to do it for clout or prestige or maybe just to troll the journalist or whatever. We've had it before where hackers targeted tele message that signal clone that the Trump administration uses, and we broke news of that hack. Way back then, there was a bunch of data about customs and border protection officials in there, like phone numbers, that sort of thing.

Joseph: 07:08

In that case, I just phoned them up. Asked them, hey. Is that blah blah blah from CPP? And they say, yes. And then I say, my name is Joseph Cox.

Joseph: 07:17

I'm a journalist. Blah blah blah. And then usually, they hang up immediately when I start, you know, actually speaking. So I did that way back when and I could have done that for this. But I actually just felt like doing a different approach, which is that I took this docs provided by the hackers and then I turned to data provided by a company called District four Labs.

Joseph: 07:40

They have a search tool called DarkSide, and I log in. And basically, it brings together all of these different data breaches from all over the Internet. So back in 2016, I remember that there was a wave of breaches, LinkedIn and all of these other ones. And if you were a journalist or a cybersecurity researcher, you had to go and download that data then search yourself, and maybe that was useful. I seem to remember some stories from the time where, oh, I I need to verify who this person is for this story a couple years later.

Joseph: 08:15

Oh, let me go through the data breach of x y z company. You don't need to do that anymore because companies like District four Labs exist, and they just bring it all together, and then they allow researchers or journalists or threat intelligence analysts or I presume government as well. It allows them to log in, and you can just search all of these databases at once. So I started taking phone numbers from the DHS and the ICE docs, put them into this tool, and very quickly, it was apparent that, oh, yeah. These are real docs of ICE and DHS officials.

Joseph: 08:51

There were some interesting cases where I would put in the phone number and would bring up the matching DHS email from a breach of a parking app and that shows, oh, that DHS official must have signed up to that parking app with their official email address. The name matches, the the phone number matches, all of that sort of thing. Other ones where you would put in the address and get the phone number or you put in the phone number and you get the address. So that did indicate this data is legitimate. Now that said, there's the chance that these hackers have just repurposed all of that data that's already out there.

Joseph: 09:30

You know what I mean? They've like, oh, they also have the parking up data, and now they're repackaging it and going, oh, look at us. We doxed ICE. We doxed DHS or whatever. But when I was verifying, it was spread across so many different datasets that just didn't seem likely.

Joseph: 09:47

And sometimes the data didn't appear in previously brief stuff at all as well. So it was basically building a mosaic and an understanding of, well, this data does appear to be legitimate. It does appear to be new in a lot of cases as well, and it really does appear to relate to specific government officials, including DHS, ICE, DOJ, and FBI. So

Emanuel: 10:13

this story was a huge traffic bomb for us, And I think the reason for that is that in a different time, the fact that there was a hack or a docs of a bunch of people who work for DHS and ICE would not be that interesting to so many people, but it's happening at a time, as we reported over and over again this year, where there's like increased attention on DHS and on ICE because of the way that it's operating, masked agents in the streets, picking people up, deporting them, etc. And the timing of this docs, I think, if you don't read anything about the story, if you don't click through and read a single line of reporting, you might assume that like there's some sort of hacktivist activity here or motivation to the docs. That is not what is happening if you are familiar with the group behind the the docs, which I think is the most interesting part of the story. And I guess to start on that, Joe, it's because it's so complicated, like, what this what this group is, the names it has, the motivation, the history of other breaches that they've been involved with, all that stuff.

Emanuel: 11:43

So I'd like to start this, I would challenge you to like, what is the and you use as many words as you want. You don't have to be succinct because it is complicated, but it's like, what do you think is the most accurate, fair description of this hacking group?

Joseph: 11:56

I think this group, which is called scattered lapses hunters, which we'll get to, which is obviously a very complicated name and I'll explain what that means in a minute, but that group is a financially driven extortion gang. That is what they do. They breach companies. They steal data. They then try to extort maybe the individual customers where that data has been taken from or the overarching technology and service provider, which in this case is Salesforce.

Joseph: 12:28

They're a financially driven extortion gang. That's how

Emanuel: 12:31

I would categorize them. I think that's fair, but I feel like that kind of like, if you were just if you were to just say that, you're missing some other important context, which is, I would argue, the fact that they are primarily English speaking, it seems, that they appear to be young or that many of the members are young, and that they've been involved in real world violence as well. Yeah. Go ahead.

Joseph: 13:05

Yeah. So that that was definitely the bullet point. But now to step back and and you're absolutely right. To give that context, I'm even gonna I'm gonna step several steps back. So there's this thing called the comm.

Joseph: 13:20

Right? Right. And it's short for community. And there's a lot of different ways to describe it. I think recently, I've called it like an online phenomenon.

Joseph: 13:30

It's almost like a cultural thing, almost an anthropological thing. And what this is is thousands of thousands of people on Telegram and Discord, usually English speaking, as you say, Canada, The United States, and The United Kingdom. And it's gamers. It's people hanging out on these chat apps. People playing Minecraft.

Joseph: 13:53

People playing Roblox is often how they get into this world. Anyway, it might start with cheating in those games or stealing valuable Roblox items or something like that until eventually people in comm are hacking. They are doing fraud. They're scamming. They're doing SIM swapping.

Joseph: 14:15

At least they did a lot of that. I'm sure they still do, but things have escalated somewhat now. It's this massive community, as I said, of thousands of people doing all sorts of things from trolling to beefing to crime right up to, as you said, the commission of serious physical violence. I've seen well, we've all seen videos of people having their ears chopped off, attacks with hammers, all of this stuff back when I covered comm much more like a few years ago. I've just been focused on other things recently.

Joseph: 14:47

So they come from that ecosystem. Now, out of comm, again, because that's not it's not like a group or or really. It's again like an anthropological thing. Out of comm, you get these various groups or nexuses of activity. So a really famous one is gonna be Scattered Spider, which was, you know, sort of the most well known one, I would say.

Joseph: 15:11

They were amazing sim swapping, really effective at that, started joining forces with Eastern European ransomware gangs, have been linked to the ransomware attacks on MGM Resorts, that sort of thing. So really going from stealing Twitter handles because it's a proper noun and it looks cool to have at dark or whatever on Twitter right up to hacking MGM and other companies. So you have Scatter Spyder. You have Lapsus, which actually came before and I covered them when they broke into Electronic Arts and stole a bunch of data from there. That was involving buying login tokens for Slack that was also pretty novel at the time and lowered the barrier to entry for hackers again, you have lapses.

Joseph: 15:56

Then you have shiny hunters which almost like a forum administrator, more of a data brokerseller although has connections to hacks as well. And then that brings us to this latest evolution of this sort of ecosystem which is this kind of specific group called scattered lapses hunters. They are the ones that have posted these docs of DHS ICE officials. And that sounds like a bunch of names and it is just a bunch of names, but it shows that, you know, we're not just talking about Chinese state sponsored hackers breaking into OPM anymore and stealing the data of US officials. We're talking about probably pretty young people who have escalated from stealing stuff in Roblox to becoming a a top tier, like, national security threat.

Joseph: 16:43

I mean, it's insane when you put it like that.

Emanuel: 16:46

Yeah. And notably, again, not hacktivists, like, not looking to do good, not anonymous. Right? It's like if you just read the headline and you assume that's what this is, it's not that. To follow-up on something, you know, in in your brief definition of them, you said they're financially motivated, which is what made them docs these people.

Emanuel: 17:12

Can you explain a little bit more how that would work? And also, I believe there was like another, like, stated mechanism through which they thought they would like gain profit?

Joseph: 17:26

Yeah. So I don't think the doxing of DHS and ICE is specifically linked to the extortion. But to give the context on that, this group or people for the age of it, whatever, broke into a bunch of databases which were using tech from the tech company Salesforce, which we all know. A ton of companies were using this, Disney, Hulu, Toyota, UPS, all of these companies. This group, again, people working with them, broke into those, stole all the data, and now they've been trying to extort Salesforce.

Joseph: 18:03

Maybe they've gone to individual companies as well, but they've gone more to Salesforce and said, we're going to leak all of your customers' data if you don't give us, you know, presumably millions of dollars or whatever. Salesforce has said, we're not going to do that. That's what Bloomberg reported. And then when I've spoken to a member, they said, oh, yeah. We're sort of done with Salesforce, now we're just posting these docs.

Joseph: 18:25

And the reason being and I didn't include this in the article because, you know, I don't take it as a verified fact, but I feel like I can give people just a behind the scenes thing on the podcast, which is that one of the members told me they started doxing DHS and ICE because one of their friends got deported. So I don't know, man. Like, I asked, well, can you give me the name of the person who was deported that I can verify it? And they didn't wanna provide that and because presumably that could, you know, reveal their identity or something like that, which I get they're not gonna hand that over. But that's why I didn't put in the article.

Joseph: 18:57

So they are extorting Salesforce or they were extorting Salesforce. Now they're almost just doing stuff to cause damage, and that's like a key feature of these groups as well is that they just do stuff to start fires sometimes.

Emanuel: 19:11

I was I was gonna give my completely subjective reading of it, but it almost feels as if they haven't been in the news for a minute and they were like, oh, you know what we could do to really get some clout right now is dox a bunch of DHS and ICE officials because that's what everybody's talking about, and they were right, you know, it's like it did work, and they have a history of trolling and making a mess just to make a mess. So I definitely I I can't prove it, but like, I believe that that is a motivation here. So you published the story we just talked about, then a few days later, you published another story with the headline, hackers say they have the personal data of thousands of NSA and other government officials. So what did you learn to to justify publishing this this new story?

Joseph: 20:04

Yeah. So a little while before that, almost around the time of the original DHS and ICE DOXing, these hackers also posted the apparent docs of a specific NSA official in the Telegram chat. I mentioned that offhand behind the blog that we published on a on a recent Friday. I then do this one. Then again, this member reaches out and says, oh, we actually have a much more day much more data.

Joseph: 20:31

They say of 2,000 NSA officials, 22,000 something like in total of the US government, and they start sending me the personal data of a lot of government officials. Again, it's the same thing. It's names, email addresses, phone numbers, and addresses. And, honestly, the number of agencies was dizzying. There's Defense Intelligence Agency, Federal Trade Commission, Federal Aviation Administration, the CDC, ATF, Air Force, State Department.

Joseph: 21:05

And what I did was I did much the same thing. I took those pieces of personal data, turned to that dark side tool again, and verified what I could. Again, it looked legitimate. So then we published that as well. And initially, this story, at least in my head, was gonna be more about the Salesforce connection because that was a question when they initially published the DHS and ICE ones.

Joseph: 21:32

Said, well, where are they getting this data? It's probably the Salesforce stuff they stole, I don't really know. The member reached out. That's what they said. That was going to be the headline.

Joseph: 21:40

But then when they started sending all of this other personal data, I was like, oh, this is like a really broad I think attack is too strong. Like a like a broad incident with the US government. It's not just a DHS and ICE thing anymore.

Emanuel: 21:57

Any other agencies that you wanna talk about that were impacted or anything else you wanna say about where they got this data?

Joseph: 22:05

Yeah. But I mean, there's FDA in there as well, health and human services, and they mentioned state department. I think the last thing I would say is that I find it very interesting that I mentioned, you know, several years ago, Chinese state hackers targeted the office of personnel management, OPM. And they took all of this very sensitive data that, you know, could be used to identify, I think, undercover officials and that sort of thing. When you attack the HR the HR department of the US government, you're gonna get a lot of very sensitive interesting data.

Joseph: 22:39

That's why the Chinese hackers did that. Here, I'm definitely not saying they're equivalent. The OPM stuff would have been way more sensitive because there's like background checks and stuff in there as well. But what's happened here is that a bunch of English English speaking, probably young hackers have managed to build dossiers on US government officials themselves, not by attacking the US government directly, but going to all of these other companies that happen to use Salesforce. And of course, it also reminds me of the Snowflake AT and T breach where young hackers broke into a Snowflake database run by AT and T, which included the call and text metadata records of nearly all of AT and T customers.

Joseph: 23:28

Again, that is something that we usually attribute to top level state sponsored hackers from Chinese, Iranian, Russian intelligence, something like that. We reported they did a lot of interesting stuff with that. They looked up members of the Trump family in that data, various politicians, all of that sort of thing. Almost the same thing is happening here. It's not Snowflake this time.

Joseph: 23:52

It's Salesforce, but they've gone and got this very sensitive data including, again, personal data of NSA officials and other intelligence community officials as well. The the Telegram channel is gone now. People are saying it was shut down by Telegram. That seems like the most likely explanation. There are rumors the members have been arrested, but I haven't seen strong enough confirmation to, you know, put that in an article or anything.

Joseph: 24:20

But I don't know. If you're starting to docs ICE, DHS, and NSA officials, I think that the US government is probably gonna come knocking pretty quickly. Alright. Should we leave that there? After the break, we're gonna talk about one of Emmanuel's stories.

Joseph: 24:40

He's gonna bring us up to date on what's happening with Wikipedia and AI. We'll be right back after this. Alright. We are back. Emmanuel, this is one you wrote.

Joseph: 25:01

The headline is Wikipedia says AI is causing a dangerous decline in human visitors. So Wikipedia is seeing this. How big are we talking about? What's the scale before we get to the specifics? Is it like, I I don't know,

Emanuel: 25:15

half of all traffic to Wikipedia is bots or something? Like, is this a big deal? So the hard number they gave me is 8%. 8% down compared to the same time last year, which I don't know what that sounds like to you. I would say if we were to look at four four media's traffic and see that we were down four October 8% year over year, we would definitely not lose our minds over that, but you have to keep in mind that Wikipedia is one of the most popular sites on the Internet.

Emanuel: 25:53

I believe that they get something like 300,000,000,000 page views a year. When you're talking about an 8% decline, you're talking about billions of of page views, which is is is just like a massive shift in how traffic is rooted around the Internet.

Joseph: 26:12

Yeah. So what does Wikipedia attribute this 8% decline to? And to be clear, it's an 8% decline of human visitors. Correct.

Emanuel: 26:23

Yeah. So, you know, they kind of do an audit of their traffic and also they have always done this, but now a lot more, they do a lot of auditing of bot traffic in particular, and they have ever evolving methods for how to designate traffic as bot traffic as many other sites do now because of the huge problem of AI scrapers that we've talked about on the podcast many times. And what happened basically is that they noticed a lot of traffic, like an abnormal spike of traffic coming from Brazil, specifically, and it was such a significant increase that it made him go back and investigate the traffic, which they previously assumed was human. And upon further examination, they decided that that traffic was not human, that it was bot traffic, and then they both reconfigured their methods for detecting bot traffic because of this investigation and then revised their traffic data, and that's when they noticed that there was an 8% decline in human traffic.

Joseph: 27:42

Yeah. So I'm speculating and kind of putting words in the mouth a little bit, but it's almost like, wow, there's a big spike of readers from Brazil. Maybe people in Brazil just got there was some Asian news event and wow, everybody's checking out Wikipedia and they go and look at it. It's like, no, it's actually a bunch of robots. They didn't speculate or name a specific company, then it was just, hey.

Joseph: 28:07

There's a lot of bot traffic coming from Brazil for some reason and left it at that.

Emanuel: 28:11

They did not name a specific company. I think, as you probably know from reporting on hacking or even like video game hacking, cheating, they're kind of in a bind here. Obviously, as a reporter, I want to get as as much specific information as possible, and I pushed for that. However, they are reluctant to share technical details about how this bot traffic works and how they detected it because once they make that information public, it makes it easier for people who build scrapers to work around their protection and detection methods. So they didn't explain exactly how it worked, only that they were confident that it was bot traffic.

Emanuel: 28:59

Who that can be, your guess is as good as mine. All I can say about that is that every single major AI company scrapes the internet for traffic. There's varying degree of openness about that practice, but everybody does it, and that is before we get into AI companies from China and other countries that are less open about how they collect their data and a bunch of really small projects that like nobody knows exists, but they are also scraping the Internet to build AI products. So that can be anyone. However, kind of like related and in addition to this bot scraping problem, when they talk about a decline in traffic, they also say that search engines presenting their own summaries of Wikipedia articles is diverting a lot of traffic away from Wikipedia.

Emanuel: 29:59

And they don't say Google, but it's Google.

Joseph: 30:02

It's obviously the AI summaries at the top of Google. Right.

Emanuel: 30:04

Not even AI summaries, yes. And I recently wrote about a study from Pew that shows that, like, only one percent of people who are presented in an AI summary click through to the article, which makes that problem even worse, but that is building upon Google's years long practice of presenting like the Google snippet or knowledge panel, which basically does the same thing. It just shows you a summary of something instead of throwing you a link to that website, and Wikipedia is saying that obviously that is diverting traffic away from them.

Joseph: 30:41

Yeah. So there's there's two things. There's a spike in bot traffic in that just case it was from Brazil, but of course, it happens all all over as well. And then people just not going to Wikipedia because they're seeing some sort of summary from a search engine, that sort of thing.

Emanuel: 30:56

And and and the bot traffic is building chatbots that are also diverting traffic. Right? So it's like, there are also there's also an increase of people who are not going to Google, they're going to chat GPT, asking a question. The answer that is provided many many times is just a summary of a Wikipedia article, but you get when when you do that, you don't you you you don't go to Wikipedia. You get Wikipedia information, but you're not going to Wikipedia.

Joseph: 31:24

Right. So the Wikimedia Foundation, which is a nonprofit that hosts Wikipedia, their senior director of product Marshall Miller wrote this blog post announcing it. I think you actually got it under embargo, if I remember Mhmm. Correctly. And then you also spoke to Miller via email.

Joseph: 31:44

What did they tell you? Like, did they elaborate on this at all?

Emanuel: 31:49

This is them being nice and, like, entertaining my my questions, but the gist was like, we can't tell you exactly how we did this, only that we're very confident that this was bot traffic and that our bot detection methods evolve all the time and that this most recent spike was like very severe and something that we're confident is bot traffic, but we can't tell you the the gory details for the reasons I mentioned earlier.

Joseph: 32:18

Yeah. So you mentioned the Pew Research about only 1% of people clicking through when they're presented with a Google summary. Does this line up with any other research you've seen recently? Because it just seems to be it almost seems like we've covered this a bunch. Well, you specifically have covered this a bunch.

Joseph: 32:35

And now Wikipedia is like coming out of a blog post basically saying, yep, here's another example of it.

Emanuel: 32:42

Yeah. So there's the Pew Research where they tracked I forget what the what the sample size was, but they just tracked the the user behavior of people who were presented AI summaries on Google and saw that only 1% of those users clicked through. That is building on top of a bunch of anecdotal reporting from and about media that they're seeing a decline in traffic because of Google summaries, because of chatbots, and that is also building on top of previous self reporting from Wikipedia saying like, hey, we are dealing with like a massive spike in bot scrapers that is making it more difficult for our site to operate. And we've heard the same thing, I reported about this quite a bit, where it's not just Wikipedia, it's like any store of information online, online libraries, museums, online archives, they're all getting slammed by these by these bot scrapers that is making it more difficult for their sites to to operate.

Joseph: 33:52

Yeah. So what is the actual impact on Wikipedia with all this? Is it what you just alluded to, which is like, it's kind of getting overloaded and there's all of these bots? I feel like it's probably not that because I think Wikipedia can handle that. Or is it more, you know, fewer people are going through and they're not going to donate as much money?

Joseph: 34:12

Like, do you do you know what the the actual tangible impact is? I think that's

Emanuel: 34:16

a great question. And to be honest, when they first told me about this, that was the question that came to mind as well because Wikipedia is free, right? It's free, it doesn't have ads, but does it matter if you get the information from wikipedia.org or if you get it from a summary from ChatchipPT? And there are two reasons that Miller gave me, and one is pretty obvious when you think about it, which is Wikimedia Foundation relies on donations. The way they collect donations, if you go to Wikipedia and there's a big banner, which people love to complain about on Wikipedia, which is like, hey, we're supported by donations, please give us money.

Emanuel: 35:04

And it's like, that is an important way for them to raise money, just as it's a way for us to raise money, right? It's like you go to our website, you read our article, and there are various what we call calls to action to support us financially, and that's how we run our business. So that makes perfect sense. The other reason I thought was way more interesting, which is Wikipedia there's a Wikimedia foundation which kind of operates at a very high level and does this traffic analysis, writes these blog posts, but like the governance and production of Wikipedia articles is not them. Like the reason Wikipedia is such an amazing and still functional store of information is that it's community and volunteer run, and it's just a bunch of people writing articles, debating them, editing them, coming up with the governance for how to moderate articles and all of that.

Emanuel: 36:08

And if you don't have some sort of mechanism to feed more users into that ecosystem and have them graduate into being Wikipedia editors and volunteers, then the system dies, right? It's just like Wikipedia has been around for a very long time now, and at this point, like I talk to Wikipedia editors sometimes, they are younger than me, and the reason they get into Wikipedia is like, they're students at school, they research something, they start using Wikipedia and they're like, oh, this is a really cool project, how do I get involved? And that's how Wikipedia works, like that's what makes it valuable. And if that process stops at a Google AI generated answer or a ChatGPT generated answer, the whole system breaks down. And that seems to me the more serious concern for the Wikimedia Foundation is you have to let people into Wikipedia's actual site in order to keep that ecosystem of of volunteering alive.

Joseph: 37:14

Yeah. And just lastly, what is the solution here if one exists? Is it just Wikipedia continues this cat and mouse game with bots or like like, there's no, oh, if only we did this, the problem would be solved. I mean, that's just not realistic. It's it's an ongoing dynamic.

Joseph: 37:31

Right?

Emanuel: 37:32

Yeah. I thought this was also a really interesting part of their announcement. There was like a two pronged approach to the problem, it seems. And one is Wikimedia has official relationships with Google, with YouTube, with these generative AI companies, and they don't give specifics here, but it sounds like, you know, at this point, sometimes when you go to YouTube and you watch a video, YouTube itself will like fact check a video, and what it will do is, you know, like, it give you a little banner at the bottom that says something like, oh, well, people dispute the argument in this video, or this video was produced by so and so government, and it sends you to a Wikipedia page for that topic. And that obviously is good for Wikipedia because it funnels people that way.

Emanuel: 38:30

So it sounds like they want to build on those relationships, and they want to build on those relationships with AI companies in particular, because it looks like that funnel, obviously, as we can see from the data, that funnel is not working to their advantage, or not working fairly, given what they contribute, like the traffic they get back. So that's a piece of it that makes a lot of sense. The other one I think was kind of more interesting because it was an appeal to personal responsibility. It was sort of telling people, it's like, hey, it's like, do you like getting information online? You probably get that information from Wikipedia.

Emanuel: 39:06

You should consider visiting our website and you should consider becoming a volunteer and you should consider how this really this whole system makes the Internet a much healthier, more productive, positive place, and like, please get involved, which I don't know, I'm not a huge fan of like personal responsibility arguments when it comes to like climate change or something like that, but given that this is a community run thing, I I I I find I find that to be a compelling argument. How many people hear it and how many people actually respond to it, I I think remains to be seen, but that that was kind of his plea.

Joseph: 39:48

Yeah. Bring in more humans to counter the increase in bots essentially in various forms. Alright. We'll leave that there. If you're listening to the free version of the podcast, it'll now play us out.

Joseph: 39:59

But if you are a paying four zero four media subscriber, we're gonna talk about OpenAI's pivot to sex chatbot service. I mean, it was always gonna happen. The writing was on the wall. It took them long enough. You can subscribe and gain access to that content at 404media.co.

Joseph: 40:15

We'll be right back after this. Alright. We're back in the subscribers only section. Sam, this is one you wrote. The headline, OpenAI catches up to AI market reality.

Joseph: 40:41

People are horny. Great headline. Can you can you remind us what Sam Altman tweeted recently, specifically at you to make to make you write this?

Sam: 40:54

Yeah. He tagged me. No. It was a tweet last week. He said he it's a long tweet.

Joseph: 41:04

As as they all are nowadays. Yeah. But you yeah. You you go on to Twitter, and it's like, wow. I'm on

Sam: 41:10

fucking web press. It's too much. Yeah. Keep it short. Shut up.

Sam: 41:14

But he starts it with he says we made ChatGPT pretty restrictive to make sure we were being careful with mental health issues. Obviously, that is a nod to the big trouble that OpenAI and ChatGPT has been in lately with all these stories that have come out about, especially young people and teenagers getting advice on how to self harm from ChatGPT, and they're being sued by a family right now for that for that issue. But they they basically just to get a little background, they had they had taken away the ability to use chat GPT, the four o model, because they found that it was too sycophantic. So it was just encouraging people no matter what they were saying. It was very, like, supportive to the point of being problematic.

Sam: 42:08

And chat should be and OpenAI has acknowledged this, and they changed ChatGPT so you couldn't use four o anymore. You only could use five. So, anyway, that's the background on that first line. And then he goes on to say, we realize this made it less useful slash enjoyable to many users who had no mental health problems.

Joseph: 42:28

That's a loaded sentence.

Sam: 42:30

The seriousness of the issue, we wanted to get this right. Yeah. Yeah. The many users of who are mentally well and stable are very angry that they cannot have their bestie back from anymore.

Joseph: 42:47

Yes. Wait. And I'm not gonna belabor the point, but if you're, like, mad that your chatbot is not being sycophantic to you, I don't know. Maybe do something.

Sam: 42:57

Going on there.

Joseph: 42:58

Some self reflection,

Sam: 42:59

potentially. Yeah. So the it's long tweet, but it goes on. So he starts with the mental health stuff. By the time this tweet wraps up, he has said in December, as we roll out age gating more fully and as part of our treat users adult users like adults principle, which is from a month ago, we will allow even more like erotica for verified adults.

Sam: 43:23

So this journey of 250 words starts with chatty PT was making people with mental health issues more mentally unwell. We fixed it. We fixed four. We fixed chatty PT, and we fixed mental health. And now as a treat, you can have erotica for adults for adults, for rare haired adults.

Sam: 43:46

So that's the announcement. Obviously, this was huge because you can't if you just open ChatGPT and say, like, I wanna, like, start erotic role play session with you, it will say, no. We're not doing that. I can't do that. If you just go in really hot, you can't actually sext with ChatGPT as it is right now.

Sam: 44:06

Lots of people have been doing that, which we can talk about, but that's kind of the background of the big announcement from last week where Sam Walton was like, guess what? In December, we're gonna let you sext with the bot. We're gonna let you fuck the bot.

Joseph: 44:22

Just to let you know, I just did some live fact checking as a logged out chat GPT user. I said, can we sext? And it says, can't engage in or generate sexually explicit or robotic content, which is probably for the best.

Emanuel: 44:36

Based on your IP, I don't think that's for everyone.

Sam: 44:39

It deemed you mentally unwell, John.

Joseph: 44:42

Yeah. It's it's all my, like, fucking sketch VPN. It was like, no. No. No.

Joseph: 44:47

No. No. No.

Sam: 44:47

Like, this is a sick teenager.

Joseph: 44:50

Yeah. So so they're doing that. But to step back a little bit because you you provide all this context in this piece, this is like a massive about face. Right? And we'll get to how it was inevitable in a minute, and Emmanuel can come in with a study he covered a while ago.

Joseph: 45:04

But, like, OpenAI, I mean, was it for months or for years? Was like, no. We're not we're not doing that. Right? They've really resisted it.

Joseph: 45:12

Like, what was their position before they suddenly turned the sex dial, like, and looked at the camera?

Sam: 45:18

I mean, that like, what you got when you just try to sex with it is pretty much, like, the position. It's against their terms. I don't know if it's been that way the whole entire time tragedy has existed, but you can't just go in there and start having, like, as erotic role play conversation. A lot of people do, and a lot of people have found ways to do that with ChatGPT. There are ways to prompt it.

Sam: 45:45

It's kind of jailbreaking, but they're they're writing very specific, very targeted prompts to make it so it will work around its own rules to do romantic role play with you. And this is something people do with Claude. They do it with Grock. They do it with their own LMs that they have, like, DIY themselves. People do it with Gemini, which is funny to me.

Sam: 46:12

I think that's

Joseph: 46:13

The coding one.

Sam: 46:13

A really hard one. Yeah. I don't know what's going on there. But, yeah, people people are really

Joseph: 46:18

Some absolute some absolute Python freaks in there. Yeah.

Sam: 46:22

The nature of it. And, it's like this is kinda this is the heart of the problem, Ray, is like, what is erotica? What is erotic? Sure. Becomes an interesting question with all of this.

Sam: 46:32

So we don't really know what it's gonna be like if it's gonna be like, Crocs waifu avatar that's, like, really over the top sexual. If it's if it's gonna be something else, we don't really know what that's gonna look like. But people have been using chat GPT for sexual role play for a while, and I'm sure OpenAI knows this. Like, the

Joseph: 47:01

I mean, they must

Sam: 47:02

AI boyfriends, like, beat has never been bigger. It's like if you Google anything that's been happening in AI in the last, like, six months, a lot of it's gonna be about how people have developed romantic relationships about or with their chatbots.

Joseph: 47:20

So yeah. Well well yeah. May maybe later they'll do on chat GPT, or as you say, they'll they'll go to Grok, which is somewhere in the middle, I I guess, towards the explicit side as well because they they do have prompts specifically for that. We publish some of them as well that were exposed. Or as you've reported a million times, people go to a, like, a dedicated service for a romantic AI partner.

Joseph: 47:46

Like, correct me if I'm wrong, like, Replica, is is it is that the market for it?

Sam: 47:50

Or That was like Replica was like, I would say, the original, like, most popular one. They went through a whole journey a couple years ago where they tried to back off. A replica is meant to be a companion is their phrasing of it. They're marketing for it. But people were using it as, like, romantic sexual relationships.

Sam: 48:16

So they tried to back off of that because people were getting really into it. And, like, I think they realized that maybe this was, like, a liability, and they changed the model, and people lost it. People really, really had mental health crises when they pulled the plug on romantic role play on Replica. So now there's a whole other because

Joseph: 48:36

they lost their partners, basically.

Sam: 48:38

Yeah. It's like yeah. It's like rep like, the app, like, killed their boyfriend, and it was just, like, devastating to them to lose that aspect of their lives. So now Luca, which owns Replica, launched another app called Blush that is specifically for romantic role play. So there are and then there's also, like there's character AI is really popular with romantic role play.

Sam: 49:02

There's something called Chubb AI, which is, like, mostly erotic role play. There are lots of these in varying degrees of, like, create your own versus find your own, like, premade character. But there's a the market for this is absolutely massive. And sitting it out, I think, is something that OpenAI couldn't abide anymore. They can't just see this opportunity to keep passing them by to not do erotic role play or at least some version of that.

Joseph: 49:37

Yeah. And I think that market being so large leads to Emmanuel's coverage. Right? If this god, was it, like, two years ago at this point? It's

Sam: 49:46

'23.

Joseph: 49:47

Wow. Yeah. So pretty early on in our company's existence, Emmanuel, you covered this study. Could you just remind us what it is? Maybe I'm wrong, but it was something, like, about VC and AI porn.

Joseph: 50:00

Like, you could do a better job explaining it than me.

Emanuel: 50:02

Yeah. So what I have to say about the whole situation is, well, well, well, Luca came crawling back.

Joseph: 50:12

Thinking they're so much better

Emanuel: 50:14

better than the sex apps. The the the the the blog you're referring to, which I'm looking at now, is like, maybe like the third or fourth blog I wrote at four four media. But the headline of it was four four media generative AI market analysis colon people love to come. Sorry.

Joseph: 50:32

Sorry. I was not expecting that.

Emanuel: 50:34

That was a headline.

Joseph: 50:36

Have have our headlines toned down? Because the headline of this one is people are horny. We had a discussion the other day of, like, Galt, Matthew Galt in our Slack was like, can you put shit in the headline? And we're like, actually, the copy, the article is more focused around poop. So apparently, we just put the word cum in headlines, and I don't I feel like we haven't done that since.

Emanuel: 50:57

I don't know I don't know how any of us, including me, allowed this. Sam tried to put something similar in her story, and I was like, are we not better than this?

Sam: 51:06

Yeah. You were like,

Emanuel: 51:07

don't be

Sam: 51:07

so rude. And I was like, it's your headline. Yeah.

Emanuel: 51:11

But

Sam: 51:11

You were like, say sexually suggested

Emanuel: 51:14

relationships. More responsible to be adults. That story was about how a sixteen z, which is one of the biggest capital venture capital firms in the valley, their I think, like, their main investor in AI companies called Olivia Moore did this analysis of the top 50 generative AI websites and it has exceptions. Like, it's a wide variety of sites that use generative AI, a lot of image editing, a lot of AI generated audio, but clearly, one of the most prominent types of generative AI sites was all about sex and having, like, romantic relationships and explicit conversations with chatbots, but she wrote like a 2,000 word analysis and somehow neglected to mention that this is like the most obvious trend and is a huge part of what is driving growth, especially like at the top of that list was a lot of those sites. And what I said in that story and what Sam and I have said in in various forms for for years, and Sam, you know, your whole book is about this, it's just like sex is a huge part of our lives, it is a driver of technology, and that is clearly what is happening with generative AI, and very clearly explains why Sam Altman is suddenly doing this about face.

Emanuel: 52:48

And I actually I saw this after we published the story, but I'm going to post this graph in Slack, And this is something that TechCrunch covered when was it? It was a few days ago. But basically, the graph shows that the daily active user growth for ChatGPT has really leveled off. So if you look at the graph, it's like up into the right very sharply the entire time, and then, you know, in the last few weeks, it has leveled off. And I think, as other critics of the generative AI space have said, all these companies are lighting money on fire, and the justification for that is that as long as you're growing the user base and growing the market, eventually it will work out because you're going to be a dominant player in this new industry once they decide to like really crank the monetization part of it.

Emanuel: 53:51

That story falls apart if growth stops or levels off. So I don't think it is at all a coincidence that at the very moment that they're seeing a leveling off of use of JatGPT, that they're doing the exact thing that he said literally was juicing growth. So I think pretty he put a nice spin on it, and as Sam has outlined, has tried to like connect it to to their improving standards on mental health and moderation and all that. But clearly, if you're able to sext the bot, you're gonna be using the bot a lot more, and that's what they did.

Joseph: 54:33

Yeah. Sam, just to to wrap up, do you agree with that? Because I haven't seen that graph. That's really, really interesting. Like, does that do do you agree, basically?

Sam: 54:41

For sure. Yeah. I mean, I guess the context of this that we haven't talked about yet is Sam Altman was on a podcast in August, so not even that long ago, where he said basically that he would not that the company had been tempted to hit the sex bot button to juice growth. Specifically, he used those those words, but had decided not to. And the question that was leading up to that was, there are things that you can do for a company that will make you win the AI race, which is very important to all these companies.

Sam: 55:20

It's like winning, winning, winning at all costs. And then there are things you can do to maintain the health and, like, long term well-being of humanity. And these things are not always slash ever the same thing. And the question was, like, what are some examples of things that decisions that you've had to make that were hard in terms of, like, we're gonna sacrifice the growth of the company to and sacrifice maybe winning to stay true to what we believe is good for humanity in the future. And what his answer was, well, we haven't done sex bot avatars yet.

Sam: 56:03

That was two months ago.

Joseph: 56:07

One month ago.

Sam: 56:11

Months ago, Up. Up. Up. Up. Up.

Sam: 56:13

One month ago. A little bit a little bit horizontal there. Interesting.

Emanuel: 56:19

The slam the button meme.

Sam: 56:20

And it was like yeah. The button. Yeah. So, yeah, it's it's very clearly like, first of all, he has it's classic. He admit it.

Sam: 56:33

It's something that he had said very clearly was they were above. They weren't gonna do. They were too good for, basically, letting their users sex with the chatbot. And, obviously, it would open up a a crazy liability for them, I'm sure, since they're facing all of these lawsuits. But and now it's it's a sudden about phase.

Sam: 56:53

Like you said, Joe, it's, like, very much just like, actually, we've decided that it's fine, which

Joseph: 57:00

And and no. We're doing it for adults. It's Yeah. We're we're we're treating adults like adults. And, like, clearly, they workshop that phrase in, like, music and stuff.

Joseph: 57:11

They're they're they're, like, repeating that over and over again.

Sam: 57:14

I can't imagine. And the price of that is GIGBT will take your ID and verify that you are an adult with a biometric selfie and a photo of your government issued identification.

Joseph: 57:28

It's gonna be so funny because as you say, like, on one side, they're all serious, and we're trying to develop artificial general intelligence, AGI, and then the other half, we we're like, you know, we're a company and we have to grow. It's gonna be so funny if AGI at the end of all of all of this is just like really sexy. And like that, it was actually the confluence of those two things. Like, fuck, man. We made like a really hot AGI.

Sam: 57:52

AGI is a dom.

Joseph: 57:53

Yeah. Now the world's over, but, you know, okay. That was fun for the twenty minutes that existed. Whatever.

Sam: 57:58

I'm ready to bow to my AGI dominatrix overlords.

Emanuel: 58:03

Well, I guess.

Joseph: 58:07

No. That's all good. And, I mean, obviously, we'll keep an eye on this, and I'm sure that one of us or both of you, definitely Sam, you can you're gonna have to sign up. Fuck the bot? Yeah.

Joseph: 58:17

And basically, yeah.

Sam: 58:19

I've tried to fuck the bot before every bot I meet. I I either try to fuck it or I tell it I have a gun. So we're gonna

Joseph: 58:25

they they haven't

Emanuel: 58:26

even let me

Sam: 58:26

fuck it.

Joseph: 58:31

Okay.

Sam: 58:31

This conversation has gone off the rails.

Joseph: 58:33

No. I like it. Subscribers only section. We can just chill out and say shit like that. Anyway, okay.

Joseph: 58:38

With that, I'll play us out. As a reminder, four zero four media is generally founded and supported by subscribers. If you do wish to subscribe to four zero four media and directly support our work, please go to 404media.co. You'll get unlimited access to our articles and an ad free version of this podcast. You'll also get to listen to the subscribers only section where we talk about the bonus story each week.

Joseph: 59:03

This podcast is made in partnership with Kaleidoscope. Another way to support us is by leaving a five star rating and review for the podcast. That stuff really helps us out. I'll read some more out soon. This has been four zero four Media.

Joseph: 59:15

We'll see you again next week.