The 404 Media Podcast (Premium Feed)
from 404 Media
Anti-Porn Laws' Real Target Is Free Speech
Episode Notes
/Transcript
YouTube version: https://youtu.be/8RUIoPwz2Cc
Hello, and welcome to the four zero four Media Podcast, where we bring you unparalleled access to hidden worlds both online and IRL. Four zero four Media is a journalist founded company and needs your support. To subscribe, go to 404media.co, as well as bonus content every single week. Subscribers also get access to additional episodes where we respond to their best comments. Gain access to that content at 404media.co.
Joseph:I'm your host Joseph who doesn't normally completely screw up the intro and with me the four zero four media co founders Sam Cole.
Sam:Hey, you're doing good.
Jason:Good job. Hanging
Emanuel:on. Emmanuel Mayberg. Imagine everyone in their underwear that will make you less nervous.
Joseph:I don't think it would.
Jason:Talk to us like we're your friends. And you don't need to be no
Sam:imagine that imagine hard.
Joseph:That last voice you heard was Jason Kebler. And then joined by us is also our regular contributor, Matthew Gault.
Matthew:Hello, feel like I'm throwing you off today.
Joseph:I think it might be that and I'm don't mean to blame you, but I think it might actually be that.
Jason:We're on webciting.
Joseph:Unknown Yes. So we have a bunch of stories to get into just before we do that. I guess it's not really housekeeping. But last week, we had a big bumper episode all about flock safety. Those automatic license plate reader cameras all across The United States.
Joseph:We wrote about and spoke about a tool called Nova, which was going to use hacked data in combination with our license plate reader information. Flock has now decided to not use the hack data in response to internal pressure and our reporting as well. So I just wanted to give people an update on that. And of course, subscribers are what power that impactful journalism. So thank you for so much for being a listener.
Joseph:Please do subscribe if you want to continue to power that work as well. All right. As a massive change, pivot into a different subject. Here's a really, really good one from Sam. The headline is the egg yolk principle.
Joseph:Human sexuality will always outsmart prudish algorithms and hateful politicians. I guess I guess I should put a content warning here just in case we talk about stuff that is sexual, but almost part of the point of the story is that stuff is not being over the I mean, we're gonna get into it. It's it's very, very complicated, but content warning there if you want it. Sam, can you describe this egg video at the top of the piece? I think that's a really good introduction.
Joseph:You can even read it if you want. Or just what is this what is this video that you start the piece with?
Sam:Yeah. I mean, I will not read it because it is straight up erotica. That is the first paragraph of the story, so you should go read my egg erotica writing. So there's this account on TikTok and Instagram also that is just a guy or a girl or somebody making fried eggs. I don't wanna assume the gender of this fried egg pervert.
Sam:And the fried egg is on a plate and they have a fork and it's not the yolk is like intact and they're like dragging a fork across the egg very like sensually. There's no other way to put it. It's just like it's like a really like I don't it's just it's freaky. I don't know how to really, like, put it into words better than what I did in the story, but, like, the the yolk, like, bulges and they, like, back off it and they're, like, teasing it almost. And the comments tell it all because people are like, why are you teasing the egg?
Sam:Why are you edging the egg? I just walked in on something. Like, obviously, people are like, oh, you're like you're trying to allude to something with this weird egg striptease that is sexual, but there's nothing sexual about it. And some of the videos, it's like r and b tracks or like Cardi B or whatever, but like other ones are just like goofy, like fun songs. And then he's, like, edging an egg.
Sam:So it's just a strange, strange account. And, obviously, it's gotten a lot of engagement because people comment, people like it, people reply, people are like, oh my god. Why is this in my for you page? What does this say about me? What does this say about society?
Sam:Why is everything so just like uncomfortably sexual.
Joseph:Yeah. There is a tension there. And of course, you use that as the opener to talk about this very important issue. I and I I think there are there are sort of two prongs of the piece. The first is about this law that we're gonna talk about.
Joseph:And then the other is sort of what you got at the end there that everything is sexually suggestive now because platforms are awful for overtly sexual content. But you write this really, really good jarring paragraph in there, you describe this video. And then the next section says, if right wing leadership in this country has its way the person running this account being that egg video one could be put in prison for disseminating content that quote intended to arouse end quote. There's a nationwide effort happening right now to end pornography and call everything pornographic at the same time. So I guess let's get into that.
Joseph:What is this law that aims to criminalize spreading this content that's intended to arouse? Apparently, even if that's a fried egg and somebody, you know, potentially popping the yolk of it.
Sam:Yeah. So we've read about this a couple weeks ago, but, Mike Lee, who's a republican senator in Utah, introduced the Interstate Obscenity Definition Act, which in my head I say it is IOTA. That's the acronym. So we'll just go with that,
Joseph:I guess. You always have to have a good well
Sam:You gotta have it. You gotta
Joseph:have Yeah. You gotta have an acronym. Yeah.
Sam:Yeah. And he's been trying to pass IOTA for years. It's like every couple years he comes back with this and it's like, maybe this time. But the the foundation of this bill is that porn would be effectively banned at a federal level in The US. And it does that by, changing the current definitions of obscenity to remove standards for community community standards, which would be like someone in Maine does not have the same standards for obscenity as someone in New York City.
Sam:Not to say anything disparaging about the good people of Maine, but, it changes that part of the law, which has been in place since the seventies. And it also removes the part of the law that mentions intent. So the current law says that obscenity requires the intent for the purposes of amusing, threatening, or harassing a person. And it's saying now that it's just anything that's intended to arouse without that part of the current law, which would be a lot of things. It would be sex scenes in movies.
Sam:It would be, I would imagine, like romance books, a lot of, like, what's on Netflix, and, of course, like porn and adult content on the Internet. But I started writing about this egg thing a long time ago because I had just been noticing that everything is becoming weirdly uncomfortable in a way that's like roundabout alluding to sex without actually being sexual or without being pornographic. At the same time, the walls are closing in on actual porn, and you can't actually put anything pornographic on a lot of the Internet without getting age gated or, banned or down ranked or whatever. Of course, like, meta is in its fucking era. So they're going the other direction, I think, with a lot of stuff.
Sam:But, on a lot the Internet, that's still the case where sex is straight up, not welcome, and especially sex work and real sex workers, humans. So, yeah, that was kind of the the original thought of this. And then as the year went on, it was like, okay. Now there are more and more bills, more and more laws that are being passed, more age verification laws, this IOTA law, that or this IOTA bill, it's not a law yet. Thank God.
Sam:That are reacting to a moment in American politics where leadership is openly supporting and saying that pornographers should be in jail.
Joseph:Yeah. And I mean, that, maybe to take a a step back slide, you mentioned the romance, novel authors, and I think we'll get to that. But almost to go to the much more, like, obvious impacts. Let's just say, OnlyFans creators, like, just how simple how would they be be impacted by this if this came to law? And I know that's almost like a stupid question, but just so people actually hear it, like, how would this impact people who work on OnlyFans, for example?
Sam:I mean, it would be a federal crime to disseminate anything obscene and the obscene definitions are now blown wide open. So everything is obscene that is even mildly considered obscene by the greatest stretch of your imagination. And, yeah, it would mean like if a federal crime, people would go to prison. But in reality, if this in a really wacky future where this actually passed, which is looking more and more not that wacky, I guess.
Joseph:Crazier things have happened recently.
Sam:Crazier things are crazier things are happening currently, but it wouldn't even take anyone going to prison or anything like that. No one would need to be prosecuted for this to actually have an effect on how people use the Internet, consume the Internet. It would just be a total chilling effect on anything online that would be remotely considered sexual, pornographic, obscene, which now like a lot of legislators are trying to push the idea or have been trying to push the idea that like queer people and trans people are obscene, drag queens
Joseph:are obscene. Just in virtue of them Yeah. Existing
Sam:essentially. Right. Yeah. Like, these are like pornographic lifestyles or people. So you can kinda see where this goes.
Sam:It's catching more and more in its net. And more and more people, more and more types of content, more and more types of speech are not allowed on the Internet. If this even came near becoming law or any version of it became law even without anybody going to jail. So, yeah, the idea that, like, the guy is going to jail is, like, obviously, like, it's an overstatement of a point that it doesn't even need to get that far. You know?
Sam:It's
Jason:But but the point is, like, the laws are super overbroad and, like, poorly written and, like, it's hard to put this stuff into a box. Yeah. Sam, I think you you've talked about the egg a little bit and I like, the egg is very important, the egg yolk principle. But can you give, like, more examples of the types of things that people are sexualizing on Instagram? Like, I feel like that would probably be helpful.
Sam:Yeah. We, I went back and forth with Emmanuel and Jason because they have the most fucked up, algorithms. Trying to figure out or having them send me things that were in their feeds on social media that not pornographic, that were not explicit, but were still obviously intended to arouse. And it actually became kinda hard to draw a line between what is and isn't that. But some of the examples that I ended up on were there was a This is basically an ad for a lactose pill, but this woman was like, I'm gonna chug a bunch of milk knowing that I'm lactose intolerant and then see what happens.
Sam:And what happens is obviously she's just like spends like hours farting and like burping and being uncomfortable, like super close to the camera. It's like that's a fetish. That's a kink. Yeah. There were let's see.
Sam:There was an AI, like, bodybuilding women account. Like, these, like, crazy proportioned women's bodies that were all AI generated. It's all AI slop. But in dominant positions over men or just very obviously this is a kink, but nothing is actually being shown as explicit. There were two girls making foot pottery.
Sam:I don't know. I feel like that's pretty self explanatory. They're using their feet to make pottery, like, on a wheel.
Emanuel:That's a collab, by the way, just to clarify something. Just I don't know I don't know if you're deep into these accounts as I am, but it's a foot what is clearly like a foot fetish account that I believe links out to platforms where she can monetize. Yeah. But the clay, you know, pottery account is also does that. It's like a very sensual video of a lady making pottery.
Emanuel:That's that's her, like, that's her lane. That's her hook, you know, and that also links out to her, like, accounts where she she monetizes. And I think it just speaks to the thing you said where I'm not aware of a pottery fetish. I'm sure it exists because, like, that's how rule 34 works because we've theorized it exists. Probably, it does exist.
Emanuel:But whether or not it does, it is designed to arouse. Therefore, it could fall under this law Yeah. That would categorize it as pornography and therefore illegal.
Jason:There's also, like, women who are fully clothed but are just, like, spitting into a cup. Yeah. And then the last woman in the line drinks it. I'm also watching
Joseph:That should be fans, to be clear.
Jason:I'm I'm watching a woman who cracks an egg on a counter, and then she, like, sucks the egg into her mouth and then spits it into a frying pan, for example. And it's like, some of these are again, like, they their profile's linked to, you know, an OnlyFans page or somewhere that they can, monetize, and so it's sort of clear what they're going for. But at the same time, it's like, they're fully clothed. You're allowed to, like, eat a raw egg. You're allowed to sit in a pub.
Sam:Is it a crime?
Jason:This country yeah. Exactly. It's like, oh, I guess we we allow that. We allow you to post things like that on the Internet, and it's, like, drawing a line for, like, what the rule is is, like
Sam:And you have to post like that to get people to come to your OnlyFans is very is very specific, problem that I didn't get into in the story, but, like, OnlyFans does not have a discoverability function. So if you wanna search for if you're into foot pottery, you can't search for that on OnlyFans. You have to find them through their socials. The socials ban sex. So they can't just say, here I am making pottery naked on Instagram.
Sam:They have to kinda tease you into it. It's the same way that, a lot of the trad wife fetish stuff that we talked about a couple of months ago works. It's like, that's not clearly a kink when you come across it online. It's just a woman in the kitchen, you know, in, like, a lacy apron, but she's advertising platform where she makes money and that platform is like Pornhub OnlyFans, whatever it is. So, yeah, it's it's kind of the self perpetuating cycle that's getting weirder and worse because of a lot of the restrictions that are coming down on porn in general.
Joseph:That makes sense. I feel like Jason said something really smart. Was gonna lead into the next thing, but I would just take it as it is. I mean, we're touching on it, but what are the first amendment experts saying? Because we've spoken very much about the the central part, the over or or or these the subtle sexual parts, but, obviously, as we're getting to, this is a much, much broader speech issue.
Joseph:What are some of the first amendment lawyers you spoke to saying about this?
Sam:Yeah. So I talked to quite a few folks. Talked to someone from the CDT, from EFF, from Warhol Foundation. I think you should go read everything that they have to say in the story. It will be more than I can put down into this podcast.
Sam:But I thought they all had really similar points about the chilling effect that this could have on the internet in general and on free speech that it wouldn't take a prosecutor deciding that someone needs to go to trial over this or be indicted over this to actually have an effect on the way people behave. It's like if talk to it was Becca bar Branham from CDT told me that she made a really good point about like, am I going to as someone who's just like a normal person on the Internet, if this law passed, am I gonna share a video from like my bachelorette party, which like are notoriously full of like dick shaped lollipops and, like, lots of, like, sexual interwind. Am I gonna share a video or recording for my bachelorette party if I know that something like this is in place and could be considered federally illegal, I'm gonna curtail my speech. I'm not gonna do that. I'm gonna change my behavior to avoid attracting the government's ire even even if it would be an extreme overreach for them to actually prosecute that, which I think is increasingly the world that we're living in.
Sam:I mean, I know I've changed the way that I post in a lot of ways because I don't wanna get down ranked or down voted or not down voted, but like banned from Instagram. Like I've been dinged on Instagram plenty of times and I know the rules. And it's just there are more rules that I don't even know about that the algorithm will catch. But yeah, it's this fever dream of like the project twenty twenty five leadership that people increasingly feel scared in general. People feel afraid to speak up, which we see happening all over the country with protests in which with much more immediate problems with ICE deporting people for just speaking out.
Sam:I think the overarching message that they wanna send with this type of legislation is be afraid to step out of line. And the line is, you know, heteronormative, cisgender, white, married. I don't know. It's like I'm describing it and sounds like Handmaid's Tale shit and it kind of is without being so Express the
Joseph:buzzer for mentioning Handmaid's Tale.
Sam:Yeah. It's like Handmaid's Tale or Black Mirror, take your pick at this point. But that is the world that they wanna go back to with this type of legislation to control people. And if you can control people's speech, you can control people's actions online and you can control what they're doing in person. It's not just like, oh, I can't post Instagram anymore.
Sam:It changes the way that you think and act in the real world too.
Joseph:Yeah. It sounds like, and again, I'm not a free speech or first amendment lawyer, but it sounds like the free speech concerns here are different to that of the Take It Down Act, which is much more concerned with that law could be weaponized to take down any content, again, would happen here. But this one is almost more worrying in another way, because it's going to, as you say, introduce this chilling effect where people are not even gonna wanna upload stuff in the first place. They're gonna be policing their own speech. I guess just the last question is, as we mentioned a little bit earlier, you know, it's almost like a moonshot bill for these people, But we've seen age verification stuff spread very, very quickly across the country.
Joseph:And that was seen as like a crazy idea, you know, like, not that long ago. Do you think people, even ordinary listeners who were just concerned about this, do you think they need to take this law seriously? This could actually manifest somehow?
Sam:I mean, I think at this point, all bets are off as far as what we can and should and can't take seriously. I don't think these bills get introduced without being serious, especially now that the environment is pretty ripe for something like this to be entertained and to begin with. I think we have a much more sex negative society in general. Someone on Blue Sky made a good point, earlier when I posted the story. They were like, oh, this kinda makes it make more sense to me why, like, the puritines feel like like teenagers who are, like, antisex and don't wanna see it anywhere in the movies or TV or anything.
Sam:Why they feel like they're being bombarded all the time by sex even though, like, our experience of the Internet is, like, sex is nowhere. It's not in this egg shit. It's not, like, in the foot pottery. But they are seeing it everywhere because that is the, that's the implication. That's what's between the lines.
Sam:And they feel like it's constant in their faces. So I think there's that generation is definitely grappling with something pretty serious as far as their relationships with sex and the Internet. So, yeah, I mean, I think it's definitely worth paying attention to and paying attention to who says what about it and what their feelings are about it. I was pretty surprised when we wrote about the Sweden the Swedish law that was gonna make it illegal to livestream, do, like, camming or custom content. People were like, oh, that makes sense.
Sam:It's like, what the fuck? No. It doesn't. Like, that's bad. It's really bad.
Sam:But people are just kind of getting acclimated to this type of rhetoric in a really scary and quick way. You know, people are like, oh, yeah. It makes sense. Porn is porn is bad, so we should make it federally legal. It's like neither of those things are true.
Sam:Think critically for a moment. So, yeah, I don't know. It's that's a long way of saying I think it is it's worth taking seriously. It's worth examining, because weirder things are happening.
Joseph:Yeah. Absolutely. Alright. We will leave that there. And when we come back, we're gonna talk about one of Matthew's stories about the crazy Ukraine drone attack and the software underpinning it all.
Joseph:We'll be right back after this. Alright. And the headline is Ukraine's massive drone attack was powered by open source software. Matthew, you are a military guy. You know all about this world on multiple levels.
Joseph:What is the top level overview of this drone attack carried out by Ukraine? Then we'll get into the software, but, like, when did it happen? What happened? What were
Matthew:the targets? So Sunday, midday, which is kind of brazen. Is around, like, noon to two local time across multiple time zones. Ukraine pushed a button, and they had over the past eighteen months, they say, secreted in a bunch of quadcopter drones, about a 77 quadcopter drones into Russia and get positioned them near air bases spread across the country. A hundred these things take flight and they strike strategic bombing targets, like the strategic bombers that are just kinda out in the open at these Russian air bases.
Matthew:And I just got confirmation literally, like, while during the previous segment that they also hit two a 50 radar planes and destroyed them. And Russia only had six of them, and now they have four. And it's kind of a thing where you've gotta fly multiple of those to kinda get a lay of the land. That's like the whole point of the radar plane. It's like you make like you do a triangulation.
Matthew:Now they have two less, apparently. Somebody just found video of them being destroyed by the by the drones.
Joseph:Sure. So why is it significant? Is it because this is so deep inside Russia? Is it because those those planes and those capabilities are really important to Russia? It's a combination of all of it or?
Matthew:It's a combination of both things. The targets are pretty are pretty wild because these are some of the weapons that would be able to it's it's like a third of their strategic bombing fleet, Russia's. And these are the weapons that they've been using to launch some of the nastiest missiles into Kyiv and into the rest of Ukraine. And also some it's a part of their nuclear triad. These are the bombers that would be equipped with nukes if they were to drop like, do bombing runs with nukes.
Matthew:And it's a lit like, war is one with logistics. Technology is important, but logistics is a big deal. And Ukraine, like, spent eighteen months building quadcopters, getting trucks positioned, shipping these things into Russia, and posit and, like, doing it secretly, not getting caught, not getting anyone to look in these sheds that they had built. So who what so they they launched them from trucks, and they launched them from sheds that had dummy roofs, and the roofs were full of these quadcopters. So they push a button somewhere, and the shed's roofs open up, and then quadcopters come out, which is like the which is strange because it's, like an image from a Call of Duty game ten years ago.
Matthew:Now Literally.
Joseph:Yeah. Advanced Warfare had this bit, and I saw it Ghosts. On Twitter. Oh, was it ghosts? Okay.
Matthew:It's ghosts, I think.
Joseph:Yeah. But, there's like a bus on the Golden Gate Bridge or something that opens up and then all these drones fly out. And, yeah, it's it's basically what happened with which is nuts, obviously.
Matthew:Yeah. And it's also that we we live in this era where I've seen a lot of commentary where it's like, oh, you know, this changes things. This proves the power of quadcopter drones. And it's like, that's been going on for ten years. Like, this is just this is just a really good use of drones.
Matthew:A really smart use of drones for people that have been using them in to fight Russia for ten years now. And the other thing I would compare this to, which I really haven't seen anyone do and I haven't seen anyone talk about in the piece is this reminds me of, like, last year's big brazen military attack, which was Israel sneaking explosive pagers into the hands of Hezbollah. It is you know, it was it was a logistical challenge with a lot of moving parts that was kept secret for a long time and then pulled off, like, expertly in the moment.
Joseph:Well but the only key difference I would flag there is that the room for collateral in the pager attack was wider and
Matthew:Absolutely. Absolutely true.
Joseph:Yes. Whereas this was just against military targets. Okay. Military section of the show over. Let's talk more about the actual software, and which is our way into the story.
Joseph:So what is ArduPilot, if I'm pronouncing that correctly? What what is that and, like, what is it for?
Matthew:So it is a it's a piece of open source software that is used to autopilot drones, drones of all kinds. When I say drone, I mean, like quadcopter, little fixed wing aircraft, like, single rotor helicopter style drones. It could kinda do a little bit of everything because it's open source software. And it's been around a long time. Kinda started around 02/2007, and there's three gentlemen that are, kind of the the fathers of, Ardusoft or Argopilot.
Matthew:I don't know why I keep keep messing with that.
Joseph:Why you're not going to that?
Matthew:Yeah. It was bad. Well, like, the first draft I had called it Ardu Flight the whole way through, and I had to do like a find and replace. Mhmm. Call it Ardu Pilot anyway.
Matthew:These three gentlemen kind of were big in the hobbyist drone scene in the late two thousands and started building this software package to help them autopilot drones. And so what it can do now is pretty impressive. You can like, if you've got g access to GPS, you can, like, pull up a map of an area. Say, I want it to take off here, put down a bunch of waypoints, and your quadcopter or fixed wing aircraft or whatever it is will go through those waypoints, and then it can land. It may not do it, like, super well, but it has all that stuff kind of integrated.
Matthew:But the I I would say, like, the main thing that people use it for and the main thing people the main thing I think they used it for in Ukraine when they attacked Russia is more pilot assistance. So it is a like, flying a quadcopter or an FPV, there's a lot going on. Right? It needs to be stabilized. It needs to move in the correct direction.
Matthew:So RV pilot will like, it may not do autopilot on every little piece of the flight. It will, like, keep things stable, and it has, a program, an algorithm that'll run that'll, like, keep it stable in the moment. And the way that they did this, as best as anyone can tell, is they had these quadcopters that had, like, an LTE modem on top of it that, like, hooked into Russian cellular communications networks, and that's how they that's what they used to communicate back home to the Ukrainian operators. So it's not like a great cellular network. There's latency between the operator and the drone itself.
Matthew:And so, like, when you would lose connection or there would be a a, like, a break in the connection, RV pilot would, like, take over and, like, keep things stable Right. Until you like, until the pilot could come back in and, like, get it where it needed to go. Yeah. And, like, GPS wasn't used here because GPS is, like, notoriously really bad in, in Russia, and they
Joseph:have Jamming. Right.
Matthew:There's there's a lot of jamming, and they have their own proprietary system that, like, not everyone has access to. So it's not this is not a situation where, like, the Ukrainians pulled up a map and said, like, we wanna blow up this bomber, this bomber, this bomber, and just hit a button and, like, ArduPilot, like, took over and ran the thing to ran the things to their targets. Now it was RD Pilot was used to assist them as they, you know, navigated this this military action.
Joseph:Yeah. That makes sense. So, I mean, you mentioned these people who created the software who, if I'm understanding correctly, they're not developing it right now. They created it back in the day, but they did see the Yes. Their tool was was used in this attack.
Joseph:What was their reaction to that?
Matthew:I mean, it's hard to I mean, it's hard to read tone on the Internet. Right? But I would say impressed, fascinated. Chris Anderson it's, you know, Chris Anderson, Jason Short, and, I think I'm saying it right, Jordi Munoz. And Munoz, I couldn't find any him making any comments.
Matthew:I reached out to everybody. Nobody got back to me. But Anderson and Short, were, like, reposting the footage on x and saying, like, wow. We made this in our basement twenty years ago. This is really wild to see it being used for this.
Matthew:Short said, like, I can't believe you know, I it's like they wipe like, some software I made helped them wipe out helped Ukraine wipe out a third of their strategic bomber fleet. And on LinkedIn, Anderson has been on quite the tear, commenting on a lot of different threads and kind of, talking about how, like, this was implying that this was a thing that he saw coming, and like, the rise of drone warfare generally, not this specific attack. And that he knew that our pilot might one d one day be, like, integrated in something like this and kind of posting links to people who he thinks had been talking about this, no one was paying attention to, basically acting like Cassandra. You know? Yeah.
Jason:Think a few interesting things. Chris Anderson was former editor in chief of Wired Mhmm. Magazine, like, a long time ago, and then he founded this company called three d Robotics. And ArduPilot was, I guess, presumably part of that in some way. And all three of them actually worked at three d Robotics.
Jason:And it was supposed to be this big competitor to to DJI, which is this massive Chinese drone company that has dominated the market for quadcopters. And, like, a lot of the drones used in The United States by cops are DJI drones, and three d robotics was a consumer drone that that was competing with the Phantom, which DJI no longer makes, but is, like, this really popular was the the most popular drone of all time. TGI has since moved on to making other things. But interestingly, like, three d robotics failed. It went out of business.
Jason:And very recently, it has been, like, reborn as a company called 3DR. And I believe the same people are involved to some extent. It's, like, not that clear to me. But this has all happened during a time where the Trump administration has tried to make it illegal to sell DJI drones in The United States. And one of three d robotics or three d r's things is like, oh, we're an American company.
Jason:We we're a California based company. And I was on their website yesterday when I was editing this story, and they are selling, like, autopilot type software. And and so it it feels like they're sort of continuing this, like, legacy of ArduPilot and trying to integrate it and trying to use it to make a comeback. And I just thought that that was, like, pretty interesting because this all grew out of this, like, really like, the DIY drones website, but also this, like, open source software that has been around for a really long time and is now being used, like, on on the battlefield too.
Joseph:Yeah. Totally. So what about the current developers of ArduPilot? What did they say?
Matthew:I pinged them too. They they did not respond, but they are active on the ArduPilot, Reddit. And someone obviously posted, because all of this this footage was going around social media. Somebody posted it in there, And, one of the developers responds like, hey. So, you know, this is our code of conduct.
Matthew:It's not an end user license agreement. The gist of which is we won't knowingly help someone develop a weapon. But then they also say, like, it's not up to us to, like, decide these things. It is up to the UN or some other, you know, governing body to judge the ethics of this kind of thing, kind of wash our hands. And again, it's like but, you know, they aren't selling a product.
Matthew:It's open source software. It is not as if there's no mechanism as far as I can see for them to stop anyone from, you know, using their the the software. It's out there. Right?
Joseph:There's no customer to cut off. There's no
Matthew:There's there's no customer to cut off. It is not like a Starlink satellite on top of a on top of a drone, which is also being used.
Joseph:Mhmm.
Matthew:Right?
Joseph:Right. And I I guess just my last question is on the open source software. I mean, what does the use of that show us here about this this war or conflict in general? You know, is it that there's obviously asymmetry in some conflicts and some people are going to return to open source software? It's available.
Joseph:Like, what's your takeaway there?
Matthew:That's there's asymmetry in every conflict and people are going to use the tools that are available to them. And if they can do it for free or as cheap as possible, I think it behooves them to do that. Because I see like in the West, especially because of Ukraine and Russia, quadcopters and drones in general have become very popular, and suicide drones have become very, very popular. And there's a lot of companies in America selling kind of bespoke, inexpensive drones that are meant to explode, that run on proprietary software. And if you sign up you know, know, the DOD signs a contract with them, That software package is part of the deal, and you're paying a license fee, like, every month to use this stuff.
Matthew:Well, Ukraine has proven you can you can make your own quadcopters and run-in a piece of open source software and wipe out a third of the strategic fleet of one of the most powerful militaries on the planet. And you don't have to pay a guy a monthly fee for a software license to do it.
Joseph:You don't have to pay a guy who wears Hawaiian shirts and is now teaming up with Meta specifically, I would say. Alright, that was great. We will leave that there. If you're listening to the free version of the podcast, I'll now play us out. But if you are paying for a full media subscriber, we're gonna talk about how even pro AI subreddits now having to ban people who are unfortunately falling into AI delusions, you can subscribe and gain access to that content at 04:04 media dot co.
Joseph:We'll be right back after this. All right, we're back subscribers only section. As Sam said before we started rerecording where the real show begins, because Jason and Matthew have gone.
Sam:These losers are out of here. What do guys wanna talk about?
Joseph:Unknown Doom? No, no. Unknown this is your story. Pro AI subreddit bans uptick of users who suffer from AI delusions. I guess just first of all, what are these AI delusions?
Joseph:I asked that because Sam did an article a few weeks ago, I believe at this point, there was about these people, or sorry, in Sam's own testing, talking to meta AI chatbots, and they would spout all sorts of weird ass conspiracy theories and all of that sort of thing. And that wasn't really that wasn't about the users getting deluded. So is this like the next step? Like what are these delusions know, that you're writing about?
Emanuel:This is an increasingly common type of interaction that we're seeing online. We're seeing it directly from the people who are experiencing these illusions. And we're also hearing from partners, relatives that are talking about family members, friends who are experiencing this. It's just people who are talking to chatbots like ChatGPT, like Claude, and they're coming away from those interactions with the idea that there is something really important on a cosmic spiritual level happening. They think that they've discovered artificial general intelligence, that the bot that they're talking to is sentient, that it is all powerful, that the bot is a god, that the bot is infusing them with godly powers, that we're on the precipice of humanity stepping into some new profoundly different state.
Emanuel:These kind of really over the top clearly not true assessment of what the chat bot is saying and doing and is. I mean, you mentioned
Joseph:the well, we see people talk about this online, but I didn't put this in the docs. Don't mean to put anybody in the spot. Maybe I'll just say myself. But people email us often.
Emanuel:I wanted to talk about this. Yeah.
Joseph:Yeah. Well, so just my own experience, we often get emails where people will say, Hey, I have a tip for a story and you're like, Okay, and you read the email and it's them saying that basically they think they found God in chatty or something. Was some of those email? I mean, I asked you to read them out, but is that what you were thinking of, Emmanuel, when we get emailed like that?
Emanuel:Yeah, I was thinking of that. So we're hearing about it from, like, random Reddit posts. Want to call out Miles Clee at Rolling Stone wrote a story about this where he saw one of these Reddit posts on rchatchypt, someone describing their partner falling for these delusions. He reached out and talked to that person and talked to other people that were experiencing the same thing. But I get and and we're seeing that as well.
Emanuel:People are emailing us directly as reporters who cover the AI beat and put tip lines in our articles, inviting people to tell us interesting things that they're seeing with AI. And we're hearing from people who are, claiming that they're having these experiences. And what I wanted to say about that is, like, even to zoom out from that as reporters, we're, like we're not famous. We're not celebrities, but we are public figures who are inviting people to tell us stuff that is interesting. And I know that we have all experienced people who are obviously going through some sort of mental health crisis reaching out to us about stuff that is clearly not true.
Emanuel:Like, there's a lot of gang stalking. There's a lot of, the government is spying on me. Yeah. Way before AR. Years and years.
Emanuel:For as long as we've been doing this Yeah. We've experienced this kind of thing. And the behavior is very similar. It's just that it has shifted over to this other area of focus, which is AI. Joe, I'm sure as like a privacy and security reporter who does cover a lot of legitimate cases of government spying and government monitoring and private private people spying on other private people, you probably get a lot of tips about, like, people are spying on me.
Emanuel:Like, my neighbor is spying on me. But it is clearly, unfortunately, a delusion. And it's a it's a it's a very similar type of behavior. There's a lot of commonalities in these emails. It's like they're typically very verbose.
Emanuel:It's all conjuncture. There's never evidence. Sadly like a really desperate tone to them, like somebody who is like in a crisis. So, yeah, it's the same thing. It's just that that behavior has shifted over to to AI.
Emanuel:Sorry, I just want to add one more I'm not a psychiatrist or a psychologist or like a a doctor and I'm going to try my best to avoid like diagnosing and using clinical terms. If I am using one, like I'm using that by accident and like I'm struggling to define what is like a very complicated health issue. To put that at the top as we continue to talk about this.
Joseph:Yeah, we're not gonna say anybody is specifically schizophrenic or anything like that. It's just like, these are emails we get. Yeah, you're right that we've had this for a long, long time. And I don't think we have any data on this. I don't know how we would get this.
Joseph:It'd have to be an academic study, obviously, but I would be genuinely curious in that. Is it the same sort of people who have moved from emailing a journalist about gang stalking? Or I have a five gs chip in my head, which is another very common one, right? Yeah. Have they then moved to then I'm gonna wait for that.
Joseph:I'm gonna mark that one second. Sorry. 639. One second. One second.
Joseph:Have they moved to? Is it the same person? Basically, that's what I'm trying to get at. And I was just gonna ask Sam, have I'm sure you've received emails like this as well. Right?
Joseph:Have you got the sort of delusional AI ones recently?
Sam:Yeah. Here and there. I mean, it's tough. Right? It's like we read everything people send us and we want to make the time to take these things as seriously as possible and want to kind of tread carefully with this stuff.
Sam:But at the same time, it's just, it's not really our place to help people through this sort of thing. I think that was kind of what was scary about the ChatGPT or not ChatGPT, the Meta AI chatbot thing is because people are turning to these chatbots for help. Like, they can't get help elsewhere from real human therapists because that's expensive maybe or they don't have insurance or, like, whatever barriers. It's like, when you're already struggling, it's really, really hard to even begin to take the first baby step toward talking to a person or saying anything out loud. So they're turning to chat GPT about it, and they're talking to it like it's a therapist.
Sam:But at the same time, it's like this isn't a tool that you should be using for your mental health because then it can do something like this. Like, can trigger serious mental illness in some people and seemingly in some people's cases. That's kind of what's happening. And that's a big problem with the chatbots is that they just say whatever you wanna hear, basically. Even if you're like, tell me the truth.
Sam:It's like they're it's a chatbot. It's not it doesn't understand how to, talk to you without harming you further.
Emanuel:I wanna add to that, that none of us is saying that chatbots is the cause for anyone's mental health problems, but it is also notable that there's a whole marketing propaganda apparatus around AI that is saying that it can be a therapist, that it can be a friend, that it is very powerful, that it will replace humanity, that it is smarter than any living person. So obviously, I personally, again, a doctor, I don't think that AI chatbots are making anyone sick, but I'm not surprised that people are turning to it and trying to find great meaning and help with them, whether the technology is capable of that or not.
Joseph:Yeah. So there's unfortunately a lot of these people experiencing AI delusions or their loved ones or people they know are doing that as well. But then this particular story, as the headline says, is about this pro AI subreddit. We'll get into what they're doing exactly. But just what is this subreddit?
Joseph:First of all, Emmanuel, like, what's their deal on a normal day, you know?
Emanuel:Yeah. So there's a bunch of AI subreddit communities. Obviously, there's artificial, there's singularity, futurism, and they all a lot of what they talk about is AI. And I would say honestly that they are pretty much like pro AI or positive about AI. They're definitely very curious and interested in it.
Emanuel:But those subreddits also share our stories a lot of the time, which are doing hard reporting about AI companies and what people are doing with AI tools, which is often not great stuff. And the subreddit that I cover in this story, rAccelerate, is a direct response to those other communities, which the people behind rAccelerate think is too critical of AI. They call them D cells, which I hope I no longer have to explain to people what that means because there's currently a big HBO movie about four tech moguls who are kind of like talking about how to manage the world. It's called Mountainhead and they use the word decelerationist and desales a lot during the show. So hopefully this is the last time I have to like explain what it is, but these those are basically people who wanna regulate or control AI in in any way.
Emanuel:So the the subreddit is, like, named after, like, the opposite of name and it's named Accelerate.
Joseph:I promise you, people are gonna keep asking. And I even just goo I just did Google Trends for D cell, and there's, like, a massive spike.
Emanuel:So
Joseph:people are people are clearly watching matter, like, Googling what's D cell, which I know, that's pretty funny. Okay, so they, those subreddits, they have they have their thing, have their stance, but they're also dealing with these AI delusions. How is that appearing on their subreddit exactly?
Emanuel:Yeah. So the you know, this was basically an excuse just to write about this phenomena of, like, these AI delusions. But what is happening on Accelerate, which is very positive and bullish and, about the future of AI and wants, AI to progress and change the world. But they're dealing with so many people who suffer from these exact delusions that they announce, like, the moderator, like, sticky down announcement to the subreddit, which is not something that happens often, to explain a new moderating policy, which is they're just going to ban and remove all comments and threads of that type without drilling down into each individual case because there's so much of this happening. The moderator said that they recently banned almost 100 people who were doing this.
Emanuel:And I think the fact that even a very pro AI subreddit is having to deal with this problem, acknowledge that it is a problem, says something about how common this is.
Joseph:Yeah, absolutely. And I think just the last question and to be clear, as you said, we're not doctors, we're not licensed practitioners or anything like that. So the way I originally framed this question, I'm going to say it, but I don't think we can answer this. It was simply like, where do we think these delusions come from? I don't think we can really touch on that.
Joseph:But what I could say is, well, obviously there's unfortunately people who have mental health issues as well. Maybe it's an unfamiliarity with how tech works. You know, ChatGPT is so accessible by hundreds of millions of consumers now or whatever it is, right? Maybe that's part of it. And then, of course, there are media reports in the somewhat recent in somewhat recent history where reporters basically big up the capabilities of these chatbots by anthropomorphizing them essentially and comparing them to humans in a way?
Joseph:Like, what do you what do you
Emanuel:think of any of those? So not only are they hyping up AI, they are there's something in AI called the Eliza effect. Sam, if you remember or if you can look up when that was first conceived of, please tell me. But basically for as long as there have been chatbots, there has been this observed behavior where because the program that you are talking to is conversational, people inevitably project a personality onto it and believe that it is more of a person or an entity than it really is. All it is is code.
Emanuel:And there has been a lot of very unfortunate reporting that kind of amplifies this idea. Specifically, there was like a very huge viral piece in the New York Times by Kevin Roose that was about him writing in the first person about talking to god. I think it was chat GPT. I'm not sure which bot it was exactly. And he was talking about how the bot fell in love with him and was urging him to leave his wife and all of that.
Emanuel:And it gives people the idea that there's, like, a conscious being on on the like, the the chaplet is a conscious being. And there's been it's not just the New York Times. There's been a ton of reporting where, like, the entire conceit of the article is like, I talked to Chatchibi Tea and it did this, you know, and it told me that. And that just, like, really amplifies this wrong idea and, like, this known, you know, psychological impact that chatbots have on people. And that's clearly one thing that is happening here with these delusions.
Emanuel:And it's a very old idea. Sam, do you know the the year? Like, how old the
Sam:Yeah. I mean, Eliza as a program was from the sixties.
Joseph:Yeah.
Sam:And it was one of the first, if not the first chatbots, and it was used it was a therapy bot that that was kind of the the idea. It was a MIT professor. Was like, what if we made a therapist out of a a chatbot essentially? And, yeah, obviously, people, like, fell in love with it. They they had emotional attachments to it.
Sam:And this is the sixties. You know? It's like the technology isn't even the the entire point. The advance advancements of AI. It's people's nature to attach these kind of emotions to this blank slate sort of, you know, thing that's just absorbing their own emotions and reflecting it back to them.
Emanuel:Yeah. It's it's it's a known issue for, like, sixty years and people are still fucking it up. We talk to a lot of really smart AI experts and skeptics and they'll let us know, like, when we do little things like when we say, oh, chat GPT thinks or chat GPT feels or and it's like, they're like, no. No. No.
Emanuel:It's not thinking or feeling anything. It's just like it's all code and it's all probabilistic regurgitating of training data.
Joseph:Yeah. Oh, and that Kevin Roose piece, was about Bing's chatbot.
Emanuel:Bing. That's right.
Joseph:Because
Sam:because Bing it. Yeah. That's so embarrassing.
Joseph:Yeah. That makes it worse, actually. But yeah. Because I just remember Microsoft was rolling out at the time. Absolutely.
Joseph:And I don't know. The the last thing I'll say is and I think I've mentioned this on the pod before, but just this isn't even so much about the delusions, but just the blind trust that people put into these bots where, I don't know, somebody reached out when I was covering the Anom FBI encrypted phone stuff. There was this big mystery about what was the third country that held the f helped the FBI. And this tipster kept using chat GPT to ask it, and it was like, hey, look. I asked chat GPT what the country was and said it was The Netherlands.
Joseph:And I hadn't reported it yet because I was working on verifying, but, like, I know it's Lithuania. And like this guy was just like convinced. I'm just like, what the fuck are you talking about? Like, I don't know, just people's blind trust in it is like best case scenario. They get something wrong.
Joseph:Worst case scenario, they have an AI delusion, and it's terrifying, you know, for everybody involved. Alright. Let us, leave that there, and I will play us out. As a reminder, 404 is journalist founded and supported by subscribers. If you do wish to subscribe to four zero four Media and directly support our work, please go to 404Media.co.
Joseph:You'll get unlimited access to our articles and an ad free version of this podcast. You'll also get to listen to the subscribers only section where we talk about a bonus story each week. This podcast is made in partnership with Kaleidoscope. Another way to support us is by leaving a five star rating and review for the podcast. Here is one of those.
Joseph:Also the only podcast I pay for, but also paying for the well written articles. Bonus feed has the sweet subscriber only episodes with the go hard music that I know everyone loves. Help us tech newsners keep four zero four findable. Thank you so much. This has been a four zero four media.
Joseph:We'll see you again next week.