The 404 Media Podcast (Premium Feed)
from 404 Media
The Underground Trade of Car Hacking Tech
Episode Notes
/Transcript
YouTube version: https://youtu.be/r-ZEfQs4ZSA
Hello, and welcome to the four zero four media podcast where we bring you unparalleled access in the world both online and IRL. Four zero four media is a journalist found company and needs your support. To subscribe, go to 404media.co. As well as bonus content every single week, subscribers also get access to additional episodes where we respond to their best comments. Gain access to that content at 404media.co.
Joseph:I'm your host, Joseph. And with me are the four zero four media cofounders, Sam Cole
Jason:Hey.
Joseph:Emmanuel Mayberg Hello. And Jason Keppler.
Jason:It's sad to do this virtually.
Joseph:Yes. Because, of course, last week, we had our second anniversary party in New York where we, well, drank and then also recorded a live podcast. I can't even remember what we spoke about. I think it was mostly q and a. But, yeah, it is way easier to do it in person than it is virtually because there's ever so slightly delay whenever you do it like this.
Joseph:Jason, what did we talk about in New York? It was questions mostly.
Jason:We we answered questions. It was we put questions in a jar or people wrote questions in a jar, we pulled them out, and we read them. And it was really fun. Went really well. It will be out on the feed in two days, so you can listen to it if you weren't there.
Joseph:Yeah. Sounds good. Sam, what did you make of the party as the person who, thank god, organized the vast vast majority of it?
Sam:I had a blast. Yeah. I think listen to the podcast when it comes out on the feed if you were not able to get into the room because the room itself was so full that people got pushed out of it entirely. But, yeah, it was a really good time. It was great to see everybody.
Sam:So cool to hear so many people listen to the podcast. I always am really excited to hear that, so thank you, and hello again.
Jason:First ever tech blog podcast, mosh pit.
Sam:True. Jason crowd surfed
Jason:Yeah.
Sam:A little bit. Uh-huh. It was good.
Joseph:Uh-huh. All I can remember is the question about what statue you would be or something. And I'm not gonna spoil the answer, but that's the only one I can remember because I answered it, and then Sam immediately took the piss out of my answer.
Sam:And so You and Emmanuel both. I wanna change my answer, though, to the Carlos Atticus laptop statue that just got revealed. He's holding a laptop as a saint. It's a statue that's dedicated to So that's my new answer.
Joseph:Alright. Well, you'll you'll see the episode on your feeds soon, and you can listen to the whole the whole podcast and whole event then. As for this week, Emmanuel, do you wanna introduce this first story?
Emanuel:Yeah. I'm taking I'm taking over. The first story is from Joe. The headline is Inside the Underground Trade of Flipper Zero Tech to Break into Cars. A lot going on there.
Emanuel:Let's get into it. The story starts with a description of a video you saw. Tell me about this video.
Joseph:Yeah. So it's first person perspective, as in clearly this person is holding a mobile phone, I believe in their left hand, and then this small white and orange gadget in their right. And it has a little screen. It has a few buttons like a d pad and I think some other stuff on the side. It has some sort of short antenna sticking off as well.
Joseph:And this person walks up to a nearby car. It just looks like some sort of normal sedan vehicle, tries to open the door and it's locked. They then push some buttons on this little gadget, try to open the car door again, and lo and behold, it now opens. Of course, this is a demonstration of the tech that we're talking about here. And you may think, well, we've seen car unlocking tech before.
Joseph:What's new about that? And that's totally right. And we will get into that, but it's really about the specific attack involved here, the gadget itself, which we'll definitely talk about, and sort of the proliferation or distribution of this tool, which is if it falls into the wrong hands, it really is threatening to sort of supercharge car thefts around The US, which I don't like to do a prediction and, like, I I really tried not to. It was more the people I spoke to were predicting that would happen, but the writing's on the wall.
Emanuel:Yeah. So let's talk about the gadget. What is he holding in his hand? What is this flipper zero?
Joseph:So the flipper zero is a pretty well known or maybe infamous piece of hacking gear now. It came out a few years ago, started on Kickstarter, raised about 5,000,000, something like that. And the attraction was that it's basically a Swiss army knife for hackers. So you can imagine physical penetration testers. These are people who well, we need to check this the physical security of a data center, let's say.
Joseph:So we have to keep people out and make sure they can't put USB keys in here and plant malware in the data center, whatever, or corporations as well, banks, all sorts of stuff require physical security and digital security at the same time. A lot of that involves cloning key cards like RFID key cards or playing around with NFC, that protocol as well. And what you used to have to do was buy all of these different hacking tools. Like, I've had ones which can clone key cards, that sort of thing. But the Flipper Zero was bringing all of those into this one thing you could just carry in your pocket.
Joseph:So you could do those sorts of attacks. You could also do WiFi, do you de authentication attacks, which if anyone has been to especially DEFCON or maybe CCC in Germany, You've probably seen this where you keep getting kicked off the Wi Fi or you can't connect to the Wi Fi because some person is messing around and performing a Wi Fi attack. This gadget can do that as well. So we can do basically well, not everything, but a lot of the things a legitimate security researcher or penetration tester would want to do if they're on a deployment. So we launched on Kickstarter, raised all this money, and now, I mean, it is a mass production.
Joseph:You can go to the Flipper Zero website and you order one of these gadgets. So think for a couple $100, you get it in the mail. And it's also been demonized over the past few years cause it does crop up into the news every so often. Sometimes legitimately, sometimes in a way that, well, it didn't really deserve to be pulled into it, into whatever story. But here, this story really is about the flipper zero because it is so accessible.
Emanuel:It is demonized, but can we just spend like a second talking about Flipper Zero's own branding and image? Like, it is fairly like a feel good positive, I don't know if you would call it white hat. We covered it a lot because I think it launched when we were in motherboard, it would come up in like these kind of fun blogs that people were doing like fairly innocent, cool things with it. Is it fair to say that like their own image that they put out there is fairly, I don't know, posy? Yeah.
Emanuel:Absolutely. And I
Joseph:think they're very, very smart to do that.
Emanuel:The mascot is like this cute dolphin. Right? Like
Joseph:Yeah. I'm looking at it right now, and on the because on the device itself, it has this little screen and you can see the dolphin, like, it, almost talks to you. It looks like a Tamagotchi, like a pretty high res Tamagotchi in a way. The marketing is cutesy. It's fun.
Joseph:It's experimental. It's playful. And that all makes sense because the flipper zero and the community around it can be all of those things, often is all of those things, often is playful. It is about experimenting, and that is the hacker ethos at the end of the day, right, for a hell of a lot of people. It's a, hey.
Joseph:We just wanna figure out how stuff works. We wanna have fun, but we also want to keep stuff secure, and that's why it's a legitimate security tool. They probably it'd probably be a very bad idea if they had dark, grizzly hacker hoodie marketing, which ironically, some of the custom software we'll talk about in a bit actually does have that marketing as well. But, yeah, it's a legitimate security tool, but a dual use security tool.
Emanuel:So what's different about this implementation of Flipper Zero? It's the software. Right?
Joseph:Yeah. The software and the firmware as well. Firmware just being, you know, basically a piece of software that acts, you know, a little bit lower down in the stack of applications, operating system, blah blah blah, all all of all of that sort of thing. So the Flipper zero is powerful as I laid out. It can do all these sorts of attacks, but some people want it to be able to do more.
Joseph:They want to really screw around with people. So some have made their own custom firmware for the Flipper Zero. And when I looked at in particular was called Unleashed, and this just allows, you know, like more types of attacks and RFID attacks and all all of that sort of thing. So people because you can basically fully customize the Flipper Zero and that is part of the appeal, people have made them more much more powerful firmware, and then you can get more patches on top of that as well-to-do very specific things, I'd say.
Emanuel:So how does this specific attack work and how does it compare to some of the other high-tech carjacking or not carjacking, car stealing methods that we've seen in the past couple of years.
Joseph:Yeah. So as well as the custom firmware, people are making these I mean, I call them patches in the article. I guess you could call it software. You could call it tweaks, you know, scripts. It doesn't super matter.
Joseph:But when you have this custom firmware, you then make and install and use these custom patches. And what I was shown is that there are a bunch of them going around which are specifically to unlock and interact with particular vehicles. So, you know, I was sent one for the Kia. I was sent one for the Suzuki. And to step back a little bit, car theft is obviously a problem.
Joseph:On the lower end of the spectrum, you have people like the Kia boys, which maybe we'll go into more detail later. But there's this phenomenon. It's on TikTok. People steal cars. They drive them around.
Joseph:Milwaukee, I think, primarily, just as where the phenomenon started. And they will smash a window, go in, steal the vehicle because there's very poor security inside the vehicle itself. Okay. Halfway in between, maybe over halfway, you have people using techie gadgets to break into cars. And then in the same sort of area or a little bit further, I think you have people starting to use these flipper zeros.
Joseph:So usually, if you're a tech focused car thief, you'll do something called a relay attack, which is, let's say, you wanna steal a very nice looking car that sat on a driveway, but the key fob and the owner is in their house. Well, you use a couple of devices to basically extend the range of that key fob that sat in a a fruit bowl on their kitchen counter or something, which usually can't reach the car. But now you're using these couple of devices to relay that signal over to the car. You activate it. You can get in the car and drive away.
Joseph:And this has been a huge problem in The UK. UK police will upload photos of this happening, caught on Ring cameras or CCTV. In The US, it's done as well. I imagine Europe, but I can't remember off the top my head if I've seen the case from Europe. But that's what usually happens.
Joseph:And in response or perhaps in parallel, a lot of car manufacturers introduce something else called rolling codes, which is, well, even if someone gets the signal and tries to use it to open the car or whatever, we're gonna change the code every single time. So they can't just, you know, relay the signal they get into the car or whatever. It's supposed to be a protection for that.
Jason:This is very similar to how tickets work now. Like, when we're talking about, like like, barcodes to get into concerts, it used to be that there was just, like, one barcode, and then both Ticketmaster and AXS went into something where the barcode refreshes every ten seconds. Right. And I feel like it's a very similar technology possibly. We don't need to get into it, but it it's very similar because, like, people figured out how to crack that, and then basically, they're like, oh, we need to, like, change this constantly.
Joseph:Yes. So yes. Because you bring that up that they did crack it because that's exactly what's happened with cars as well in that the people I spoke to, a couple of hackers, they say they've been able to essentially reverse engineer how those rolling codes work by a lot of development, a lot of reverse engineering, and apparently buying source code related to particular vehicles as well. And what the Flipper Zero custom patches are able to do is basically figure out what the next code is going to be. So they push that and it's like, oh, even though the code has changed, this device knows what it is in advance.
Joseph:And that's sort of what makes us different to previous attacks, Emmanuel, to finally answer your question after several minutes of me going down that rabbit hole.
Emanuel:Yeah. I don't know if you know, but shouldn't the rolling code problem you just described be solved by now? Like, isn't there, like, common forms of encryption and, like, randomly generating numbers that should not allow them to, like, reverse engineer what the next code is? Like, are they using really old software to to do this? Or
Joseph:You would think that, and I often think that sort of car manufacturer security is similar almost like industrial control systems or airplanes sometimes where they just use these very antiquated ways of going about things. And the actual patches I was sent, again, the Kia and the Suzuki, which are not luxury vehicles, obviously. I I've spoken to people who've sold tech to break into luxury vehicles specifically. But if you go through those scripts and other screenshots I was sent, I was sent, it actually shows there was no encryption on those. So, of course, it varies from car model to car brands to manufacture manufacture or whatever.
Joseph:But the hackers who go by Daniel and Darrow, they have this PDF, which is a couple of pages, but it has like 200 different models of vehicle and maybe a dozen or so brands overall, you know, not just the Kias and the Suzukis, but I think there's Ford on there, Fiat as well. I'm bringing up now Skoda, Citroen, Peugeot, Mitsubishi, Honda is in development. So whereas the Kia boys were obviously targeting Kias, and I think Hyundai is as well because the security was crap. This seems to be a potentially bigger issue because it's unlocking all of these other cars Right. Which which have presumably better security.
Joseph:One of
Emanuel:your sources mentioned the Kia boys as well. Why is that?
Joseph:So my source, one of the people I spoke to who's sort of in this community, and they were the one that was actually reverse engineering some of these patches. They mentioned the Kia boys because they think that eventually they will become flipper boys. Again, Kia boys, they break into Kia's and Hyundai's, RareRo, low tech. For a long time, you've been able to get phone unlock sorry. Not phone unlocking, car unlocking technology, but it's been pretty expensive.
Joseph:You try to go buy one of these online. And if you don't get scammed, which I think could be a big thing, it's gonna cost you potentially thousands of euros or dollars to buy a device like this, like a code grabber or a relay attack tool or something like that, what the Flipper Zero potentially could do here is that, well, it's just $200. So somebody with not as much money to spend who but but who still wants to mess around with cars like the Kia boys could potentially get a flipper zero and mess around with it that way. And I should just say that Straight Arrow News covered this particular attack before. They tested out and they verified it worked on a vehicle, which is very interesting and that made my job a little bit easier with this article.
Joseph:But this was focused much more on the trades, which is that you go to Daniel or Darrow, the two hackers who are selling this tech, one for Flipper Zero, one for a Raspberry Pi, and you have to pay them between 600 and a thousand dollars. So still an investment, you know. People I spoke to, that software is leaking out. They're reverse engineering it so you don't have to pay the money anymore, and there's this looming threat. Well, maybe it's gonna leak, and it was sent to me.
Joseph:So clearly, is leaking. I'm not gonna post it online so then everybody can just unlock cars. That would be an insane thing for journalists to do. But I wouldn't be surprised if this trickles out very, very soon, and that is why it could apply to communities like the Kia Boys,
Emanuel:for example.
Joseph:Because they might get hold of it eventually.
Emanuel:Yeah. It's like in case it wasn't clear, at the moment, this software is fairly exclusive both in the price and just like who has access to it, but it's software. It is not hard to imagine that it leaks and becomes more widespread. Finally, you mentioned that Flipper Zero, for good reason, I think is, let's say, aggressive in explaining what its technology does and defending its brand. So I'm not super surprised about their reaction reaction to the article, but I thought the article was very clear and very fair about what the Flipper Zero's role in it is, and yet they had like a pretty wild response to to the article.
Emanuel:You wanna you wanna talk about that?
Joseph:Yeah. So initially, when I'm reporting and writing the article, I ask Flipper Zero for comment. I actually explained to them in my email saying because I wanted to make it clear to them. I'm not a random journalist covering this. I've been covering hacking and cybersecurity for, ten years, and I said, I understand that this tool this is not what the Flipper Zero is designed for.
Joseph:It's for legitimate security researchers. I already know all of that context. So, basically, as a wink wink to them, you don't need to patronize me by by telling me that sort of thing. They then gave a very, very thoughtful statement, which reiterated, yes, it's a dual use technology. It's used by security researchers, but they very rightfully said that this is much more about the car manufacturers and their poor security.
Joseph:And I totally agree with that. We published a story, a few outlets aggregator, you know, Gizmodo and that sort of thing. The sort other sort of technology websites that would cover a sort of car hacking thing. And then Flipper Zero comes out with with a pretty well, I mean, you said as expected, I I guess I'll say unusual blog post, and they have the headline, can flipper zero really steal your car? Spoiler, no, it can't.
Joseph:And then they do like a mocking tweet. The media is talking about how flipper zero is on the dark web, can break into like, really sarcastic and over
Emanuel:the top. There's an image of their, you know, mascot dolphin pulling a hacker and a Guy Fawkes mask out of the garbage, and I was like, is that Joe? Is that Joe in the supposed to be in image? But no. I guess it's like the hacker, and there's people in in cars around it looking angry.
Joseph:Yeah. Just just for people. But then yeah. So so there's the sarcasm and, like, the defensiveness, which is okay. I don't know.
Joseph:I thought the article was pretty fair. I thought your statement was great. And then they go through some technical stuff, which they say it relates to certain vulnerabilities, but my understanding is that it relates to actually to a different it it relates to the rolling codes, and they don't seem to fully understand it relates to rolling codes essentially without getting too technical. So I don't know. It seems very defensive about it.
Joseph:And I can again, I kinda get why, like, the Canadian government tried to ban flipper zeros before. Like, they've had to deal with some very, very crazy stuff. But, hey, this is a legitimate concern now because criminal hackers are making this for the flipper zero, and I'm afraid that's just a fact.
Jason:Yeah. I I wanted to talk about this because, I don't know, I I just think about this stuff a lot. Like, I I really, really think about how our work is perceived and and also, like, how we are in this mess, this mess being, like, a total collapse of trust in the media and that sort of thing. And it's stuff like this. It's like this is a really bad blog post by them.
Jason:It's very damaging because they're they they're a very popular product, as we've just discussed, and there are a lot of people who have no idea how journalism works. They have no idea how, like, the information ecosystem works, etcetera. And this blog post is very much, like, in the vein of fake news. This has nothing to do with us. This is totally wrong.
Jason:And that like, you multiply that across, like, the entire like, every politician's been doing this for many years. Lots of lots of companies have been doing this for years. Like, anytime that there's any sort of, like, negative coverage of anything, there the sort of gut reaction is like, this isn't our fault. This is, like, journalists sensationalizing something. And it's like, this was a very thoughtful article that was very clear about the issue.
Jason:It's a very serious issue. And it's like, I think Flipper Zero is very cool. Like, I I have long thought that this is a very cool company. I've long thought that this is, like, a very cool device that's being used for, like, really interesting security research. I still believe that, but it is, like, really disappointing to see a response like this because the response is one that is, like, it's just not in good faith.
Jason:It's, like, not it's not a good response, and it's very disappointing. Like, I think it's fucked up. And I don't know. Like, maybe I'll see Flipper Zero representative at some hacking conference at some point, and they'll be like, why did you say something mean on the podcast? But it's like, this is super damaging.
Jason:Like, it's really damaging to trust in journalism, to trust in in this sort of thing. And I understand that there are politicians who, like, blow stuff out of proportion and that are trying to ban them and that, like, they really wanna make sure that their product is seen as something that is, like, a force of good and a force of research and a force of, like, STEM tools and teaching kids about stuff like this. And it's like, yes. These car manufacturers used bad security for their their key fobs, like, a long time ago, but this is, like, a factual article that describes something that is happening. You can't just, like, stick your fingers in your ears and close your eyes and be like, blah blah blah.
Jason:This is old. This is not happening because it is happening. It's a it's a really disappointing response.
Joseph:Yeah. I agree. Especially, as you say, they are a legitimate actor in this space and they do really good stuff. Then to have a response that isn't as good faith kind just undermines it all.
Emanuel:Before we move on, to tease it somewhat, our next story is about nostalgia. And when I think about the Flipper Zero, I always think about this Casio watch that I just posted in Slack. Did you guys have this? Did you have have a naughty kid at school that ever used one of these?
Joseph:A calculator watch? A what? The
Jason:calculator watch?
Emanuel:No. Not the calculator. It's like, it can copy the whatever, the RF signal of a remote.
Jason:Oh, it's the TV remote watch.
Emanuel:Yeah. So there was always a kid in class who could, like, turn off the TV if we were watching something or, like, turn off the air conditioner. It always it's kinda like always reminds me of the Flipper Zero. It's a it's a proto Flipper Zero.
Joseph:Yeah. Definitely the same sort of vibe. We should all get these. Just wear them. They're cool.
Joseph:Well, since you teased the article, Emmanuel, I'll just take you to the next one. When we do come back, it is about nostalgia, specifically the eighties, and of course, AI slob. We'll be right back after this. Alright. And we are back.
Joseph:This is one that Jason wrote and I believe with Matthew Gault as well. Yes. I opened the link to check that. The headline is eighties nostalgia AI slob is boomifying the masses for a past that never existed. So, Jason
Jason:Stranger Things, Ask videos, AI videos.
Joseph:Right. What is the latest AI slop then? Is it what is this meta that we're seeing now?
Jason:It's interesting because these are not new. Like, people have been making these for a few years. It's just that they've caught on recently because, I don't know, they've hit they've hit the algo in a certain way. But, basically, it's like, imagine you're watching Stranger Things or The Goonies or something like this. It's like very prototypical, like, eighties TV show slash movies, like teen teen after school special vibes.
Jason:And so they start basically with, have you heard what's going on in 2025? Like, they have they're always scrolling through their phones, and they have this thing called Instagram, and it makes people upset. And they're like, wouldn't you love to go back to 1985, you know, where we drink Coca Cola and have a nice time out of glass bottles and I don't know. It's just like it's nostalgia porn. It's like you can imagine it.
Jason:Maybe we'll play a little clip here.
Joseph:You made it back. How's 2025? I heard no one talks face to face anymore, and everything's lived through something called social media. I heard people don't even know their neighbors anymore. They don't knock on doors.
Joseph:They just talk through something called Instagram. What the hell is Instagram? Heard in the future. No one even goes outside anymore. Everyone just stares at screens all day.
Jason:But these are going wildly viral, and they're just, like, really annoying and really upsetting. And they're real they're they're a type of AI slop that is super just like it seems innocuous on its face, but it is something where it it, like, imagines this false past that didn't actually exist. I was born in the eighties, but I don't remember them because I was born at the end of the eighties. But as I understand, like, the eighties sucked ass. It's like mortgage rates were, like, 1000000%.
Jason:Unemployment was, like, 25%. You know, there there was various it it was not, like, a good time. There there's all sorts of, like it it was also as many periods in American history are was, like, not the most progressive of era eras for, like, cultural things. It was the era of Reagan. It just, like, was not, like, a good time.
Jason:But, like, these AI slob videos sanitize all of that, and they just remind you, like, don't you wanna live in in the Goonies and back to the future and things like that? And they're not, like, hiding that they're AI in any way, shape, or form. So these are not, like I don't think that they're tricking people. I think that people understand that this
Joseph:is But not deliberately.
Jason:Well, you know, I think that people understand, like, oh, this is, a fantasy type vibe, but it's like I think I describe it as, like, astral projecting to just, like, a shitty past that never existed. And it's it's kind of, like, emblematic of the type of AI slop that we're starting to see go viral where it's, like, not intended to trick people, but it's just, like, allowing people to live in a fantasy world that lets them kinda, like, ignore the all of the decisions that have led to the current situation that we're in. And then, also, it's like we laugh. We laugh and we laugh and we laugh and we laugh at Mark Zuckerberg's shitty Horizon Worlds metaverse that no one uses and that no one goes into and where people don't really have good legs and where they spend billions and billions of dollars. And it's like, yes.
Jason:Like, let's laugh at that. It's very stupid. But, like, what we're seeing is that you don't need the headset and you don't need the metaverse to allow people to transport themselves into this, like, nostalgia reality that's very similar to the stupid, not very good book, Ready Player One, even worse film, I feel. That's just, like, full of references. It's just like, here's here's, like, a reference to a bunch of old shit, and it's like 1,000,000 views, like, you know, it's not cool.
Jason:I don't like it.
Joseph:So let me read out this quote and then, Sam, let me throw it to you because I feel like you were one of first ones out of us to to see these. But the the quote is from the article that Jason and and Matthew wrote. These videos, like a lot of AI slobs, do not try to hide that they are AI generated and show that there is unfortunately a market for people endlessly scrolling social media looking to astral project themselves into a hallucinatory past that never existed. This is Mark Zuckerberg's fucked up metaverse living here and now on Mark Zuckerberg's AI slob app. I know you touched on that.
Joseph:I think you just put it very, very well. Sam, how did you first see these, and what did what was your reaction to them?
Sam:Mean, I think I saw them like everybody else did, which was, like, in my algorithm. And I didn't really think much of it when I first saw it. I was like, oh, that's more bait, basically. It's just like engagement bait from AI creators, which Jason has covered quite a bit of. There's, like, a bunch of other versions too.
Sam:There's, like, a nineties one that I've seen. There's several versions of the eighties ones. There's wasn't there, like, a 02/1931, which I think is super funny? Because, like, I do recall 02/2003, and it wasn't that cute.
Joseph:What were they showing? Like
Sam:They were like, the game we're playing Game Boy sitting in the living room, and the movies are on in the background of the TV.
Joseph:It's just Who was playing a Game Boy
Jason:in 02/2003? Starting to do them. They're starting to make them for the pandemic. They're make they're like, remember 2020 when you logged on with your boys to Warzone? And it's like
Joseph:Hell yeah. Animal Crossing sign.
Sam:And, like, Joseph just yesterday?
Jason:Yeah. Yeah. Remember yesterday when things were so simple?
Sam:Yes, actually. Yeah. I mean, yeah, let alone the eighties being, like, the height of the AIDS epidemic. Let's ignore that, I guess. And, also, like, Challenger and Chernobyl happened.
Sam:Like, the eighties were I know. I know what my parents were up to in the eighties, and it wasn't like sipping soda pop at the sock hop. Like, it was not wholesome.
Jason:Have any of you seen heavy metal parking lot?
Emanuel:No. I have. I've that's what I thought when I saw the videos. Like, that is the genre that it's trying to AI generate.
Sam:What is that?
Jason:Heavy Metal Parking Lot is I believe they're outside of a Pantera show possibly at RFK or no. I think it's RFK Stadium or it's like the capital center in Maryland, and literally, it's just a documentary. It's basically like they're making TikToks, but in in 1985
Emanuel:There's or that one. There was like high school in the nineties, and there's, like, like, seven Eleven in the eighties. They're all the same genre.
Jason:Yeah. They're just talking to drunk people in a parking lot before a metal show, and everyone's, like, has mullets and is wearing, like, jean jackets and going, like, woo, like a lot. It's very good watch, but they that's the it's this. The as Emmanuel just said, it's this is it. This is it.
Emanuel:I'm gonna I'm gonna attempt to give a voice to Matthew Gault who co wrote this piece because I talk to him about this all the time, and I feel like this is his perspective and it's one that I agree with. And that is that part of what is alarming about the virality of these is that they are not clearly, right wing coded, but nostalgia is inherently a regressive desire that we express because it is literally about looking to the past and saying things used to be better rather than looking forward accepting change and trying to make that change better. Right? It's like and again, I don't think this is embedded clearly in the videos, but Make America Great Again is a nostalgic movement and statement. And I think that is always what is kind of toxic about this looking back and romanticizing the past.
Joseph:Well, at the same time that these are going viral or rather these are going viral now on TikTok or whatever, these AI generated ones. I don't know. Over the last couple months, if you care to log on to Twitter / x, you'll have some right wing grifter account post a video being like, remember what they took away from you? They being, I don't know, the left or black people or something. I'm not entirely sure who who they're reigning against.
Jason:Immigrants. There's a lot there's a lot of that now. Like, straight up straight up where they're like, look at the here's a chart of, like, immigration, and here's a chart of, like, something else and, like, crime or something, and they'll be like, look. Look at this.
Joseph:So they have that, and then they've been making videos like this, which I can't remember if they're AI generated or not, it would just be a white blonde woman sat on a car drinking some soda or something like that. And there's obviously a connection between those coming from right wing grifters and then people making this AI slot which is going viral, which just brings me to another question, Jason. Do we know who's making this AI slot? Because you've looked a lot into who makes this and often it's, you know, young men in India who are doing it to make money and game meta's platforms. Do we know who's making this yet or not not?
Jason:Well, so that's what I was gonna say. It's like the people who are consuming it are not boomers. They're like gen like, we make fun of boomers for, like, getting tricked by AI or whatever, and it's like, this is Gen X elder millennials, and they're like, remember when we used to play outside and know our neighbors and stuff? And it's like the people who are consuming this could easily get to know their neighbors. They could become like a force for good in the community.
Jason:They could talk to each other and, like, have a nice time, but instead, they're like, many people are, like, posting on Nextdoor about how, like, the kids are being too loud, and they're calling into their city council meetings, and they're, like, making it harder for, like, people to play pickleball at the at the park or whatever. So there's that. But then there's the fact that the people who are making this are one, we don't know everyone who's making it, but I found one account that had, I think, 800,000 followers on Instagram, and it does only this. And it is a guy it is a digital nomad who's in his twenties who lives in Bali, and his entire thing is I'm trying to become a billionaire. It's like, these people don't give a shit.
Jason:Of course, they don't give a shit. I'm on all of these Discords where this stuff is made and people are like, this is popular now. Here's how you make it. Here's how you script it. Like, people are eating it up.
Jason:Let's make it, and let's put it out there. It's like there's not even it's just so cynical. It's so it's without any sort of, like, political it has a political undertone to it, but, like, the people who are making it are making it because it performs and because they saw one thing that performed, and they're like, I'm gonna make 40,000 of these now.
Joseph:What and so for those for those who may be new to the AI slot phenomenon, I'm sure people have seen it, and we've obviously spoken about it on the pod before, but quite a long time ago at this point. Just could you explain that dynamic a little bit more in that there's a meta almost like a video game where, oh, this is the thing that's really strong on the algorithm now. What are just some other examples of what's been in the meta before?
Jason:I mean, well, Jesus is always in there.
Joseph:A reliable choice.
Jason:Religious content is a very reliable choice. Another thing that's done that doesn't necessarily require AI, but that is like a strategy used is clipping, like, popular podcasters. So just taking, like, Joe Rogan clips and posting them on your own account. There has been a lot of, like, Reddit, ask me anything type things. So they'll people will, like, just take content from Reddit and put, like, video game footage behind it and then have an AI voice read it, and it will be, like, am I the asshole type content.
Jason:I have seen a lot of just, like, really bizarre, like, horror stuff, like, grotesque things.
Joseph:It it grabs
Jason:your attention. Jump scares, like, things like that that are AI generated. There's a lot of, like, you know, SpongeBob content, a lot of Mickey Mouse content, a lot of like, we we've talked about it before, but a lot of, like, basketball players like LeBron James, Steph Curry, AI generated and put in, like, weird situations. And, like, this is just sort of the latest one. And, you know, a lot a lot of times, it's like a it's someone who who has a lot of accounts and posts a lot of stuff, and they see which performs, and then they make more of it.
Jason:And then when other people in this space see that stuff performing well, they hop on that same bandwagon, and then the platform gets flooded with just, like, tons and tons of either images or videos that are very similar. And at some point, there's diminishing returns because something else becomes popular, and so they they pivot off of it to the to the next thing. So it's, you know, meta just basically means, like, this is the strategy of the moment, and we'll we'll pivot off of it when things don't work. And the way that they make money is if you go viral, you often get, like, a fraction of ad revenue if you're on YouTube, or you get paid out through these, like, bonus programs on TikTok and on Instagram where it's, like, it's basically the same thing. It is a portion of ad revenue, but they don't call it that.
Jason:But you basically get paid by view and by engagement, and it can add up to very real money.
Joseph:Yeah. I guess we're gonna see what the next meta is going to be. I'm not gonna make any predictions. Who knows at this point? We'll leave that there.
Joseph:If But you're listening to the free the free version of the podcast, I'll now play us out. But if you are a four zero four media subscriber, a paying one that is, we're gonna talk about how Citizen, the crime awareness app, is pushing alerts written by AI basically with no human oversight, which is not good. You can subscribe and gain access to that content at 404media.co. We'll be right back after this. And we're back in the subscribers only section.
Joseph:This is one I wrote, but I think I'm probably the least familiar with citizen here. Maybe that's not true. But the headline is Citizen is using AI to generate crime alerts with no human review. It's making a lot of mistakes. Jason, maybe you just wanna tell what people people what Citizen is before I ask others whether they use it or not?
Joseph:I swear we've spoken about this on the pod before, but I'm just gonna do it again.
Jason:Citizen is an app that it bills itself as a public safety app, and it uses your location. So you open it up and you see a map, And on that map will be a bunch of things that you can click, like little points of interest, and these points of interest are all either crimes or, like, nuisances or, you know, someone being loud. Like, it's all sorts of different stuff. And as, you know, we've reported in the past, that can range from anything from, like, the police are here and there's an active shooting to, like, man is yelling at dog or something. You know, I've seen I get a I get a lot of you can get alerts also.
Jason:So it's like it it if you give it your location, it will give you an alert when there's, like, a quote, unquote crime or something of note happening near you. And I've gotten them where it's, like, kids playing with plastic swords, for example. So it really, like, ranges the gamut. And for a while and I guess still, it's, like, largely crowdsourced, but then they also hook into police radio. And I guess that that sort of brings us to, like, this what they're doing here.
Joseph:Yeah. I'll just briefly say that they hire employees and also contract workers to listen to police radio, to figure out if something important is going on, quickly transcribe that, and push an alert or a notification based on that. So shots fired x y z or whatever. That said, before we go into the story, Sam, do you use Citizen, or do you know anybody who uses it? Or
Sam:I have downloaded and deleted and redownloaded Citizen about 400 fucking times. So, no, I don't use it actively. It's not on my phone right now. But living in a city one of the best parts of living in a city is when something goes down outside your window, running to the window, and being like, what's So going and I mean, like, beefing or, like, arguing, not, like, serious things. But, like, when you hear, like, sirens or something, you're I'm always like, I'm nosy.
Sam:I'm always like, what what's going on? What is that? Or you see a helicopter, and there's a police helicopter circling my house pretty frequently for some reason. I download Citizen, and I'm like, I wanna see if anybody knows what that is. And then I'm disgusted once I find out or don't find out, and then I delete it immediately because it's very stressful.
Sam:Like, the push notifications are constantly, like, crazy shit happening half a mile from you. It'll be like, man with a big stick running around on your block, and it's like, it's just constant. Yeah. And that's just, yeah, that's just Joe running around outside my house, which is weird. It's it's very, like, anxiety inducing, which I think is probably a big part of why the app is as successful as it is because people stay on it for that reason.
Sam:They wanna know what's going on, you know, if a scary thing is happening in my neighborhood.
Emanuel:I'm gonna assume rudely assume that I'm the only one who's seen Bo is Afraid. Has anyone else seen that? Is Alyssa our producer? Did you see it? Yeah.
Emanuel:She's nodding. Okay. So she she'll understand. But there like, the movie, basically, like, the conceit is, like, what would the world look like if you entertain every single anxiety and fear you had? What if all of that was real?
Emanuel:And there's this really amazing sequence where the main character lives in a city, and it's just like the city is a portrayal of this nightmarish crime ridden city as if it's like the reality portrayed on Fox News. It's like he runs to his apartment building door, and there's drug addicts chasing him. There's a naked guy with a knife stabbing everyone. Like, everything is just like this crazy, violent, comically scary world. And I haven't used Citizen, but I have used, like, the Ring equivalent that is, like, the same app that does the same thing.
Emanuel:And now there's
Jason:That's Neighbors. Right?
Emanuel:I believe it's Neighbors. Yeah. And that's what it does that's what it does to you. Like, I had Ring for, like, a month, and I had to delete it because that is how I started to see the world. Like, everything outside my door is just crime and murder and and natural disasters.
Emanuel:Yeah.
Joseph:Damn.
Jason:So sorry. Before we get into this, I wanna say, like, if you Joseph and I, like, worked on a documentary, like, quite a lot, actually. Like, we're very involved. We're also both in it about Citizen for Vice back when we were at Vice. It's on Tubi.
Jason:It's free to watch, and it's called Vigilante Inc.
Joseph:I'm gonna see if I can put a link in the show notes because Tubi is
Jason:a free one. It's it's good. It came out, like, around the time we left, but it turned out really well. And it's about both Citizen, but then also this wildfire in Los Angeles, not the most recent ones, but back in 2022, I believe it was. And, basically, Citizen led a manhunt for a homeless person who they believed started the fire, and that person did not start the fire.
Jason:And they, like, did a live broadcast. They did, like, push alerts that where they sent, you know, push alerts to, like, nearly a million people in Los Angeles, I believe, who had the app, and it was insane. Like, it was really crazy thing that happened. And, you know, they got the wrong guy. I think that that's actually in the deck here.
Jason:Inside a crime watching apps manhunt for an arsonist that put a $30,000 bounty on the head of an innocent homeless man. So there you go. But that was Joseph's and my reporting. We stayed up all night writing an article, and then we turned it into a documentary. So Yeah.
Jason:Check that out.
Joseph:I just put a link in the show notes right now as we were talking. Yeah. I think you should maybe just watch it while you can because who knows with streamers, you know, and vice news or whatever. So I don't know. Just watch it while you can.
Joseph:Anyway, that so kind of brings us to this week's story, which again is about citizen using AI to generate the all of these alerts. So as I said, usually, citizen staffers are listening to police radio. They quickly transcribe it. They figure out what alerts to push, that sort of thing. Citizens now using AI to listen to that audio, to write the alerts themselves, then push them out, and there's no editing before it goes live.
Joseph:And this is according to free people familiar with Citizens' process for using AI that I spoke to. And as you might expect, it is making some mistakes. In one instance, it mistranslated motor vehicle accident to murder vehicle accident. It interpreted addresses incorrectly, like put the wrong numbers in the wrong places. It added gory details to alerts such as, you know, man shot in the face, and it's like, actually, by the editorial guidelines, you're not supposed to send a push notification saying that to millions of people or whatever.
Joseph:License plates details in unconfirmed reports. It would sometimes get confused when an officer well, because the the radios, police radios, right, there's the dispatch, and and they're talking there. And they'll say one thing, and the AI will push an alert. The dispatch will then provide an update because presumably officers are on their way or something. And the AI will think, oh, that's a new event.
Joseph:So we'll then create another one. Another example is maybe there's somebody experiencing a mental health crisis in one location. They then move a few blocks because time has passed and they're they're moving somewhere. Again, the AI may treat that as another result. So these people I spoke to, they said they would have this map of alerts and it was starting get crowded even though this is just one thing that's actually happening.
Joseph:So really bad mistakes. And I don't think this even made into the article because honestly, there was so much reporting here that I had to trim it down. But apparently, citizen users started to leave comments where they would say, was this generated by AI? Because it was that sloppy, essentially. And I think the last one, if I didn't already say it if I said it already, I apologize, but I'm just looking the list in front of me.
Joseph:It would generate a report that a homeless person is here, and, well, that's not a crime as, you know, so well, maybe it is certain places now, under the Trump administration. But that's that's really, really bad. And I think if you zoom out for a second, kind of what Emmanuel was saying with that comparison and then Sam's anxiety from downloading the app as well, it could really create a danger of like a misperception of how much crime is actually going on. If you have a ton of alerts all around this map, it's like, my god. There are a lot of people walking around with weapons.
Joseph:It's like, no. It's one guy. So I know it can make people less informed. You know? Yeah.
Joseph:I was I was just kind of blown away by how many examples there were, how many of these accounts all lined up when I spoke to these various sources. And clearly, and as one of the sources told me, speed was the name of the game with using this AI. And they laid off 13 unionized workers, which we put that right at the top of the story. It's very unfortunate, but I think that AI was the way to talk about those layoffs. Like, you could write an article that's just like, citizen laid off 13 people.
Joseph:I don't think anybody's gonna read that story, unfortunately. If you show that, hey. Citizens using AI making mistakes, and they laid off the people who were apparently doing this work. Oh, that's a lot more compelling now.
Jason:Yeah. It's interesting because I no. I feel like when we did that documentary about Ciitizen, Ciitizen was everywhere. Like, people were really it was very much in the public conscious, and a lot of people I know were using Ciitizen. And I feel like it's less popular and less relevant now, and I don't know why that is.
Jason:I I sort of I I think I think that episode with the wildfire was, like, pretty damaging for it because it it was, you know, not just us reporting it, but it ended up being front page of the New York Times. It was, like, a bunch of other places. And some of the people involved, like, left the company. People there unionized. Like, I I just feel like I don't see that much about it anymore, but maybe I'm wrong.
Joseph:I feel anecdotally the same as well. It was like, it really, really was everywhere. And now I know maybe a couple people who use it. I think beyond the fire incident being pretty damaging, I think there's just also something that was reflected in our reporting at the time was that the quality of the alerts was at the time just going down. Like, think we had one that was literally a cat is in a tree.
Joseph:It's like, dude, I don't need a push notification. I'm very, very sorry for the cat. I don't need a push notification for that.
Jason:Yeah. I think also, like, you were doing a lot of reporting at the time couple years ago about how they were trying to monetize the app and how like, there there was a period, and maybe they still do it, where they you could, like, hire someone to be on a video chat with you while you walked home, for example.
Joseph:Citizen Protect or something like that.
Jason:Yeah. Yeah. And then they also experimented with having private security in LA, which was a really crazy thing that they did.
Joseph:Which is funny now because other companies are trying that. A few of them.
Jason:Yeah. But what I'm getting at is, like, at the time, almost all of Citizen was free. Like, you could see most things on the app for free, and now a lot of it is paywalled. And so I'm wondering if maybe it's just a situation where, like, only the most scared among us are using Citizen, but then other people stopped using it because there's very little that you can access anymore without going past a paywall.
Joseph:Yeah. But I don't know. Yeah. I think I think that makes sense. The the last thing I would say beyond sort of how they're making money is that just recently, watched a very interesting video from it's like a it's like a conference from the law enforcement contractor Axon.
Joseph:You know, they make body cameras and and tasers and that draft one AI system, which does an initial draft from body cam audio using ChatGPT, all of that. They are now integrating with Citizen. I believe it's in a private beta, and you have to contact them to be part of it. This is for law enforcement because Axon also has something called FUSUS, which we've covered before, which is a sort of, they call it a single pane pane of glass interface and supposed to bring all of these tools together for law enforcement. You have your drone footage.
Joseph:You have your license plate readers. You have your cameras. Very similar to what Flock is trying to do as well. But Citizen now is integrating officially with FUSIS and Axon. So police officers, if they're looking at some sort of area and they're trying to get footage around there, they can instantly pull footage from citizens citizen users into their investigations.
Joseph:Now I'm sure that might be pretty beneficial for law enforcement. I imagine potentially the paid users of citizens would be very happy to do that as well to give that footage. I don't know if all citizen users really understand that if you film this, it's not just going to this, like, weird crime awareness social network. It's also potentially later on going into a direct tap into law enforcement, essentially. But, yeah, go read the article.
Joseph:If you want more examples of mistakes AI was making, we'll leave it there and I'll play us out. As a reminder, four zero four media is journalist founded and supported by subscribers. If you do wish to subscribe to four zero four media and directly support our work, please go to 404media.co. You'll get unlimited access to our articles and an ad free version of this podcast. You'll also get to listen to the subscribers only section where we talk about the bonus story each week.
Joseph:This podcast is made in partnership with Kaleidoscope. Another way to support us is by leaving a five star rating and review for the podcast. That stuff really does help us out. This has been Thorough for Media. We'll see you again next week.