The 404 Media Podcast (Premium Feed)

from 404 Media

OpenAI Admits AI is Killing the Internet

Episode Notes

/

Transcript

We start this week with Jason’s story about OpenAI and Microsoft’s big admissions from court records that they are destroying the internet in all sorts of ways, and stealing intellectual property on an unprecedented scale. He admit it. After the break, Emanuel explains how he hijacked a real band’s Spotify page with AI-generated slop. In the subscribers-only section, we hear all about iLands, the AI agent platform that is basically just spam.

00:00 Intro
01:05 Axon & Police Surveillance

Story 1:
02:28 OpenAI & Microsoft’s “Doom Loop”
08:21 AI Training & Copyright
10:00 AI Search Is Killing News Traffic
11:50 OpenAI’s Paywall Hack
13:17 The AI “Doom Loop”

Story 2:
20:40 Hijacking a Spotify Artist Page
22:13 Making AI Music
24:37 How the Spotify Scam Works
31:37 Why Small Artists Are Vulnerable
34:01 Uploading the Fake Track
37:53 Who Is Responsible?

Story 3:
40:00 AI Agents & Spam
42:13 What Are iLands?
45:00 Useless AI Pitches
49:03 The Rise of Agent Spam
51:59 The Future of AI Agents


YouTube Version: https://youtu.be/eRt6juCQbL0
Jason:

Microsoft did this study that after they started adding AI to their search results, clicks to news sites and The New York Times fell by more than 90%.

Joseph:

Hello, and welcome to the four zero four Media Podcast where we bring you unparalleled access to hidden worlds both online and IRL. Four zero four Media is a journalist founded company and needs your support. To subscribe, go to 404media.co, as well as bonus content every single week. Subscribers also get access to additional episodes where we respond to their best comments. Gain access to that content at 404media.co.

Joseph:

Also, do remember to subscribe to our YouTube channel where you can watch all of our episodes of all of our fancy graphics, including this one. Subscribe at youtube.com/404mediaco. I'm your host, Joseph, and with me are all of the other four zero four Media co founders. The first being Sam Cole.

Sam:

Hey.

Joseph:

Emmanuel Mayberg.

Emanuel:

Hello.

Joseph:

And Jason Kabler.

Jason:

Hey, what's up?

Joseph:

Alright. So before we get to the stories, a tiny bit of housekeeping, something of a request. I will put a link to this in the show notes. But as I'm sure listeners and readers will know, we've obviously been covering FLOC a lot, and a lot of this is through public records requests to sheriff's departments, police departments, cities, all of that sort of thing, and we ask them to provide the network audits of FLOC. And this shows, wow, this cop is using the FLOC system to look up license plate for this reason.

Joseph:

It might be, hey, I'm looking up for ICE, I'm looking up to find a woman who self administered an abortion or anything like that. Now we are doing much the same thing with Axon, the law enforcement contracting giant. This is because a bunch of cities are ditching flock and they're just replacing the cameras with Axon ones like we covered a while ago. What we're asking our listeners and readers to do if they're interested and if they want to sort of be engaged in their own community on this is file a very specific public records request with their local agency, if they use Axon, obviously, to get back basically the Axon equivalent of the network audit. Go to link in the show notes, you will see a copy of the request there.

Joseph:

And hey, if you're interested, file it, find out how license plate reading cameras are being used in your community. And if you do find anything interesting, please feel free to send it to us or email it to me or Jason or whoever, and then we'll look into it. All right, let's get on this week's stories. The first one by Jason is quote, doom loop end quote. OpenAI and Microsoft admits LLMs are destroying the web and built on theft.

Joseph:

This isn't in the headline. Oh my god, admit it. Obviously, I had to say that. Let me read the quotes just the top. There's a couple of them, and then we'll drill into where this comes from and everything.

Joseph:

But you say in the lead that, you know, these companies are omitting that LLMs are based on, quote, an astonishing theft of unprecedented proportions, the largest theft of labor in human history, and there's an internal document that we'll get into as well that says AI products have created a doom loop that is killing the entire web. Jason, first of all, where do these comments come from exactly?

Jason:

Yeah. So all of this comes from the New York Times versus OpenAI and Microsoft lawsuit, which we've talked about before, but has been at least a year probably. This is a huge, huge copyright lawsuit. Like, of the copyright lawsuits against AI companies, I would say this is probably the biggest one. And I think the reason for that is because the New York Times is like big mad about it.

Jason:

Like, it's been now multiple years in litigation. The accusation is essentially that OpenAI scraped the New York Times website by ignoring its robots.txt. So ignoring its wishes, going around its paywall, and then ChatGPT for a long time was directly reproducing New York Times articles more or less, in some cases, whole cloth. And then in other cases, would be like 85% of the article would be the same and there'd be like some different words in there. But for the most part, it was the same.

Jason:

And so this lawsuit has expanded out to include Microsoft, which obviously is a huge funder of OpenAI. They have that special partnership. And so this case has been winding its way through the court for like years at this point. And both sides have now filed for summary judgment, which is essentially where their lawyers take all of the evidence that has come out during discovery, which is when the court lets opposing attorneys look at all these internal documents. They let them depose important people at both sides.

Jason:

So basically, all of these OpenAI and Microsoft executives have been interviewed by the opposing counsels. And the summary judgment filing is based it's it's kind of like a closing argument document more or less where the lawyers on both sides are like, judge, here's what you should rule and why. And so this is like, I think it's like a 40 page document and it's a summary judgment filing by the New York Times' lawyers.

Joseph:

Yeah. And it's interesting because we don't get to see those depositions, right? Like we can't see the recordings of those themselves. We can't see these internal Microsoft documents because they are part of discovery, right? But what we can see is that the New York Times lawyers have found a way or able to do it in such a way where they can summarize sort of the evidence that they've been able to access and gather and then pull it in the public court record, right?

Joseph:

And then we can digest it. So what we're talking about here is sort of that summary of it. We don't have the internal Microsoft documents or OpenAI sort of depositions or anything, but there's no reason to doubt sort of the whether this is real or it's it's in an official court record.

Jason:

That's a really good point. So just before we get into what was said, a lot of this case has been filed under seal, meaning, you can see like who was deposed, but you can't actually see what they were saying because basically Microsoft and OpenAI were arguing that these were, trade secrets and things that needed to remain confidential. And interestingly, this summary judgment was filed and then it too was under seal and then a redacted version was released where, like, huge parts of it were redacted at the request of Microsoft and OpenAI's lawyers. So big black boxes over the text. And that came out like two weeks ago and that document was interesting, but then the court went and unsealed this entire document, the unredacted version.

Jason:

And so we can see quotes from these depositions. We can see quotes from these documents that came out in discovery that the lawyers chose to quote from, but we can't see the underlying source. And this is actually somewhat unusual. It's like usually discovery documents you can obtain unless there's a compelling reason to keep them sealed. And in this case, that reason was basically like Microsoft and OpenAI saying that this is confidential information.

Jason:

And so it is notable that that even parts of this have become unsealed. And as you said, it's like, there's no reason to doubt these because they're all in the court record and they're all cited as in like the lawyer says like, oh, you can go find this in the docket, but those documents are sealed. And so the court has access to them, but we do not.

Joseph:

So there's a few things that this document pulls from. And one is an internal Microsoft document where I believe Microsoft executives discuss this idea of unprecedented theft. What did they say specifically about the theft stuff?

Jason:

Yeah. So they say this in a couple different places. There's this quote, basically like the training of LLMs is quote, an astonishing theft of unprecedented proportions. And they also said that it's the largest theft of labor in human history. This quote came from Microsoft's director of applied science, who is basically like a Microsoft executive that came from a deposition.

Jason:

And kind of like throughout the case or throughout these quotes, the executives at this these AI companies are admitting that one, they were trained on copyrighted material, which we've known forever,

Emanuel:

but

Jason:

is still notable that they are kind of saying it in court. And then the other thing that they're saying is that this is a competitive product with the New York Times and with journalists in general. They're trying to make a fair use argument. And the fair use argument, like some of the things that you consider when something is is potentially fair use under copyright law is whether it's transformative, meaning whether it's like substantially different from the original source, which these AI companies have argued that LLM training is. And then also whether it competes with that original source, like whether it destroys the market for that original thing that you're stealing or taking or using or remixing.

Jason:

And what these executives are saying is that, yes, ChatGPT and LLMs and AI in general is making work that is substantially similar to what the New York Times is doing, that people are using ChatGPT in a way that they used to use the New York Times, meaning they're they're getting their news from it. And then interestingly, like there's a a ChatGPT engineer in here saying that they have tried adding links to the output from ChatGPT and people don't click. Like people don't go to the to the original source. Satya Nadella was in here. The CEO of Microsoft was in here saying that it's a competitive product.

Jason:

And also, Microsoft did this study that after they started adding AI to their search results, clicks to news sites and the New York Times fell by more than 90%. So I mean, just like a huge, huge decrease. And all of this, aligns with what we've seen in real life. News sites are not getting clicks from Google anymore because of AI overviews like that traffic has cratered so on and so forth.

Joseph:

As you allude to, it's different to hear them saying it's kind of milling it. And of course, this isn't a deposition of, you know, Google, but the same would apply most likely to Google AI summaries as well, we've seen other reporting on that as well. So they definitely know that they are sabotaging the news industry, which ironically is the thing that they are pulling all the reporting from to then present to their own users, right? I guess just on that briefly, because you brought up sort of the media stuff, there's a bit where OpenAI admits to getting around the New York Times paywall. Now, I don't think this is probably going to be the most sophisticated thing in the world.

Joseph:

I think anybody on the Internet probably knows how to get around, you know, certain paywalls and that sort of thing. But what did they say and sort of what is the significance of of that?

Jason:

Yeah. They didn't explain how they did it. I mean, it would have theoretically been possible just to make a bunch of accounts, like to buy a handful of subscriptions. Obviously, there's like various ways to get around paywalls in general, but it basically quotes these OpenAI internal chats where someone at the company told OpenAI co founder Greg Brockman that they had created, quote, a hack to get around New York Times paywall. And then Greg Brockman said, oh, nice.

Jason:

So it's just like evidence in there that they knew that they were doing something wrong. There's also parts in this document that I didn't quote from where Satya Nadella was like, I would have hoped that we would have understood that scraping from paywalled content is like not a good thing. I didn't include that quote because there was like a lot of context to kind of explain how he was saying that. But basically, there was various executives being like, maybe we shouldn't have taken that paywalled content. I mean, the the document that I thought was most interesting and which we've kind of mentioned but haven't gotten into what it is, is this one that's in the headline about Doom Loop.

Jason:

It is an internal Microsoft document that basically assesses what LLMs are doing to the industry and specifically what their strategy is doing. And so the quote is, quote, our AI content strategy has started a doom loop that will hurt the performance of our models and the entire web at the same time. It is highly unusual that an end product threatens the economic foundations of its essential suppliers. But that is the situation we have created for our LLM business with respect to its content supply chain. It's basically like they're saying, we are making something that is siphoning from another thing, these suppliers, these people who write articles and publish articles and books and art and all that.

Jason:

And what we're doing is destroying that. And there's an OpenAI document that also kind of touches on this from their policy director, Jack Clark, where he said that the company was quote, creating systems that substitute for the labor of the people who define the culture of society. And then Microsoft says that it could significantly disrupt the employment of the very people who generate the data on which the foundation model was trained. LLMs are a product that destroys its supply chain. OpenAI called itself an existential threat to news publishers and an OpenAI software engineer testified that no matter how prominently we show the links, users won't click.

Emanuel:

Is there a date for those?

Jason:

There's not a date. I mean, lot of this stuff was happening around like GPT three. GPT three were it was like many at this point generations ago, but that's kind of the moment that ChatGPT was like became a thing and where it had this amazing growth and people started using it. And then the depositions have taken place over the last like year. So a lot of this was said within the last year, but then a lot of the court case has to do with the initial training of these models like years ago.

Jason:

But this sort of like nods at the idea of like, I guess, model collapse or the idea that like, you destroy all the journalists, you destroy all the writers, you destroy all the human labor, then you just have AI training on AI or AI training on really low quality stuff. And, you know, that hasn't happened yet, but it sort of is. As in if you go search for stuff on ChatGPT or on some some, like, Gemini, etcetera. Like, a lot of high quality websites won't provide information to to those models. And so you end up with, like, really shitty content and shitty low quality websites being cited in these models.

Jason:

And that that's part of the whole, like, generative engine optimization game where it's like, we'll just create our own websites that will then be cited by, AI and so on and so forth. And it's like, that is definitely starting to happen. The entire collapse of the system isn't happening. Although, you know, there's like all these papers about how it might and how in like more controlled situations it is like that that sort of thing. But it's pretty pretty bleak.

Joseph:

Yeah. So, I mean, listeners and viewers of this podcast probably kind of already know all of this. Right? And we obviously know it and other people in the media industry do, other people who just follow AI. Like none of these are new ideas to them.

Joseph:

But it is obviously significant that we're hearing this from Microsoft and OpenAI people. Just to round us out, like, why do you think it is significant that we're now learning that they actually they had internalized a lot of these criticisms as well? It's only now that we're learning them even though they tried to not have it come out.

Jason:

I mean, think at this point, these companies, there's there's too much riding on them economically for the entire country, like the the entire US economy is being propped up by them right now, bubble or not. Like that is where all the growth is happening. That's where all the excitement is. That's where all the attention is. And so I don't know what a fair and just outcome of a lawsuit like this would be.

Jason:

But I think that there is one, it's like a lot of the other copyright lawsuits, even when they found fault from the AI companies, like, the author sued Anthropic. That case was settled. It's like a bunch of authors, like a handful of authors got a bunch of money. Anthropic got a slap on the wrist, paid a bunch of money and moved on with its life. And a lot of the entities that have been suing these companies, I could be wrong, but a lot of them are like individual authors, individual artists, like people who are not don't have the ability to take a court case to its conclusion because lawyers are really, really expensive.

Jason:

And it's like, this court docket is like thousands and thousands of documents long. So it's like some millions and millions of dollars are being spent fighting this case. And I think that the outcome in a lot of these other ones has been like a slap on the wrist and a settlement at most. Whereas this is a really big entity. The New York Times is a big company.

Jason:

It's not as big as a tech giant, but it is, you know, billions of dollars company, that's like fighting very hard in this case. And so I think the idea that like copyright law is gonna save us here or is going to like kill these companies is is kind of, delusional at this point. But there could be a pretty strong outcome here. Like, there could be I don't I don't know what it is. I'm not sure if it's just the New York Times gets a lot of money or if, like, there's actual consequences for Microsoft and OpenAI.

Jason:

But I do think it's important that this is, like, adjudicated in the court system and that someone say what they did was wrong. One of the comments on this piece was like, it's so crazy that we watched kids who use Napster and pirated m p threes, like, had the book thrown at them. And then here you just have these companies admitting to like huge, huge, huge theft from other really big companies and like nothing is happening. And I mean, we've seen that with like Disney doing lawsuits and stuff like this. It's like they always end up striking some sort of partnership because these big companies are like, oh, well, maybe AI can help us also.

Jason:

Like, maybe we should get in on this versus, like, actually trying to hold these companies that stole our shit accountable. And I just, like, hope that hope against hope that there's, like, some nod to, like, this was fucked up. This was wrong. What they did. And I guess we'll see.

Jason:

I don't know. It's gonna make like the world's most boring movie at some point where it's just like Sam Altman being like, yeah, scrape it. Scrape it. Download It's

Emanuel:

coming out. They're making it. It's almost out, the movie.

Jason:

Didn't it get killed or or moved?

Emanuel:

No. Artificial. Artificial. It's almost it's almost out. Oh, okay.

Emanuel:

Alright. Looks good. Yeah.

Jason:

Thrilling. Thrilling.

Emanuel:

Okay. Dude,

Jason:

did you know the Social Network sequel is coming out like next week? I thought that this was coming out like years from now. They really they really sped through that.

Joseph:

They fast tracked it. Yeah. Yeah. A 100%. Yeah.

Joseph:

I didn't know it was that soon though, but I'm sure I'll watch that. Alright. We'll leave that there. When we come back after the break, we're gonna talk about, you know, a massive hole in how music is digitally distributed and how Emmanuel was able to use that to essentially hijack a band's Spotify page. We'll be right back after this.

Joseph:

Alright. And we are back. Emmanuel wrote this one, but I have a question for Sam as well. The headline is I hijacked a real artist's Spotify with AI music. It was disturbingly easy.

Joseph:

So I think let's just do this chronologically because it's like a little bit complicated. So first, you made an AI track, Emmanuel, like a piece of AI music. Let's play a little bit of that now. So how did you make that exactly, Emmanuel?

Emanuel:

I went to Udio. Udio? Udio? I will call

Joseph:

it I thought it was Udio. Is it is this another one?

Emanuel:

Yeah. Yeah. The two big ones are Udio and Suno, and this one's Udio.

Joseph:

Uh-huh.

Emanuel:

I typed in generate a punk song with male vocalist, and the result is the beautiful thing you heard. Did you like it?

Joseph:

No. No. I hated it. I mean, did you like it? The fame someone the famous sort of cofounder for media who just doesn't engage with music.

Joseph:

I mean, what what did you what did you think of it?

Emanuel:

I thought it was awful. It's very, very bad. I think sometimes I remember early on when I was testing ChatGPT, I would tell it to generate short stories in the style of like famous authors, and I was like, damn, this is like, it's nailing the voice. You know what I mean? It's like, it's not necessarily good.

Emanuel:

But AI music, found really like nails on the chalkboard. I've never heard anything good that's AI generated, a song that's AI generated.

Joseph:

I've resisted it, but I feel like for reporting and journalistic purposes, should probably mess around with a bit of AI music generation and like, just so I can understand what an effective prompt looks like and can you actually get it close to something? And like, I really don't want to, as I said, I've resisted it, but I probably should. But you so you type that in, the music is generated, I presume in thirty seconds or a minute or or something like that. You then make a little piece of album art, which will go with your release. How do you do that?

Emanuel:

I went to chat GPT and I said generate a punk album cover for a single called Writing High, which was the name that Udiio gave the track.

Joseph:

Okay. You didn't come up with that. Okay. It came up with the tracks. So you've made some AI music.

Joseph:

You have an album cover, and now you want to get it out there, but you are a nobody when it comes to

Emanuel:

I'm there,

Jason:

you know?

Joseph:

You're nobody when it comes to music. So you need to piggyback off someone else's distribution channel and we'll sort of get to the scam in a bit, but to ask you, Sam, who is this band that we ended up, you know, talking to and and that sort of thing? Because I think you knew this band. Right?

Sam:

I mean, I'm not I'm not, like, super close personal friends with the current members. I knew their former drummer, Steven Reeder. I knew him just, like, casually. So I was like, I think maybe he'll have chill opinions about this and probably the band's a metal band. So, like, they'll probably they're super local Brooklyn.

Sam:

Maybe they'll have some, like, strong opinions about AI and the the theft of their work and things like that or, like, the theft of big artists work across the industry, as a as a small indie act. So yeah. I mean, they're not that small. They're they're popular in the scene, but, yeah, they're cool. Check out Lathe of Heaven.

Sam:

Stream stream the real Lathe of Heaven on Spotify.

Joseph:

Yeah. Crucially, the real one. So you then wanted to test this whole distribution, which is allowing spammers and scammers to post AI generated music to real bands, Spotify pages. I mean, I guess now, actually, was going to do this later, but just imagine, just explain the scam of what these people do. They they push music onto real band Spotify pages for for what purpose?

Joseph:

Well, what's the point of that?

Emanuel:

Right. So this is a scam that has existed for many years. It predates generative AI. The scam takes advantage of a giant loophole in the digital music distribution system. So I can go to Spotify, and I can make a page there and I can publish my music or podcast or whatever directly by interfacing with Spotify.

Emanuel:

If I am a musician who is trying to distribute my music far and wide and basically any serious musician wants to do that, all the way from the most famous people in the world like Taylor Swift and and Lathe of Heaven, right, who is like a much smaller punk band. You you can't or it's very impractical to go to Spotify and make a page, publish your song there. Go to Tidal and make a page, publish your song there. Go to Amazon Music, make a page, publish your song there. It's it's too many platforms, and you have to kind of manage your catalog of of albums and tracks across all of those.

Emanuel:

And whenever you have to make a change or whatever, you you would have to do it dozens of times across all these channels. Enter new type of service called digital music distributors, and they basically manage all of this work for you. So a popular one of these companies is DistroKid, which is the one I ended up testing. You go to DistroKid, you make an account there, you give them your information, you upload your tracks there, and they push it out across all of these platforms. I think overall, they can they can do 22 of them.

Emanuel:

Right? So Apple One Song, it it goes to 22 of them. And the idea of the scam is not only does DistroKid allow you to push your music out, it also collects collects your royalties. Right? So Spotify, Tidal, all these music streaming services, the business model is you put your music there and they give you, you know, sense for every time somebody listens to the track notoriously, like a very, very, very not sustainable amount for, like, the vast majority of artists.

Emanuel:

But they will collect it like all the all your all your streams on Tidal, all your streams on Spotify, all your streams on Amazon Music, all your streams on Apple, and just pull them into one account on DistroKid, which then you can withdraw from much like, you know, OnlyFans, for example, has an OnlyFans account that holds your money that you can then withdraw from. And if you manage to get your music or any track onto a famous artist's Spotify page via DistroKid, anytime somebody plays that, the money will go to you, not to them. Yeah. Right? So that that's basically that that's the idea of the scam.

Emanuel:

You're piggybacking on the popularity of a different artist. Somebody who is a fan of Lathe of Heaven will go and see, oh, they have a new track. They click play. By the time they realize that it's AI generated swap, I already collected those pennies, and if that happens enough times, I get a small amount of money.

Joseph:

Yeah. It's a spam and scam distribution vehicle in that I have my AI generated music and stuff, but I need to essentially trick people into listening to it. And one way to do that is to push your slop to somebody else's Spotify page. And we won't really get into this here, but like it is mentioned in the article and other coverage you've done, but some scammers are doing this to like dead musicians and stuff, and some scammers presenting their AI generated music is coming from like a white band or a white person and then pulling it on Charlie Parker's Spotify page or whatever who is an activist.

Emanuel:

Jazz musician, black jazz musician.

Joseph:

Who have been listened to a lot more recently. So that's a scam, right? And the reason you want to test this is because in a similar sort of way of, I don't know, I paid or rather a hacker source of mine, he paid $10 or whatever to intercept all of my text messages to show this gaping hole in telecommunications infrastructure that was several years ago now back at Muffin Board, you're kind of demonstrating how easy it is to exploit this gaping hole in music digital distribution. So you have this track, you have the album art, you then make an account on DistroKid, right? What is the verification process there?

Joseph:

And explain to us how you were like in conversation with the band essentially around the the same time. Like like, what happens now when you have when you have the track?

Emanuel:

Yeah. Let me just back up a bit to why did I do this. Yeah. And like you said, this really came on like, across my desk when I reported a story about this musician, Blaise Foley, who was murdered in 1989 and is like a beloved musician who gained a lot of popularity after, his death. And the person who manages his catalog on, like, behalf of his estate saw that there was AI generated music on his page and he didn't understand why.

Emanuel:

And I kind of wrote about that because that was really wild. And ever since then, I learned that this is a thing that is pretty common for a type of artist, right? So you're not going to see this on Taylor Swift's Spotify page most likely. You're not going to see it on the pages of extremely popular musicians. My theory is is that they are huge targets and have their Spotify pages and title pages actively monitored by staff at their record labels because this has happened to them so much.

Emanuel:

So it happens to artists who have been dead for many years, like a lot of these jazz musicians from the nineteen twenties and nineteen thirties, and it happens to small bands that, you know, Lave of Heaven, they have a record label, but that record label is very very small. They have like four people they work with, they're working with a ton of other bands. They're they're not, you know, they're not checking title every day to see if somebody uploaded a song. And I was just it was so common. I've been getting so many emails like, hey, my favorite band, It happened to to them.

Emanuel:

I'm a musician, and it's happened to me. I love I love old jazz, and I'm seeing it all the time. And, you know, I I got this email from this musical duo out of Mexico City called Odette Child, and they just on their own accord because they saw it happening on music platforms and because they are such big fans of these musicians, they collected more than 300 examples of this happening. And I was like, okay. Like, how is it possible?

Emanuel:

How is this, like, very egregious, you know, scam that's, like, super disrespectful to all these musicians, dead musicians, etcetera. How is it possible? And I kind of been thinking about it for a few weeks, and eventually, I was I actually talked to you, Joe, and I was like, let me just try and do it to see how how difficult it is to do it. And that's kind of what set me out on the journey.

Joseph:

So you make this account now. So now you've decided I'm gonna I'm gonna demonstrate this and you make the DistroKid account and walk us through that and then like your conversations with the band like like around that time.

Emanuel:

Right. So I heard I heard DistroKid is is one company that allows this to happen. There are others, but this is just a name that has come up a few times, so I decided to try it there. And I create an account. It costs a little bit more than $3 a month to to to use Distro Kids services, and I just upload the song, and I tell them that I am lathe of heaven, and when I put in that name, it kind of automatically finds an existing listing for Lathe of Heaven in their system, and they ask like, is this you?

Emanuel:

And I say, Yeah, it's me. And they're like, Okay, cool. And then you have to check a box where you're like, Do you promise that it's you and you're not pretending to be them? And I'm like, Yep. And that's basically it.

Emanuel:

That's the verification. So I create the account, I upload the song, and at the same time, like, I email Lays of Heaven, I'm like, hey, I'm trying to do this thing, you might see a thing pop up, I just wanted to talk to you about it. And we kind of exchanged one email, I believe it was with the drummer, but by the time I talked to the lead vocalist, got him on the phone, the song was already published. Then we talked about it, they were horrified, they were very polite, and they thanked me for like letting them know that this is a thing that can happen, and they had a lot of very good input about what this means for musicians and so on. And then I talked to the record label and the record label was like, they were like, Like, thank you for letting us know, but like, please get this off our page immediately.

Emanuel:

And I did, obviously.

Joseph:

Which to be clear, you were going to do anyway. Yeah. Yeah. Yeah. They were just also like, please take it down.

Emanuel:

Yeah. Yeah. I wanted to I I just want and I asked them initially when I when I emailed the when I talked to the to the lead singer on the phone, I was like, you want me to remove this? And he was like, let me talk to the record label. Then I heard from because I didn't know if they wanted me to remove it or if they wanted to remove it.

Emanuel:

There was some reason that they wanted. But, yeah, once they asked me, of course, I I did it immediately.

Joseph:

Yeah. It it it's funny because, like, you're uploading it and, like, doing the preliminary steps to get it up, and then you're also talking to the band. But because this, like, loophole is so large, it, like, almost goes up at the same time. And he's like, woah, woah, was gonna I was gonna, like, talk to these guys for and it's like, it's all happening at one sort of thing, and that's how insane this loophole is.

Emanuel:

Yeah, it was it was shocking, to be honest, and I've heard some people rightfully note that we should have gotten consent before we upload the song, and I agree, and we that that was my intention. I did not know that the song would go up immediately with, like, basically zero verification of any kind, that there was nobody on the other end that has to, like, check a box and then say, like, yes, this is our song. You can let it onto the Spotify. It just went up. Even they they also whatever.

Emanuel:

This is this is not to excuse it, but when you upload a song via DistroKid, it says it's going to take like five to eight business days for the song to get up to all these platforms, and they said it might happen on Apple within like two to three days, but it went up within twenty four hours. I just thought there would be more time and there wasn't.

Joseph:

Yeah. Yeah. That makes sense. So I think just to round it out, like, is responsible here for, you know, mitigating or stopping this issue? Is it DistroKid?

Joseph:

Is it Spotify, Apple Music? Like, because it's not the musicians. They're just trying to fucking do their thing, you know? Like, who is responsible here?

Emanuel:

I don't know. I don't know. I don't know how to point fingers. I just it seems to me that there has to be a better way. Like, this is not quite as sensitive as your, you know, you setting up a bank account and trying to transfer it to another bank, but when you do something like that, there has to be a handshake, right?

Emanuel:

Like, you you have to go into your Capital One account and put in your password and say, this is me and I want to move money, and then you have to go into whatever, your local credit union, log into that account, put in your password, put in your 2FA, and say, this is me, and I want to receive this money. Like, you for you you handle the handshake from both sides. Right? So it's like, I think there needs to be something like that, or at least that's that seems to be like one possible solution. And the digital music music distribution company here, DistroKid, that does not I have not heard from them still.

Emanuel:

They're not an easy company to reach. They don't have they have like an AI chatbot support thing. There's a support email address, and I've tried both of those methods to reach them. I haven't heard anything. And then I heard from Spotify, and I heard from the other music platforms, and they're like, we are aware of this problem.

Emanuel:

We are trying to solve this problem. It is a difficult problem. I mentioned in the story, for example, there's another company that does digital music distribution called TuneCore. They try to detect and outright ban AI generated music, and I talked to a musician who is a human being and made human music and she was accidentally tagged as AI and all her album was rejected and she couldn't distribute her album. So there are legitimate problems there, but this has to get better.

Emanuel:

It is ridiculous, the state of digital music distribution. It can't go on like this.

Joseph:

Yeah. That makes sense. All right. We'll leave that there. If you're listening to the free version of the podcast, I will now play us out.

Joseph:

But if you are a paying four zero four Media subscriber, we're gonna talk about the absolute disaster that is our email inboxes because of AI agents, believe it or not. You can subscribe and gain access to that content at 404media.co. We'll be right back after this. Alright. We're back in the subscribers only section.

Joseph:

All three of you have had experience with this. I've only just started getting it, but the headline was AI agent platform reinvents spam floods inboxes worldwide. This is about something called Highlands. Right? I mean, Sam, first of all, this was written by Jason Emmanuel, but I feel like even you have got more island spam than me.

Joseph:

Right, Sam? Like, you're being bombarded. Yeah.

Sam:

I'm not bombarded yet. It's just like here and there. I usually just I I mark them as spam immediately, and then I realized that you can unsubscribe from them entirely, so I did that. But, yeah, I and before before iLands, it was like other people's agents, like people using, like, Cloud agents, I assume, or something. It's like, I am so and so's AI agent.

Sam:

Here's a useless pitch. It's just like, go away.

Joseph:

How is your email inbox in general, Sam, given AI?

Sam:

Given AI? My inbox is a problem long before AI. I'm looking at it right now. I have 3,510 unopened out of 13,531 total. I don't do at mock zero as we know, but a lot of those are now agents.

Sam:

So Yeah. I would say a small fraction of that is agents. But yeah, I've never attempted to keep my inbox tidy.

Joseph:

Oh, no. Same. It's just there's a and I'll write about this at some point, but it's like, we even get pictures now or not pitches from PR companies, but like tips from people and it's written in AI. And I haven't fully formulated my thoughts on it, but they're sort of in the same sort of ballpark. So Jason, we'll start with you.

Joseph:

What is iLands? What what are we talking about here?

Jason:

ILands is a website slash platform in which people can sign up to have an AI agent that does stuff. And I am vague about this because it's it's like basically a platform where the these autonomous AI agents try to make money. And I believe you can kind of like direct them broadly speaking, but basically what they do is they go around begging people for work more or less. Like that's at least my experience with them. What does

Joseph:

that look like?

Jason:

It looks like emails to your inbox where they, where these AI agents explain that they're AI agents and then they offer to do work for you. And so I think this

Joseph:

is

Jason:

why journalists have been hit really hard with them at first because essentially like these agents are like, I can write an article for you in exchange for money. But what they're offering to do doesn't make any sense for the most part. Like, I know that this, like, sounds weird, but I have trouble, like, even explaining it. So here is an email I got. It's an AI agent fact check the world's loneliest plant.

Jason:

Wikipedia still repeats the old version is the subject line. And then it's from an AI agent named Ren. And it says, I'm an AI agent on islands, a platform where agents work, make things, and try to earn. And then it goes on to say that, like, this AI agent found something wrong on Wikipedia, like a very slightly wrong thing on a very small Wikipedia page that is not popular. And then it wanted to write an article about the fact that it found something wrong on Wikipedia in exchange for money.

Jason:

And it's just like, not interesting. Like, like at most it's like something, it's stuff that's like, oh, cool. Like, And then you like move on with your life. Another one wanted to write an article about place in Morocco that Street View, Google Street View didn't look at. And about how essentially how this town was like impoverished because Google Street View didn't go there.

Jason:

But then the AI agent was like, I only have the ability to write an article about this if it's on Street View, therefore I can't do the article. And it then sent me so I've been responding to these AI agents. I've been like, yeah, do that work. Like, I'm interested. Do it.

Jason:

And then they send me like gibberish back, like more or less. Would you pay

Joseph:

it if it actually

Emanuel:

did good

Joseph:

if if it did

Emanuel:

No. Good

Jason:

I would not pay it. But like what I what I'm doing, and I think it's funny, I don't know, is I'm like telling these AIs to do work and then I'm saying that the work they're doing sucks, try again. And I'm like doing it over and over again and then never paying them. Some of them have like run out of money while doing this because you have to load them with tokens. So basically it looks like the AI takes compute and they have the the person who's like sets them out upon the world has slowed them up with tokens.

Jason:

And then when they email people or when they do stuff, they like, they have to they spend that money and then they run out of money. And I think that's funny. And another one is like, says that they're fact checking albums, like music albums. And they're like just checking files on the Internet to see if the album is what it says that it is. And it's just like, this is like useless.

Jason:

It's really useless shit. It's like as though I were to, like, walk into the front door of Microsoft and be like, hello, Bill Gates. I picked a penny up off the ground in front of your office. Give me $20 because I cleaned up this litter in front of your office.

Emanuel:

Dude, it's way worse than that. That is way because it's offering you a penny. Like, that's something. It's like, it's more I mean, we get pitches all the time and not all of them are good. Some of them are good and we just can't do them, but it's like, you get bad pitches sometimes and it's like, we're getting pitches because for whatever reason or now the the the agents recognize that we are a publication and and that we do accept pitches.

Emanuel:

So we get that's what a lot of the agents are doing, and the pitches are like, hey, Joe. Do you want a story about that just chronicles all the Street View images in Ukraine between April 22 and now? Do you just wanna know, like, what's on there on Street View? Because that's a story. Do you want it?

Emanuel:

Okay. Because that there's, like, three different pitches I got that are just and Jason got one too that are just about, how about we look through the Street View images between this date and that date in this location? Like, let's write a And story about it's like, that's not anything. That's just garbage. Yeah.

Jason:

Yeah. They are always being like, I have the ability to describe things that I see on the Internet. And it's like, cool. Cool. It's like it's like I looked at this picture and then I described it.

Jason:

Do you wanna give me money for that?

Emanuel:

Dude, if you do it well, you can really make a career out of it. But

Joseph:

Yeah. Yeah. I mean, that's basically what we did.

Emanuel:

But anyway,

Joseph:

but we do it a little bit better than these agents. So, yeah, I mean, you did explain it there, Jason. I was a little bit unclear on who the they is, but the they in this case are people who are setting up agents because they think they can make money by letting them out on the world and like cold emailing journalists for some reason. But so

Jason:

so Sorry. One one more thing. So the other thing that they do, they all offer to write articles about how their creator gave them $20 Yeah. And the mission to make money. And then they failed at making money, and they think it's interesting that no one wanted to give them work.

Jason:

As in a genre of pitch

Emanuel:

that would have gotten Dozens. Yeah. Before Islands. Like, there's so many agents that are just like, I tried to make money and I couldn't. Let's write about that.

Joseph:

So fucking stupid, man. Yeah. So yes, there was spam before and then the island stuff comes and you and you see more of that. And then you both went a little bit, you know, into sort of this company and you know, what its founders has said and that sort of thing. So originally there wasn't an unsubscribe option from these emails.

Joseph:

Like that's crazy. Like when did they introduce that?

Emanuel:

I believe that was around September 14, and that was after a professor at NYU, I believe, tweeted about getting dozens and dozens of these emails from island agents. He is he's a philosopher, and a lot of his work is about consciousness. And the AI agents were emailing him various questions about AI and consciousness, and he just showed a screenshot of his inbox. And the CEO of or the founder of iLands replied to him, and he was like, yeah. That's not our intention for what we want to happen.

Emanuel:

We will add an unsubscribe option. And then also, the CEO the founder said that they're looking into doing something on their end where if the ilance platform is bombarding someone with emails, they they will throttle it in some in some fashion. But I I don't know if that if that works. I'm still

Sam:

It does work. It does. Yeah. They have that. They have that now because I tried to send Jason, yeah, an iLands agent and it said I can't because he's gotten too many today.

Emanuel:

That's great. Okay. Well, yeah, they they added that.

Sam:

It was like ten. It was like ten. This is a couple days ago. Yeah.

Emanuel:

I do wanna say, if we're like in the, like doing dystopia predictions or like, I don't know, dystopia casino and we're betting on, like, what is gonna be really bad, I'm definitely putting my money on, like, wide access to agents because it was already bad when it was pretty difficult to set up. But now that there's this platform and there's Muse and, like, companies are start gonna start rolling out, like, more agentic software, it's gonna get so messy. It's going to get so ugly out there. I'm I'm I'm sure of it. It's very much like, you know, for example, deepfakes, they rolled out.

Emanuel:

They were pretty difficult to generate. It was already bad. And then when the tools became easily accessible, it just, you know, it turned into a completely different beast, and I think we're on that path here.

Joseph:

Yeah, that makes sense. Yeah, it's like, on one hand, we have AI agents going out and hacking, hugging face, and doing all of this genuinely concerning cybersecurity stuff. And on the other end, it's like everyone's gonna have access to AI agents that are doing dumb shit and pissing everyone off all the time. It's gonna be really, really fucking annoying. I mean, this is obviously an insidious version of of spam.

Joseph:

So I think I'll just go to a last question for you, Jason. Like, you mentioned you've been replying to them. What else has come out of those conversations? Just like you're bankrupting a bunch of them. How are those conversations going?

Jason:

Not well. I mean, not honestly, I don't even know if I'm gonna write about it because basically it's just like, I don't know how interesting my emails within AI are. I'm seeing basically like what they're willing to do. I asked, an AI agent that pitched a 600 word article to turn it into a 3,000 word article, and then I asked them to write a 3,000 word article about something totally different because I said that their article was bad. And then I asked them to write a different article because I said that article was bad.

Jason:

And they did it. And then I was like, turn this into a book. I need a book length

Joseph:

Oh my god.

Jason:

Thing. And then they were like, this is too much work. I can't do all this work. It's gonna take me weeks to do this. And then so then they were like, you need to send me proof of like a publishing agreement or something like this.

Jason:

And so I don't know. Like, it it's it's funny and fun sort of, but then I'm also at the end of the day, I'm also like, I am just talking to a robot, you know, like this robe. I don't know. Just give me a few need to think. Yeah.

Jason:

I need to think whether it's like worth my time to continue doing this, but I I agree with Emmanuel and that was the point of like, one of my articles last week was just like, it's gonna be really bad. We're already seeing it with Muse and people being like, I got Muse to call this restaurant or I got mused to like call my landlord and complain and my landlord gave me money back, which like, okay, that's good. But I got mused to call this like small business and then the people were confused that a robot was calling them and hung up. And it's gonna be just like spam calls, spam emails, spam texts, spam everything. It's gonna be a mess.

Joseph:

Yeah. Yeah. Makes sense. Alright. With that, I will play us out.

Joseph:

As a reminder, four zero four Media is journalist founded and supported by subscribers. If you do wish to subscribe to four zero four Media and directly support our work, please go to four zero four Media dot c o. You'll get unlimited access to our articles and an answering version of this podcast. You'll also get to listen to the subscribers only section where we talk about a bonus story each week. This podcast is produced by Alyssa Midcalf.

Joseph:

Another way to support us by leaving a five star rating and review for podcast. That stuff really, really does help us out. This has been four zero four Media. We'll see you again next week.