I’d probably say Meta trained their models using all self-hosted, public AU citizen’s data.

But it doesn’t really sound as scary as “scraped” to non-technical users.

In some legislations there are rules about scraping. And for many less technical people it sounds scary.

If it is trained on private information, then I would have issue with it.

Commercial art producers have always ripped off minor artists. They would do it by keeping it very similar to the original but just different enough to avoid being sued. Despite this, I personally know two artists who have sued major companies who ripped off their work for ads, and both won million-plus settlements. Why would we embrace this now that a computer can do it and there's a level of deniability? I don't understand how this benefits anyone.

Generally I don't think people are arguing that copyright law should be more lenient to AI than it is to humans. If your work gets ripped off (a substantially similar copy not covered by fair use) you can sue regardless of tools used in its creation.

Question would be whether machine learning, unlike human learning, should be treated as copyright infringement. There are differences and the law does not inherently need to treat them the same, but it could.

As to why it should: I think there's huge benefit across a large range of industries to web-scale pretraining and foundation models, and I'd like it to remain accessible to open-source groups or smaller companies without huge data moats. Realistically I think the alternative would likely just benefit Getty/Universal with near-identical outcomes for most actual artists.

When the very basis of copyright is for the "progress of sciences and useful arts", it seems backwards to use it in a way that would set back advances in language translation, malware/spam/DDoS filtering, defect detection, voice dictation/transcription, medical image segmentation, etc.

No, the question is whether those genAI we have around are mass copyrights violation machines or whether they "learn" and build non-violating work.

And honestly, I have seen evidence pointing both ways. But the "copyrights protection" institutions are all quickly to decide the point dismissing any evidence on philosophical basis.

I refer to the training process in question, which may or may not be be violating copyright, as "machine learning" since that's the common terminology. Question is whether that process is covered by fair use. Whether or not it actually "learn"s is not irrelevant, but I'd say more a philosophical framing than a legal one.

Yeah, you go for the red herring.

All of the worthwhile debate is about the real violations. But the public discourse is surely inundated with that exact red herring.

With humans, copyright law deals with knowing and intentional infringement more severely than accidental and unintentional infringement.

With an AI, any infringement on the part of the AI end-user is very likely going to be accidental and unintentional rather than knowing and intentional, so the legal system is going to deal with it more leniently, even if actual infringement is proven. The exception would be if you deliberately prompted it to create a modified version of a pre-existing copyrighted work.

With humans, whether infringement is knowing or not, intentional or not, can turn into a massive legal stoush. Whereas, if you say it is AI output, and it appears to actually be AI output, it is going to be much harder for the plaintiff (or prosecution) to convince the court that infringement was knowing and intentional.

That's just how the internet works. Don't put something on the internet if you don't want it to be globally distributed and copied.

> I personally know two artists who have sued major companies who ripped off their work for ads, and both won million-plus settlements.

Ultimately "AI did it" should never be allowed to be used as an excuse. If a company pays for a marketing guy who rips off someone's work and they can be sued for it, then a company that pays for an AI that rips off someone's work should still be able to be sued for it.

Until now, this has been an acceptable tradeoff because there's some friction to theft. Directly cloning the work is easy, but that also means an artist can sue or DMCA. It also means the original artist's work can go more viral, which, despite the short-term downsides, can help their popularity long term.

The important difference is that imitating an artist's style with new work used to take significant time (hours or days). With an LLM, it takes milliseconds, and that model will be able to churn out the likes of your work millions of times per day, forever. That's the difference, and why the dilemma is new.

> Ultimately "AI did it" should never be allowed to be used an as an excuse

With the exception of an LLM directly plagiarizing, the only way to prove it didn't is by not allowing it to train on something. LLMs are the sum of everything. We could say the same about humans, sure, we are a model trained on everything we've ever seen too. But humans aren't machines who can recreate stuff in the blink of an eye, with nearly perfect recall, at millions of qps.

Just because the tradeoffs of sharing on the internet used to work before AI, doesn't mean those tradeoffs continue to be workable after AI.

It's like having drones follow everyone around and publish realtime telephoto video of them because they have "no expectation of privacy" in public places.

Maybe before surveillance tech existed, there was no expectation of privacy in public places, but now that surveillance tech exists, people naturally expect that high-res video of their every move won't be collected, archived and published even if they are in public.

currently, that'd be an unrealistic expectation. I'd agree that it would be nice if that wasn't the case but laws need to catch up with technology. Right now, AI doesn't change things too much since a company who publishes something that violates copyright law is still breaking the law. It shouldn't matter if an AI was used to create the infringing copy or not.

I'm all for new laws giving extra rights to people on top of what we already have if needed, but generally copyright law is already far too oppressive so I'd need to consider a specific proposed law and its impacts.

You can find youtube videos of people outright screaming at photographers in public, insisting that no one has a right to take a picture of them without their permission while the entire time they're also standing under surveillance cameras.

When it's in their face they genuinely seem to care about privacy a lot, but they also know the phone in their pocket is so much more invasive in every way. They've been repeatedly told that they're tracked and recorded everywhere.They sign themselves up for it again and again. As long as they don't see it going on right in front of them in the most obvious way possible I guess they can lie to themselves in a way that they can't when they see a man with a camera, but even though on some level they already know that the street photographer is easily the last thing that should concern them, they still get upset to the point where they're screaming in public. I really don't understand it.

Leaving that aside, people probably react that way because the corporation just wants to gather advertising data from everyone, impersonally, while the photographer is taking a direct and specific interest.

There's nothing more personal than the collection, use, and sale of every intimate detail of your life. And corporations don't just want to gather advertising data. They want to collect as much data as they possibly can in order to use it in any and every way that might somehow benefit them and the vast majority of the time that means using it against you. It stopped being about advertising decades ago.

That data is now used to set the prices you pay, it determines what jobs you get, it influences where you are allowed to live. Companies have multiple versions of their policies and they use that data to decide which version they will apply to you, how you will be treated by them, even how long they leave you on hold when you call them. That data is used to extract as much money from you as possible. It's used to manipulate you and to lie to you more effectively. It is used against you in court rooms. It can get you questioned or arrested by police even if you've done nothing wrong. It gets bought by scammers, extremists, and activists looking for targets. Everyone who collects, sells, and buys your data is only looking to help themselves so that data only ever ends up hurting you.

More and more that data has very real world consequences on your daily offline life. You're just almost never aware of it. A company that charges you 10% more than the last person when you buy the same item isn't going to tell you that it was because of the data they have on you, you just see the higher price tag and assume it applies to everyone. The company that doesn't hire you because of something they found in the dossier they bought from a data broker isn't going to inform you that it was a social media post from 15 years ago that made them pass you over, or the fact that you buy too much alcohol, or that you have a history of depression, they'll just ghost you.

If the data companies collect only ever determined what ads you see nobody would care, but that data is increasingly impacting your life in all kinds of ways. It never goes away. You have no ability to correct errors in the record. You aren't allowed to know who has it or why. You can't control what anyone does with it.

The guy taking pictures and video on the street probably isn't looking to spend the rest of his life using that footage against you personally, but that's exactly what the companies collecting your data are going to do with it and if/when they die they'll sell that data to someone else before they go and that someone else will continue using it to try and take something from you.

AI does cause a lot of problems in terms of scale. The good news is that if AI churns out millions of copies of your copyrighted works you're entitled to compensation for each and every copy. In addition to pushing out copies of copyrighted material, AI is also capable of writing up DMCA notices and legal paperwork.

> With the exception of an LLM directly plagiarizing, the only way to prove it didn't is by not allowing it to train on something. LLMs copy everything and nothing at the same time.

An AI's output should be held to the exact same standard as anyone else's output. If it's close enough to someone else's copyrighted work to be considered infringing then the company using that AI should be liable for copyright infringement the same way they would be if AI had never been involved. AI's ability to produce a large number of infringing works very quickly might even be what causes companies to be more careful about how they use it. Breaking the law at speeds approaching the speed of light isn't a good business model.

You could make the same argument about paper. "That's just how photocopiers work! If you don't want your creations to be endlessly duplicated and sold, don't write them down!" Heck, you could make the same argument about leaving the house. "That's just how guns work! Don't go out in public if you don't want to take the risk of getting shot!"

But it's a bad argument every time. That something is technically possible doesn't make it morally right. It's true that a big point of technology is to increase an individual's power. But I'd say that increased power doesn't diminish our responsibility for our actions. It increases it.

No, the argument would be about photocopies, not paper. "That's just how photocopiers work! Don't put something into a photocopier if you don't want photocopies of it." It isn't possible for anyone to access anything on the internet without making copies of that thing. Copies are literally how the internet works.

Shooting everyone who steps outside isn't how guns work either so that also fails as an analogy.

The internet was specifically designed for the global distribution of copies. If that isn't what you want, don't publish your works there.

> That something is technically possible doesn't make it morally right.

Morality is entirely different from how the internet works, but in practice, I don't see anything immoral about making a copy of something. Morality only becomes an issue when it comes to what someone does with that copy.

"Women are oppressed in Iran. Well, that's just how Iran is. Just leave it if you don't want to be oppressed"

Oh my. Yea, and whatever is some way, is that way – "it is how it is, deal with it". It's an empty statement. The topic is an ethical and political discussion in light of current technologies. It's a question of whether it should work this way. That's how all moral questions come about – by asking if something should be the way it is. And the current state of technology brings a dilemma that hasn't existed before.

And no, the internet was not designed for that. Quite obviously. Sounds like you haven't heard of private messages.

I'm very surprised this has to be stated.

No, because Iran can stop oppressing women and still exist as a functional country. oppressing women today is "how it is". The internet on the other hand is designed to be a system for the distribution of copies. That isn't "how it is", but rather "what it is".

The internet cannot do anything except distribute copies and anything that doesn't distribute copies wouldn't be the internet.

> Sounds like you haven't heard of private messages.

Private messages are also not what is being discussed here. The comment being discussed said: "I don't agree because it creates this dilemma for creators: you need to put your work out there to get traction, but if you put your work out there and anything public is fair game, then it will be sampled by a computer and instantly recreated at scale."

"anything public". For what it's worth though, private messages are still copies.

we don’t have rules set yet, that’s why this discussion is active, ie, not just a niggle from after a couple of beers. it’s a question of respect for the author.

Is it just "someone else may be using what I published publicly to make money somehow in a way that is legal and doesn't infringe on my copyrights but I still don't like it because they aren't giving me a cut?" What would the new law look like?

Copyright is a thing we made up because of "people not liking the idea of other people doing something they don't like". The current specific boundaries are a careful compromise about exactly when we protect "using what I published publicly to make money somehow".

Those boundaries in large part exist because of responses to specific technologies. Before the technologies, people didn't care. After, people got upset and then we changed the laws, or the court rulings that provide detailed meaning to the laws shifted.

As an example, you could look at moral rights, which are another intellectual property right the laws for which came much later than copyright. Or you could look at how intellectual property law around music has shifted in response to recording, and again in response to sampling. Or you could look at copyright for text, which was invented in response to the printing press. And more specifically in response to some people using pioneering technology to profit from other people's creative work.

And we might not need any changes in the law here. The people at OpenAI and elsewhere know they're doing something that could well be illegal. They've been repeatedly sued, and they've chosen to make deals with some data-holders. They wouldn't be paying for published data at all if they knew they were in the clear, but they've chosen to cut many deals with publishers powerful enough to sue. They're hoovering up data anyhow because, like too many startups, they've decided they'll do what they want and see if they can get away with it.

Most paper doesn't come with Terms and Conditions that everything you write on it belongs to the paper company. I hate Facebook (with a fiery passion) but people gave them their data in exchange for the groundbreaking and unprecedented ability to make friends with another person (which has never been done before). It sucks, but don't use these "free" systems without understanding the sinister dynamics and incentives behind them.

People make the same arguments about the NSA. "They aren't doing anything bad with the data their collecting about every US citizen." Well, at some point they will. Stop borrowing against future freedom for a tiny bit of convenience today.

Even if FB's T&C made it clear they could do this (something I haven't seen proven), that at best means people would have a hard time suing as individuals. They can still get upset. They can still protest to the regulators and legislators whose job it is to keep these companies in line, and who create the legal context that gives a T&C document practical meaning.

And if someone takes a picture of your artwork, or takes a picture of your person, and posts that to the internet without your consent? Have you given up your rights then?

My answer: Absolutely not.

If someone took a photo of my copyrighted work, their photo becomes their copyrighted work. They also have a right to post that picture on the internet without my consent. Every single person who takes a picture of a painting in a museum and posts it to social media is not a criminal. There are legal limitations there too however and that's fine because we have an entire legal system created to deal with that which didn't go away when AI was created.

If a company uses AI to create something that under the law violates your copyright you can still sue them.

Or borrowing a thick stack of books from the library, reading them, and using that knowledge as the basis for fiction. That's a transformative work, and those are fine as well.

My take is that training AI models is a bespoke copyright situation which our laws were never designed to handle, and finding an equitable balance will take new law. But as it stands, it's both legal and encouraged for a human to access a Web site (thereby making a copy) and learn from the contents of that website.

That is, fundamentally, what happens when an LLM is trained on corpus data. The difference in scale becomes a difference in kind, but as I said, our laws at present don't really account for that, because they weren't designed to.

LLMs sometimes plagiarize, which is not ok, but most people, myself included, wouldn't consider the dilemma satisfactorily resolved if improvements in the technology meant that never happened. Outside of that, we're talking about a new kind of transformative work, and those are legal.

Correct analogy is like someone taking pictures of the paintings, going home and applying a photoshop filter, erasing the original signature and adding theirs.

The law already covers that very much so.

Not always. The copy must be easily identifiable as copy. An exact reproduction can't have the same dimensions as the original for example.

Drawing just a person or a detail of the picture, or redoing the picture in a different context or style, is encouraged.

Selling a full scale photo of the picture is forbidden. The copyright of famous art belongs to the museum.

> An exact reproduction can't have the same dimensions as the original

This is a rule, not a law, and a traditional and widespread one. Museums don't want to be involved in someone selling a forgery, so that rule is a way of making it unlikely. But the difference between "if you do this a museum will kick you out" and "this is illegal" is fairly sharp.

> The copyright of famous art belongs to the museum.

Not in a great number of cases it doesn't, most famous art is long out of copyright and belongs to the public domain. Museums will have copyright on photos of those works, and have been known to fraudulently claim that photos taken by others owe a license fee to the museum, but in the US at least this isn't true. https://www.huffpost.com/entry/museum-paintings-copyright_b_...

This is true for average people. Is it true for the wealthy? Is it true for Disney? Does our law acknowledge this truth and ensure equal justice for all?

Whatever crimes either of you can get away with using your copies is another matter entirely. Any rights you had under the legal system you had before AI haven't gone away, neither have the disadvantages you have against the wealthy.

If someone asks "how can I share my work online without it being copied?", "actually, you can't share it without people copying it into RAM" is not the answer they're looking for. That answer it too technical, too focused on minutia, and our laws recognize that.

Their problem was copyright infringement and like you said, our laws recognize that problem. We have an entire legal framework for dealing with companies that publish infringing copies of copyrighted works. None of that has changed with LLMs.

If a company publishes something that violates copyright law they can be sued for it, it shouldn't matter if an AI was involved in the creation of what was published or not.

Or we could be ethical and encourage others to be ethical.

Encouraging people to be ethical isn't actually a real way to prevent people copying photos you put up online.

You don't need to train the AI on the work for that. You don't even need to show the AI the work itself. You can just give the AI a vague description of the work and it is able to replicate something very close to it.

That's something you can try today, hand Claude or ChatGPT an image, let them describe the image, put that description into your favorite image generator. The output is a clean-room-clone of the original. It won't be a photocopy, but it will contain all the significant features that made up the original, even with surprisingly short descriptions of just a 100 words.

Won't be long and you can hand the AI a movie trailer and the AI will build you the rest of the movie from it.

> Why would we embrace this now that a computer can do it

You can't stop it long term. That old "How To Draw an Owl"-meme is reality now. You give the AI some key points and it will fill in all the rest. The issue here isn't so much copyright, but that we'll be so flooded with content that it will be impossible for anybody to stand out. We might be heading towards the death of static content and heading into a world were everything is generated on the fly.

I'm not convinced that the philosophy of copyright is a net positive for society. From a certain perspective, all art is theft, and all creativity builds upon preexisting social influences. That's how genres develop, periods, styles... and yes, blatant ripoffs and copycats too.

If the underlying goal is to be able to feed creators, maybe society needs better funding models...? The current one isn't great anyway, with 99% of artists starving and 1% of them becoming billionaires.

I'd much prefer something more like the model we have for some open-source projects, where an employer (or other sponsors) pays the living wage for the creator, but the resulting work is then reusable by all. Many works of the federal government are similarly funded, where a government employee is paid by your taxes but their resulting work automatically goes into the public domain without copyright.

I don't buy the argument that nobody would make things if they weren't copyrightable/paid directly. Wikipedia, OSM, etc. are all living proof that many people will volunteer their time to produce creative things without any hope of ever getting paid. As a frequent contributor to those and also open-source code, Creative Commons photography, etc., a large part of the joy for me is seeing how my work gets reused, transformed, and sometimes stolen by others (credit is always nice, but even when they don't mention me, at least I know the work I'm doing is useful to people).

But the difference for me is that I don't rely on those works to put food on the table. I have a day job and can afford to produce those works in my spare time.

I wish all would-be creators would have such a luxury, either via an employer relationship or perhaps art grants and the such. I wonder how other societies handle this... back in the day, I guess there were rich patrons, while some communities sponsor their artists for communal benefit. Not sure what works best, but copyright doesn't have to be the only way society could see creative outputs.

It absolutely is, just not in it's current overpowered form.

> where an employer (or other sponsors) pays the living wage for the creator, but the resulting work is then reusable by all.

Some creators want control over their own narrative, and that's entirely reasonable, at least for a limited time.

> I don't buy the argument that nobody would make things if they weren't copyrightable/paid directly.

That was never the argument as far as I'm aware. There are other concerns, like a creator losing all control of their creation before they had a chance to even finish what they wanted to do/tell.

I'm ok with that. But the philosophy of copyrights is not under debate here. All that is being debated is if it should protect small people from big corporations too.

When I put something on Wikipedia or any other commons, I don't worry about which other person, algorithm, corporation, or AI ends up reusing it.

But if my ability to eat tomorrow depended on that, then I would very much care. Hence, copyright seems an integral part of people's ability to contribute creatively.

My argument is that by detaching their income from the reusability of their work, we would be able to free more creators from that constraint. Under such a system, the little guy would never get rich off their work, but they wouldn't starve when a big corporation (or anyone else) rips them off either.

If our combined creative work until this point is what turns out to be necessary to kick-start a great shot at abundance (and if you do not believe that, if it's all for nothing, why care at all about the money wasted on models?) it might simply be our societal moral obligation to endorse it -- just as is will be the model creators moral obligation to uphold their end of this deal.

Interestingly, Andrej Karpathy recently described the data we are debating as more or less undesirable to build a better LLM and accidentally good enough to have made it work so far (https://youtu.be/hM_h0UA7upI?t=1045). We'll see about that.

While I would love Star Trek society has been going very much towards Cyberpunk aesthetic aka "the rich hold all the power".

To be precise AI models fundamentally need content to survive but they need so much content there is no price that makes sense.

Allowing AI to monetize without enriching the people who allowed it to exist isn't a good path forward.

And to be clear I do not believe there is a fundamental rift here. Shorten copyright to something reasonable like 20 years and in a decade AI will have access to all of the data it needs guilt free.

There is pretty good reason to believe that this will be a solved problem inside a decade. We're moving towards processing video and the power put behind model training keeps increasing. How much is a style worth when computers can just generate 1,000 of them at high speed? It is going to be cheap enough that legal protection is almost irrelevant; ripping off a style will probably be harder than just creating a new original one.

We can wait a bit to find out where the equilibrium is before worrying about what the law should be.

This is no different than the current day, copying already happens (as your friends have seen) AI makes it a little easier but the same legal frameworks cover this - I don’t see why AI stealing is any different than a person doing the same thing. The ability to copy with zero cost was incredibly disruptive and incredibly beneficial to society. Settled case law will catch up and hopefully arrive at the same conclusion it has for human copyright infringement (is it close enough to warrant a case)

AI that needs painstakingly curated training data isn't interesting in the same way that early lightbulbs that used precious metals and cost too much to be commercially viable aren't interesting.

In all our conversations, we have and we understand there are degrees of privacy; that which we share with family, that with friends, that with strangers.

When I post on-line, I both expect and expected that my conversations would be between me and the group of people I conversed with. I knew who was reading, and I was fine to write whatever I was writing to that group.

I may be wrong, but I think this is generally how people feel, how they act, what they expect, how they are, as humans. We think about who we are writing to. It does not come naturally to imagine that third parties are listening in, or will listen in, in the decades to come.

This brings us to now, with a third party, reaching back over ten or fifteen years, for absolutely everyone, everywhere, taking copies of everything it can get access to, for its own use, whatever that may be.

I profoundly reject Microsoft, and Google, and all entities and companies which act in such ways, these smiling evils, with their friendly icons and bright colours, happy faces and hundred page T&Cs to utterly obscure and obliterate the truth of their actions.

It's less compelling when you see the technology as noisy content generators that will flood the network with spam and devour the livelihood and opportunity to learn for low-market artists and programmers.

In the former perspective, you may look at this is "well, what's the best way we can make this happen?" while the latter sees it more like "So you insist on making this happen. Are you sure there's a suitably responsible way for you to do that?"

Secondly, Facebook is behind a login, so it's not "public" in the way HN comments are public. You'd have gained more kudos had you argued that point.

Thirdly this article I about MetaAI not OpenAI. So, no, OpenAI isn't claiming anything about your Facebook post.

I'll assume however that you digressed from the main topic, and were complaining about OpenAI scraping the web.

Here's the thing. When you publish something publically (on the internet or on paper) you can't control who reads it. You can't control what they learn from it, or how they'll use that knowledge in their own life or work.

You can of course control republishing of the original work, but that's a very narrow use case.

In school we read setwork books. We wrote essays, summaries, objections, theme analysis and so on. Some of my class went on to be writers, influenced by those works and that study.

In the same way OpenAI is reading voraciously. It is using that to assign mathematical probabilities to certain word pairings. It is studying published material in the same way I did at school, albeit with more enthusiasm, diligence and success.

In truth you don't "own the things you write" not in the conceptual sense. You cannot own a concept, argument or position. Ultimately there is nothing new under the sun (see what I did there?) and your blog post is already a rehash of that which came before.

Yes, you "own" the text, to the degree to each any text can be "owned" (which is not much.)

This isn't necessarily true for a user content host. I haven't read Facebook's TOS, but some agreements restrict what the host can do with the users' content. Usually things like save content on servers, distribute it over the web in HTML pages to other users, and make copies for backups. This might encourage users to post poetry, comics, or stories without worrying about Facebook or Twitter selling their work in anthologies and keeping all the money.

>In school we read setwork books. We wrote essays, summaries, objections, theme analysis and so on. Some of my class went on to be writers, influenced by those works and that study.

Scholarly reports are explicitly covered under a Fair Use exception.

https://www.copyright.gov/help/faq/faq-fairuse.html

But also be careful not to anthropomorphize LLMs. Just because something produces content similar to what a human would make doesn't mean it should be treated as human in the law. Or any other way.

If there were an actual AI system which was trained by continuously processing direct fetches from the Web, without storing them but directly using it when for internal state transitions, then that might make the reading analogy work. But then AI engineers couldn't do all the analysis and annotation steps that are vital to the training process.

The terms of use of most sites (including this one) include giving the site owners a license to use what you post, often in any way they see fit.

A statement that extraordinary would be interesting if it had some reasoning alongside it.

Also, Facebook posts aren't really "in the public sphere" / publicly accessible, but that's a nitpick.

Machine intelligence is anthropomorphic in utility, that is it serves as either surrogate or substitute for a human cognitive capability. This permits enumeration of AI utility categories. Broadly we can distinguish between creative, knowledgeable, analytical, judicial, predictive, and directing.

As an example use of this approach, consider the case of the AI trained on all public domain material and optionally having had training access to private matter (think Vatican archives). Such an instance should generally not be afforded creative rights, but we would be remiss to restrict its utility as a knowledge base.

The other parameters noted in terms of dual of public|private can of course have bearing on setting type specific constraints.

If AI companies, and whatever comes next, are expected to take advantage of everything shared online, regardless of copyrights, it seems reasonable that people will stop sharing most things of value.

Maybe that's unimportant and no different than being able to make physical copies, though good forgeries haven't always been so easily done and the forgery is meant to be an identical copy of the original work. It could just be me, but the idea of an attempted identical copy of a well known work feels different than a new creation being passed off as the work of a well known artist. For example, you can claim to have a really god copy of the Mona Lisa but that wouldn't be as valuable as claiming you have a previously unknown, unique work from the artist.

It seems that "AI" is biased toward (1) only bits, and (2) only bits that are published to the internet.

If I'm a Facebook user, I definitely don't see posts that I meant to share among friends as something that should be considered part of the public sphere.

Can't give informed consent if you don't know what the EULA means or what the machines can do.

I wonder if things would be different if software could be copied and then recreated by these models by the mega corps. Would there still be such push in favour of it?

Also your theoretical software cloner would also make clones of proprietary software, right? I think that would be welcomed just fine.

No idea if that matters retrospectively in legal terms—it’s seems to me that the main problem was providing service to the kids in the first place—but it’s icky either way. Then again there’s an ickiness to the entire project of pretending casual users’ arcane permissions settings from 15 years ago indicate affirmative consent today…

I don’t use Facebook. I’m not sure if they can peek into WhatsApp messages.

This is such an unconstructive attitude. This is the first time they have publicly admitted it.

Maybe it is surprising to some that they this particular research used that data, but it really shouldn't be to anyone aware of how LLM development is done.

To me it's related to the sort of person you'll see who on every startup failure says how they knew it wasn't going to work. Which again doesn't require particular smarts; most startups fail, so predicting failure doesn't take a genius. What I think is much more interesting is spotting a problem before it's obvious and naming it in advance. Or better, fixing it early on. That doesn't get you many internet points, though.

Ah, well, that makes it alright then. Move along, people, there’s nothing to see here. What’s that? What do you mean you didn’t know about the company slurping your data for their own personal gain, or knowingly poisoning you¹ and selling you defective deadly products²? Didn’t you read that one forum comment by a random person that one time? You shouldn’t be surprised. They’ve always been evil, so what can we do other than cross our arms? All hail our corporate overlords!

> I’m not sure if they can peek into WhatsApp messages.

But someone knows. And if you use WhatsApp and get screwed later, wouldn’t you rather not having your concerns dismissed?

I don’t use WhatsApp, but do have an anecdote. I have a friend who just started using Instagram and purposefully follows no one. She doesn’t have Facebook. Recently a friend recommended to her a specific product on WhatsApp and then she started getting ads for it on Instagram.

¹ https://www.sydney.edu.au/news-opinion/news/2024/05/02/how-c...

² https://www.decof.com/documents/dangerous-products.pdf

There have been numerous scandals over the years regarding their lack of care when handling personally identifiable data.

Meta is not who I look to for being extremely diligent with data.

LLM training is also apple and oranges with regards to how they handle data in the course of normal processing.

The objective answer feels like No but the subjective answer feels like Yes. Humans will never understand how an animal truely thinks but we understand how to control them.

    People just submitted it.
    I don't know why.
    They 'trust me'.
    Dumb fucks.

Things change, but this never stop being a concise summary of Meta's ethos as a company.

While I've no sympathy for mishandling private user data as Meta has of course been guilty of in the past, I think users getting 16 years of free service in exchange for their public posts being used in this fashion is not that bad of a deal.

People got 'free service' in exchange for putting ads around content not for this.

It's a terrible deal no one asked the users if they want to agree to.

It's visiting a website and saving an image and claiming the website owes you for storage.

The real question is whether the agreement that was made between them and their users during 2007-now covered uses like training AI, but most likely it was a very broad "we can use your data how we want" type of statement and that agreement wasn't fought (or fought hard enough?) back then.

Every single user of Facebook was asked and agreed to it. [0]

This was not unexpected. Anyone who read the ToS of most large tech companies in the past 10 years, and gave it more than a few seconds thought, would have realized that they were giving them access to use the data however they want - a fact many of us have been warning people about for many years.

LLMs have also been an expected evolution of technology for many years. Nobody should have been surprised about this.

[0] https://www.facebook.com/terms.php

The "facebook is not useful for me so I don't get it" needs to die seriously

Of course I'm sure Meta is also training their AI on content that they scraped from the internet/other sources without permission...

This is not a made up quote though I didn't transcribe it exactly, it was an actual message he sent during the early days of building Facebook when asked how he obtained so much personal contact information from Harvard students, so entirely relevant to this context.

https://www.theatlantic.com/magazine/archive/2024/03/faceboo...

The sum of all variants of human communication in text is pretty good for this and examples of wrong or different also matter: there was that article a few months ago about how a chemical prediction model performed better when it was also trained on invalid SMILES representations compared to totally sanitized datasets.

... or does everyone around here think anything different is happening to their posts?

It's not that there aren't normal people on there, it's that organic feel-good posts have a harder time gaining traction vs the sea of flamebait and sponsored ads and astroturfed spam. The signal to noise ratio was very, very low by the time I left. I don't know how it is these days...

It's just such a toxic, manipulated environment... I don't use social media anymore, just text some friends, and am much happier.

And there's extremely strong moderation here that does the opposite of Facebook: It optimizes against controversy and vitriol rather than encouraging it.

We end up with a bunch of nerds mostly talking shop and sometimes complaining about the job market, but that's still far less ragebaitey than most social media.

Much of what's bad about Facebook and other social media pretty much boils down to their need to show as many ads to as many eyeballs as possible.

Something doesn't add up here. Nerds talking shop and controversy and vitriol about the their technical preferences is the same thing.

Perhaps what you're saying is that controversy and vitriol is only apparent when you're an "innocent bystander" who doesn't have a passion for the subject? Which HN avoids by usually remaining focused on a fairly narrow set of subjects, to a user base generally passionate towards those subjects, so you don't notice?

That is an astute observation, if that is what you're trying to say. Indeed, if you showed your non-technical grandmother HN, she would no doubt see it the way Facebook was talked about earlier.

I don't think – I hope, at least – no one is going to start a shooting war over their framework of choice. You can't say the same about much of the content circulating around social media.

(Edit: You added more to your post after I replied. To your point of "Which HN avoids by usually remaining focused on a fairly narrow set of subjects", that's not just my observation, that's the actual guidelines: https://news.ycombinator.com/newsguidelines.html. We self-select into a narrow slice of nerdtalk or we end up getting downvoted or banned from the site. To me that is the big difference between an interest-based forum that generally stays a functional monoculture vs a general social media site that brings diverse strangers together into shouting wars about whatever the controversy du jour is.)

Is there? Perhaps the trouble here is that your examples are too far apart to recognize how they compare?

What if Facebook, instead, said "Fat workers are too bloated, you should hire skinny workers"? Or if HN said "so and so projects are all ____ and they should all be ____, and oh, your vacuum doesn't suck".

I see no practical difference. It seems the only difference is that Facebook tends to talk about people, HN about tech. But that's not a significant distinction – aside from where your interests lie. Certainly tech-minded folks often find people to be uninteresting.

The difference is that talking about a framework with passion doesn't generally end up with severe real world consequences for other people.

The difference between tech and people is that people are... people. They have lives, feelings, all those fun human things. Passionately talking about how X group of people should die (or ) is quite different than passionately talking about how Y framework is the worst thing programmers have ever concocted. Generally the latter won't end up with someone getting stabbed. (Generally...)

> What if Facebook, instead, said "Fat workers are too bloated, you should hire skinny workers"? Or if HN said "so and so projects are all ____ and they should all be ____, and oh, your vacuum doesn't suck".

I think this is a terribly unfair switcharoo to make. Let's fill in the blanks for that second quote.

> "so and so group of people are all inhuman and they should all be killed, and oh, your mom sucks"

> "so and so projects are all terrible and they should all be deleted, and oh, your vacuum doesn't suck"

A technology may not be literally stabbed, but I think it is fair to say that it very well may get stabbed metaphorically. It may even lead to the complete demise of the technology.

Passionately calling for the death of Next.js isn't really any different than calling for the death of a person, other than you may perceive it as being different if you hold people more dear than technology. But, in that case, that's just your arbitrary opinion that means nothing to anything other than you. There are no doubt some who would be more concerned about the loss of Next.js (bad example; nobody would miss Next.js, but you get the idea) than the loss of a person, which is equally valid. The true arbiter of truth, the universe, certainly doesn't have greater feelings towards one than the other.

Admittedly, when you consider people as a technology, one does start to wonder what it would take to fix the glaring bugs. But then that questions if the technology provides enough value to bother with at all.

Oh, and my vacuum does suck just fine, thank you very much.

Why do you say that? I share in the tech-minded proclivity towards not having much interest in people, so I admit to being largely out of the loop, but what is there to be worked up about where you wouldn't equally get worked up around some tech-based topic?

It just sounds boring an uninteresting to me. When I have encountered discussions about those topics, all I have is some laughter at how silly the people sound. Just like I'm sure how the "Next.js is awful" conversations sound to the metaphorical grandma.

But out in the greater society, people get all worked up about those other topics, and will go protest in the streets, blockade representatives' houses, send death threats, rage endlessly online, attempt assassinations, go on shooting sprees, etc. Those things are usually triggered by some emotional attachment to some tribal identity and anger towards some out-group.

In a monoculture like this, we're mostly peers. Yeah, we might disagree about some particular things, but there's less of an "us vs them" mindset as you see politics, geopolitics, values-based communities, etc.

Discussions of technical merit, even when heated, thankfully don't reach those emotional heights quite as often. In my experience, at least...

I understand that. Believe me, I can definitely get worked up about Next.js, if you really want to go there...! It is in no way surprising that people interested in people can end up in the same place around topics related to people.

> In my experience, at least...

But the underscore here is that your experience is based on where your interests (perhaps not the perfect word to describe this, but it's what I got) lie. If you have a greater interest in people, you're going to feel more strongly about it. I'm not bothered by it at all.

But, by the same token, what is to say that I don't feel more strongly towards technical controversy than you do?

It doesn't need to, of course. Moderation, outside of blatant spam perhaps, is rather pointless. If someone wants to exclaim "Next.js is the second coming of Jesus", I want to hear it. There is a reason they are saying it.

To not get stuck in my bias: For those interested in people, why wouldn't you want to hear "so and so group of people are all _____ and they should all be ________, and oh, your mom sucks"? It is a ridiculous statement at face value, admittedly, but it is someone's attempt at communicating. What they are actually trying to say might teach you something important. Wouldn't you want to surface what they are truly trying to get at?

I certainly would for its tech-based counterpart. Especially if it seems to contradict what I hold dear.

> But I hope your experience on HN is still a net positive.

Naturally! I'm out of here the second it is in any way not positive. No sense wasting your free time on something you don't enjoy.

  > And there's extremely strong moderation here that does the opposite of Facebook: It optimizes against controversy and vitriol rather than encouraging it.

I used to have an account for marketplace and our local neighborhood group and that's my experience.

Most users also don't post much.

Most Facebook posts (as opposed to users) are nonsense and memes.

But also, ML model training is a bit like the stock market: the noise/stupidity in individual examples points in a bunch of random directions, and so ends up cancelling out; while the signal all points in the same direction, and so ends up captured in the distilled model. (You might call this the "Anna Karenina principle of Information Distillation": all right answers are the same, while each wrong answer is wrong in a different way.)

The "initial fine-tuning of a 'raw' base model to produce a 'generalized pre-trained' base model" process, is commonly talked about in terms of "alignment" — making the model ethical, making it not swear at you, making it refuse to engage with certain content, etc. And really, that part is all optional, with there existing "non-aligned" or "orthogonalized" models that don't have these steps performed on them or have had them reversed, but which are still useful.

But a large part of this initial fine-tuning process, consists of debiasing the model's default activation, moving it away from making associations with "common misconceptions" and toward making associations with "right answers." And this process is crucial to a model being able to reason intelligently — as these common misconceptions aren't coherent in a chain of reasoning they appear in, and so lead to the chain of reasoning falling apart / being non-productive. This is, in large part, the "secret sauce" that makes a model of a given size "more intelligent" than another model of the same size.

Every base model that anyone actually cares about or uses — "aligned" or not — has had some process of de-biasing like this applied to it; or, at least, has short-cutted this process by training on a training dataset generated by or filtered by a model that has already had this de-biasing applied to it, such that the derived training dataset doesn't contain the "common misconceptions" in the first place.

And when OpenAI and Meta brag about using RLHF, a large part of what they mean, is crowdsourcing recognition of long-tail "common misconceptions" at scale, to allow a much more thorough version of this de-biasing process[2].

---

[1] Of course, if nobody in the training data ever demonstrates the "right answer" knowledge/associations, then the model will never learn that knowledge/associations. But then, given that these training datasets usually represent decent samples of the population, a "right answer" being missing entirely would likely mean that nobody on Earth knows the "right answer" — so we humans wouldn't be able to recognize the model was wrong. The stock market can be wrong too, for the same reason.

[2] Which, perhaps surprisingly, can add up to more than the sum of its parts. The more of this human-labelled "common misconception"-response RLHF data you have, the more you can derive patterns from this bias data. You can distil out negative examples that you can use to prompt a model to filter the training dataset; but more interestingly, you can distil positive examples of the sort of structured chains of reasoning that work best to inherently avoid triggering the bias. Where, if you can overlap the activation-space hyperspheres of many such inversion-of-bias examples, then you get, essentially, the hypersphere within the model's activation space that contains its instrumental rationality. You can then just bias the model toward living in that part of activation space as much as possible — and this shoots its apparent reasoning capacity way up.

I mean hop on chat GPT right now and ask it to come up with any kind of novel joke. I can almost guarantee it will either be some kind of word play or other equally low hanging kind of pun that you would find on Reddit fifty replies deep.

Where's the Mensa member only social media dating platform for a properly erudite and snobbishly arrogant LLM to train on?

- Video games (epic waste of time and moral debasement) gave us GPUs and NVIDIA

- Social media (rage bait-fueled argument machine and cat pic repository) gave us a huge corpus of text on which to train language models. Yes scientific papers and Wikipedia are necessary but probably not sufficient?

- Crypto (fraud, giant waste of resources) gave us a generation of young people who were comfortable building ludicrously oversized GPU clusters, and to some extent funded NVIDIA and TSMC R&D

all of which led to where we are with AI, which is that it is making (in my humble opinion) impressive dents in the problem of solving intelligence -- cue HN commenters telling me that AI is a giant fraud and waste of resources also :)

Of course they'd do this! How did people think feed ranking worked?

The only reason this is being reported now is because there's a chatbot and I guess that feels different to people.