Harvesting the dataset isn’t the problem. Using copyrighted work in a paid product is the problem. Individuals could still train their own models for personal use
I don’t think you’re familiar with the sort of resources necessary to train a useful LLM up from scratch. Individuals won’t have access to that for personal use.
I’m not familiar with the exact amount of resources, but I know it takes a lot. My point was about what specifically is in contention here.
Also, you were the one pointing out that this case could entrench “giant fucking corporations” in the space. But if they’re the only ones who can afford the resources to train them, then this case won’t have an effect on that entrenchment
They’re the ones training “base” models. There are a lot of smaller base models floating around these days with open weights that individuals can fine-tune, but they can’t start from scratch.
What legislation like this would do is essentially let the biggest players pull the ladders up behind them - they’ve got their big models trained already, but nobody else will be able to afford to follow in their footsteps. The big established players will be locked in at the top by legal fiat.
All this aside from the conceptual flaws of such legislation. You’d be effectively outlawing people from analyzing data that’s publicly available to anyone with eyes. There’s no basic difference between training an LLM off of a website and indexing it for a search engine, for example. Both of them look at public data and build up a model based on an analysis of it. Neither makes a copy of the data itself, so existing copyright laws don’t prohibit it. People arguing for outlawing LLM training are arguing to dramatically expand the concept of copyright in a dangerous new direction it’s never covered before.
What legislation like this would do is essentially let the biggest players pull the ladders up behind them
But you’re claiming that there’s already no ladder. Your previous paragraph was about how nobody but the big players can actually start from scratch.
All this aside from the conceptual flaws of such legislation. You’d be effectively outlawing people from analyzing data that’s publicly available
How? This is a copyright suit. Like I said in my last comment, the gathering of the data isn’t in contention. That’s still perfectly legal and anyone can do it. The suit is about the use of that data in a paid product.
But you’re claiming that there’s already no ladder. Your previous paragraph was about how nobody but the big players can actually start from scratch.
Adding cost only makes the threshold higher. The opposite of the way things should be going.
All this aside from the conceptual flaws of such legislation. You’d be effectively outlawing people from analyzing data that’s publicly available
How? This is a copyright suit.
Yes, and I’m saying that it shouldn’t be. Analyzing data isn’t covered by copyright, only copying data is covered by copyright. Training an AI on data isn’t copying it. Copyright should have no hold here.
Like I said in my last comment, the gathering of the data isn’t in contention. That’s still perfectly legal and anyone can do it. The suit is about the use of that data in a paid product.
That’s the opposite of what copyright is for, though. Copyright is all about who can copy the data. One could try to sue some of these training operations for having made unauthorized copies of stuff, such as the situation with BookCorpus (a collection of ebooks that many LLMs have trained on that is basically pirated). But even in that case the thing that is a copyright violation is not the training of the LLM itself, it’s the distribution of BookCorpus. And one detail of piracy that the big copyright holders don’t like to talk about is that generally speaking downloading pirated material isn’t the illegal part, it’s uploading it, so even there an LLM trainer might be able to use BookCorpus. It’s whoever it is that gave them the copy of BookCorpus that’s in trouble.
Once you have a copy of some data, even if it’s copyrighted, there’s no further restriction on what you can do with that data in the privacy of your own home. You can read it. You can mulch it up and make paper mache sculptures out of it. You can search-and-replace the main character’s name with your own, and insert paragraphs with creepy stuff. Copyright is only concerned with you distributing copies of it. LLM training is not doing that.
If you want to expand copyright in such a way that rights-holders can tell you what analysis you can and cannot subject their works to, that’s a completely new thing and it’s going down a really weird and dark path for IP.
Harvesting the dataset isn’t the problem. Using copyrighted work in a paid product is the problem. Individuals could still train their own models for personal use
I don’t think you’re familiar with the sort of resources necessary to train a useful LLM up from scratch. Individuals won’t have access to that for personal use.
I’m not familiar with the exact amount of resources, but I know it takes a lot. My point was about what specifically is in contention here.
Also, you were the one pointing out that this case could entrench “giant fucking corporations” in the space. But if they’re the only ones who can afford the resources to train them, then this case won’t have an effect on that entrenchment
No, it’ll just lock out the common man running an LLM in a homelab completely from it. No biggie I guess.
They’re the ones training “base” models. There are a lot of smaller base models floating around these days with open weights that individuals can fine-tune, but they can’t start from scratch.
What legislation like this would do is essentially let the biggest players pull the ladders up behind them - they’ve got their big models trained already, but nobody else will be able to afford to follow in their footsteps. The big established players will be locked in at the top by legal fiat.
All this aside from the conceptual flaws of such legislation. You’d be effectively outlawing people from analyzing data that’s publicly available to anyone with eyes. There’s no basic difference between training an LLM off of a website and indexing it for a search engine, for example. Both of them look at public data and build up a model based on an analysis of it. Neither makes a copy of the data itself, so existing copyright laws don’t prohibit it. People arguing for outlawing LLM training are arguing to dramatically expand the concept of copyright in a dangerous new direction it’s never covered before.
But you’re claiming that there’s already no ladder. Your previous paragraph was about how nobody but the big players can actually start from scratch.
How? This is a copyright suit. Like I said in my last comment, the gathering of the data isn’t in contention. That’s still perfectly legal and anyone can do it. The suit is about the use of that data in a paid product.
Adding cost only makes the threshold higher. The opposite of the way things should be going.
Yes, and I’m saying that it shouldn’t be. Analyzing data isn’t covered by copyright, only copying data is covered by copyright. Training an AI on data isn’t copying it. Copyright should have no hold here.
That’s the opposite of what copyright is for, though. Copyright is all about who can copy the data. One could try to sue some of these training operations for having made unauthorized copies of stuff, such as the situation with BookCorpus (a collection of ebooks that many LLMs have trained on that is basically pirated). But even in that case the thing that is a copyright violation is not the training of the LLM itself, it’s the distribution of BookCorpus. And one detail of piracy that the big copyright holders don’t like to talk about is that generally speaking downloading pirated material isn’t the illegal part, it’s uploading it, so even there an LLM trainer might be able to use BookCorpus. It’s whoever it is that gave them the copy of BookCorpus that’s in trouble.
Once you have a copy of some data, even if it’s copyrighted, there’s no further restriction on what you can do with that data in the privacy of your own home. You can read it. You can mulch it up and make paper mache sculptures out of it. You can search-and-replace the main character’s name with your own, and insert paragraphs with creepy stuff. Copyright is only concerned with you distributing copies of it. LLM training is not doing that.
If you want to expand copyright in such a way that rights-holders can tell you what analysis you can and cannot subject their works to, that’s a completely new thing and it’s going down a really weird and dark path for IP.