Of course it does this. The training data is full of examples that associate e.g. "New Yorker-style cartoon" with Loper's signature in the corner, because Loper's signature is in the corner of a lot of them. There's nothing to make it treat the signature as anything special by default; that would have to be trained in explicitly.
One would hope that the person prompting ChatGPT would notice this sort of thing and do something about it before sharing it publicly, out of a genuine desire not to cause confusion etc. But I guess that's way more personal responsibility than we can expect average people to take on nowadays.
>she wrote she had simply asked ChatGPT to make “a New Yorker-style cartoon.”
A "style" can't be copyrighted, at least in US law. They might have a stronger case of trademark/likeness infringement, but the fact that the person knew it was AI generated would make that difficult. Of course they knew it wasn't made by Brendan Loper. Of course, if they then published it, the other people viewing it might not know this, but who published it?
There's a difference between reproducing a signature in an encyclopedia or in some way that makes it clear that you are recording the thing as it is.
Putting a signature on a work is forgery and in most jurisdictions charged as fraud.
If you produce an artwork in the style of someone and then clone the signature of someone who produces art in that style, there is a reasonable case for fraud.
Why would ChatGPT be sued exactly? They didn’t publish the picture, they did what the user asked. The user is responsible because they directed the creation and publishing. The user is the entity who should be sued. Or dmca’d. Or whatever.
Think if the user commissioned the art from an outsourced creative shop nobody has heard of. Then they published it. They wouldn’t go after the creative shop, they would go after the publisher.
(I am just addressing publishing here, training on the artist’s works is a different, well discussed issue)
If you think Aaron Schwartz would not have fought for exactly the freedom of information that would enable AI training, you misunderstand Schwartz and the nature of information freedom.
I continue to find that people strongly advocate for justice on exactly opposing sides, depending on who they have been told to think the “bad guy” is.
Voters might feel differently once they wake up and realize that it's 5% of GDP because of the anticipation that so many voters will be thrown out of work.
> once they wake up and realize that it's 5% of GDP because of the anticipation that so many voters will be thrown out of work
But it's not. The only layer anyone has a handle on is consumers and companies paying AI ridiculous sums for AI. Some of those users are probably justifying it with labour replacement. But a lot may not be. So far, we haven't seen the employment effect outside recent college graduates at enterprise companies.
> as implemented in the United States it's a malignant tumor, a theft of labor by capital, and it must be (minimally) adddressed with confiscatory taxes applied to all those involved with it's creation and operation
It's also a godsend of economic growth. Growth other countries who are trying to balance their books would kill for. Without AI, we'd be in a failure state. Maybe we are, if this is all a bubble. But as it stands, there is paper wealth that can and has–limitedly–been taxed. That gives everyone options.
> If a single AI billionaire exists in the year 2030, then the US is a failed state
This is silly and projecting a narrow view of the world onto a larger voting population. Voters don't care so much that there are billionaires as that living standards haven't kept up with the rate at which they're being minted. Double tax brackets, add more on top, raise the minimum wage, raise Social Security taxes and benefits, expand Medicare, beef up antitrust, establish a progressive property tax on wealth that starts at 1,000x the median American's wage (about $65mm) and billionaires are fine.
> All of these people need to go to jail after what happened to Aaron Swartz
Their wealth has exceeded an escape velocity beyond which they won't be put in prison (or if they are they would quickly be pay-for-play pardoned) unless they are seen as a threat to even wealthier people.
See, for example: Devon Archer, Jason Galanis, Benjamin Delo, Arthur Hayes, Samuel Reed, Trevor Milton, Carlos Watson, Paul Walczak, Todd and Julie Chrisley, Lawrence Duran, Marian Morgan, Imaad Zuberi, Changpeng Zhao (CZ), Joseph Schwartz, et al.
Some of these people are broke bitches compared to the group of people you're talking about now, and yet still hit the threshold of being above the law as long as they play the corruption game.
Nobody seems to care about AI automating coders out of a job.
If anyone can just prompt all their basic “information needs” however how sloppy, then what remains of the economy? Health care, child care, handyman?
Most people won’t even pay for ad free YouTube. I don’t think any software business can survive AI as a substitute good even if it’s inferior (and it might not be).
Meanwhile housing, health care,
food, energy, and things like college all get more expensive while wages are stagnant (and falling when inflation adjusted). Now on top of that we toss a new jobs disrupting technology into the mix.
You’re making a lot of things objectively better, but none of them are the essentials that people need.
I’m not saying it’s bad to make AI bots or video games or network apps. I’m saying that the blanket statement “we make things better” is oblivious to a lot of realities.
Nope, because there's always been an executive to shaft said engineers out of success. But they went to business school, so I guess what's yours is theirs. I think that's biz 101.
You may not like AI but their business model is a silly twitter-worthy dunk. This is obviously not their business model. I didn't know I could've plagiarized all this code myself the entire time!
This is one of the reasons I'm extremely unimpressed by complaints from openai and anthropic that other labs are "distilling" their models based on them... Basically: "You're training your model by running it again our own model which is itself a gargantuan copyright and content violation of a scale never seen before by humankind, HOW DARE YOU"
Steal one mp3 and you might get fined thousands, steal a book from your local shoppe and the police would come visit you. Forge a signature and you would also be in trouble. Hack a government website and you will have to answer some questions.
Steal all the books in the world, forge millions and this story begins to tell and nothing happens.
The part of the AI generating the mashup doesn’t really understand what the signature means, or how it may be interpreted, it’s just a visual part of a New Yorker cartoon. Elsewhere in ChatGPT there is plenty of information about signatures and what they mean, and what plagiarism is.
It’s not organized like a human brain, it shouldn’t be surprising that unusual results occur. They are approximating human intelligence from a different angle. It’s interesting to see the improvements in areas like this that require introspection that isn’t fully wired up yet.
[edit] I should add that a human making a New Yorker cartoon is extremely iterative and introspective. Current generative AI is meant to push it out, and you can do the iteration and introspection yourself.
I have used a simple feedback loop to generate AI art: 1) the initial prompt 2) ask for criticism of the generated image 3) apply those changes and generate another image 4) repeat the criticism / fix cycle again
>I'm certain if I drew a cartoon and used the signature that looks just like a real cartoonists', I would get sued and liable for the damage.
Would you? Always? Suppose I hate Obama drone striking people, so I made a satirical cartoon of him signing an executive order to "bomb brown people" or whatever, affixing his signature[1] to that image. Would that get me in trouble, even if the image was clearly satirical? What if someone takes that, then passes it as non-satire, either intentionally or unintentionally?
There's nothing illegal unless someone seriously thinks Obama signed it. The easiest example would be his signature on his wikipedia page. Clearly he didn't sign that page, nor (probably) did he authorize it.
I don't understand your argument. Wikipedia and a satirical cartoon are both things clearly not made by Obama, nobody in their right mind would be "fooled" by the presence of his signature.
These cartoons are intentionally drawn in the style of a cartoonist and feature their signature. The goal is forgery. The whole point of these AI-generated images is to look like the real thing.
This has been a perennial problem with my own generated comics with both Nano Banana Pro and ChatGPT (all generations). I often have to put in an extra edit to erase the false signature. It is annoying and I'm unsurprised most users don't bother.
And then people wonder why the default mood of AI is so pessimistic. It's just revealing all of society's broken windows and adding a few more in the process.
Despite what all the clickwrap warnings and "AI can make mistakes" subtitles might lead you to believe, the service offering of AI is explicitly designed to be as "one and done" as possible. The inherent nature of these tools is to service laziness, and disincentivize too much scrutiny.
Engineering manager at my company put a comic at the end of our sprint demo that was signed bloper. Except it wasn’t funny at all, and kind of weird. I asked him, sure enough it was ChatGPT and he didn’t notice the signature.
this is trivial to reproduce ever since early stable diffusion models, it's not really an oai exclusive issue.
for example if you ask any of these models (just about any image-gen) to produce Japanese ukiyo-e art they will almost always produce it with a hanko[0] that has been seen a lot in historical art pieces, usually having nothing to do with the era or style of the replica but seen so often in 'Japanese artwork' that it's just permanently tokenized into it as a defining characteristic.
Nice. This demolishes the "LLMs can reason" (but not enough to avoid this sort of basic error) and "humans make mistakes too" (not like this) talking points from the LLM promoters.
This is completely silly. If you don’t think LLMs can reason, you’ve either never used them to do tasks that require reasoning, or you don’t understand enough to recognize what’s involved in the responses you get.
In this case it’s clearly the latter, because you’re confusing image generation models with LLMs. There are very big differences between the two. No-one is claiming that image generation models are capable of reasoning.
The thing about a generative language model that’s trained from a massive but unknown corpus is, it’s practically (if not theoretically) impossible to evaluate the extent to which data leakage contributes to any particular output.
But I would argue that, as things currently stand, “sophisticated engine for approximately querying a pastiche of the results of human reasoning that comprise its training corpus” remains a more parsimonious explanation than “it’s doing actual reasoning” for how this neural network architecture produces the phenomena we’ve been observing.
>sophisticated engine for approximately querying a pastiche of the results of human reasoning that comprise its training corpus
Well if the thing can find and fix bugs in something that is using non-mainstream stuff that is surely not in it's training dataset, that's better than a rubber duck already. Whether it has soul is a different question of course.
Don’t even bother. These people almost always have some goofy ass, non standard, fluid definition of “thinking” or “reasoning” that cannot ever be met.
This reminds me my early attempts to use GitHub Copilot when it just straight added some guy's name in a javadoc copyright note in the code it generated.
> “It’s like somebody attributed a quote to me that I didn’t say.”
> Katzenstein considers the reproduction of his signature by ChatGPT to be more than just a violation of intellectual property; to him, it’s closer to false impersonation. “[ChatGPT] is attaching my name to work that I do not endorse or like. It’s slop, and unlike the other slop that I’ve encountered, this is slop that’s pretending to be me.”
> “I’ve had people hack my credit card,” said Joe Dator, a New Yorker contributor for the past 20 years. “That feels like less of a violation than this. When they hacked my credit card, they didn’t dress up like me.”
So this has morphed from plagiarism and copyright infringement (bad) to impersonation (also bad, arguably worse, and maybe more provable in court). It’s chilling to think of the implications of having one’s signature attached to a document or to words that are not one’s own.
And I think maybe it's time for that. People need to learn that there's real, expensive legal liability for doing stuff like this. And AI companies the same.
I am very much not an advocate of "sue everybody for everything". This is major enough that it clears my threshold.
It seems like all the criticisms of Gen AI and LLM seems to concentrate on OpenAI and their products over products from anthropic and others.
I am pretty sure that this faking of signature can be done by Gemini, claude as easily as chatgpt.
Whatever the original intention, this is clearly a bug and should be fixed. But should ChatGPT sign its cartoons with its own name or leave them unsigned?
No one should be allowed to claim they drew something when they didn’t draw any part of it and LLM’s are not people/can’t work without a person. We don’t credit pens and paintbrushes after all.
One could argue nobody should be allowed to claim it. It just exists.
Even if the person can't claim the copyright of the image produced they ARE responsible for the use of their tools and what they do with the output.
In this case, they released an image with someone else's signature on it. That is wrong, the person should take the blame for that.
The person releasing the image may take it up with the AI service that their tooling led them into making such a mistake. But good luck with that in court...
i’m not saying it’s airtight, this is a pretty unbaked idea. But I think it pretty clearly has some legs to stand on. Can’t be any worse of an argument than somebody prompting an AI for a (facsimile of a) photograph and going “I made a photo.” And mind you I’ve heard people argue that that absolutely constitutes taking a photograph (which I wholly disagree with) here on HN.
Somebody “made something.” But just because you do something doesn’t mean you get to claim whole ownership of it and get to sign it with your name. Plenty of examples in life.
I think what you're seeing is the probability of a particular signature or style of signature appearing on a particular style of cartoon, not an intent to sign.
This. It's also while you'll sometimes get a mangled Getty Images watermark on some image generations, or a logo in the bottom left corner. If it's a prominent feature in the training dataset it'll show up, exactly how these models are supposed to work.
The 'bug' here is whatever post-processing step or system prompt is in place to steer the model away from doing this.
Why is it a bug? If other parts of the generated illustration are similarly taken from an artist, why not the signature as well? Why is a signature crossing the line but the rest of the image isn't?
For the same reason I'm allowed to draw, paint, or write things very similar to what others have drawn, painted, or written but I have to sign my own name not theirs.
You are a person, LLMs are not. You know this, which is why you know that if you signed someone else's name it would be forgery, but when you see the machine do it you call it a bug.
If the machine is like you, the machine is a forger. The machine is not like you, it is simply blending the work of others to order. Adding someone else's signature is simply part of that statistical process.
I've seen photorealistic generations add (distorted but still somewhat recognisable) watermarks too, because that's what the training data had.
Everything is a derivative work, and always has been. AI is just making that salient fact so much more visible, and now everyone who believes in the delusion of Imaginary Property is scared at that truth revealing itself.
Incidentally, this is also what young humans learning to draw will do. They start by copying what they've seen.
Man, forget the debate about AI being dangerous. The real reason all these companies need to be shut down is copy right infringement. None of this is "fair use." Does fair use imply doing things that actively harm the creators of works? Because training models to be able to replicate works only makes their skills less valuable...
Honestly, you all kind of mind fuck me that you're not more pissed off about AI basically replicating a large portion of your skills. This effects so many professions now and it's only going to get worse. I would expect a far greater outcry from software engineers trying to organise to ban this shit. But its like none of you even care?
I follow anti-LLM discourse quite a lot, and across the main bulletpoints: energy/carbon emissions, content worker harm, job displacement, deskilling, mental health effects, and copyright/plagiarism, the plagiarism one seems to have the most attention, and it's also the most solvable, if there were only more serious effort on ethically sourced models that can actually do the real science / math / code work that is what LLMs are best at. The whole world of LLMs to create videos/books/art/literature is where most of the offense is (the video/imagery side of it is where most of the energy/carbon emissions problems are too. and content worker harm).
I really wish there'd be a split among these disciplines (science/math/code vs. videos/art/literature) - one is vastly more problematic than the other.
i think math is close to art (just to be a contrarian, but kind of really)
its somewhat funny that math people are in a conundrum as to support or not support but this might partially be because some wish to believe that math itself is and can be useful and therefore accelerating is good
but the art people have no such delusions so they’re just strictly against
imo proof writing is more akin to art than coding/tech but…
Yep. I’d probably be a lot less chastised in some circles for using Claude Code at work if it wasn’t misconstrued as being in support of, I don’t know, encroaching on the hypothetical commissions of a chronically online instagram furry artist or something.
It is very tiring to say “I don’t necessarily disagree with you about AI ‘art’, but in my field—which you do not understand, and in which the underlying build process is often not the creative output—AI presents very real productivity gains” for the umpteenth time.
I am skeptical of there being sufficient data to build “ethical” training datasets, and I’m confident that much of the same contingent will (somewhat rightfully) argue that ‘second-generation’ copyrighted AI material has already irreversibly made its way into every modern dataset.
> has already irreversibly made its way into every modern dataset
The "gray goo" scenario finally happens... for AI. That's actually the good ending for humanity. I love it! Poetic and believable. Data doesn't "heal" like nature. :D
2. Acquire rights/licenses to any datasets that do not fit #1. e.g. the Google deal with Reddit for 60m/yr.
3. Offer programs to have creatives willingly submit their data, with some sort of residual output based on the number of times their assets are sampled.
4. If all that is still not enough, hire creatives to create assets for you. This is something Spotify did recently with "ghost artists"[0]. The intentions here are suspect, but a non-consumer facing artist providing work for an LLM wouldn't have the same ethical dilemmas
5. Lastly, if all that still isn't enough: governmental programs to either provide grants, subsidies, or more outreach to get the ball rolling.
Would this cost tens, hundreds of billions of dollars? Yes. But clearly, that was not a barrier to entry for the industry anyway. So we can chalk this down to the personality of leadership or the wider culture of modern big tech
>“I don’t necessarily disagree with you about AI ‘art’, but in my field—which you do not understand, and in which the underlying build process is often not the creative output—AI presents very real productivity gains”
Being able to prove such gains in better products would be a start. And an emphasis on how it assists existing engineers/mathmaticians/researchers, not that any accomplishment made with AI assistance is "AI solves problem".
I don't know whatever happened to "words are cheap". I guess it literally made money to say words, so that adage is false for the time being.
>I am skeptical of there being sufficient data to build “ethical” training datasets
Well if all those scam job ads paying 100/hr to create AI training content was not a scam and instead the approach from the start, there may have been a chance to bridge that gap ethically. The industry chose to break things and is trying to act mad that people are mad at all the broken stuff.
These results are entirely a consequences of the actions chosen. And I don't believe there was ever an honest consideration of there being ethical training datasets. They just thought they could brute force society with fearmongering and bribes. The BOTD was already low in the beginning but completely gone now.
I think you can train on math /science using synthetic generation to a significant extent. Training for coding requires more of the "scraping github / stackoverflow" angle but IMO that's a shallower hill to climb than scraping copyrighted art and literature.
There are actual models trained on ethical datasets but they are obviously not very high powered. If companies with the resources of an anthropic or openai were doing it (ha) it would be more feasible
> It is very tiring to say “I don’t necessarily disagree with you about AI ‘art’, but in my field—which you do not understand, and in which the underlying build process is often not the creative output—AI presents very real productivity gains” for the umpteenth time.
That’s not a justification. If a company were poisoning the water to your home as a byproduct, would you be satisfied if they told you “we don’t necessarily disagree with you about polluting the water, but in our field—which you do not understand, and in which the underlying build process is often not the water pollution—what we’re doing presents very real productivity gains”?
> I am skeptical of there being sufficient data to build “ethical” training datasets
Then you don’t build any. What fucked up world we live in where people think it’s OK to be unethical because they want something and can’t think of any other way to do it. What monumentally selfish rotten babies.
A reasonable next move, if the US was interested in acting like a democracy, would be legislation forcing this decision:
- prove that you had the rights for all of your training data
- open source the model
Give the labs a 3 month grace period in which to comply, so competition can persist even with dubiously sourced data, but the people can't be locked away from derivatives of their contributions for any significant amount of time.
> I really wish there'd be a split among these disciplines (science/math/code vs. videos/art/literature) - one is vastly more problematic than the other.
I disagree that they can be separated. Practically, I think they can't. Because the mere invention of new tools inspires even more AI advancement and that in turn will cause the other side (artistic side) to degenerate even more.
I'm anti-LLM all the way, 100%, no exceptions. Zero tolerance.
So you recognize that discovery cannot be cut off from other discovery, that it's never just one or the other, and your solution is to say shut down all discovery?
I don't really see the difference, code is protected by copyright (or copyleft) as much as art is, and yet the LLM scrapers use it without scruples. Same goes for math and science publications.
openai and anthropic trained on actually stolen data since it was pirated datasets.
google OTOH already had a lot of this dataset in their possession (e.g. Google Books etc), still questionably licensed for how they used it, but not quite as bad. They did apparently break through NYT paywalls and stuff like that though, still theft.
While that is true, theoretically a regulation could be enacted that output tokens must focus on STEM research and other practical tasks and the LLM must refuse tasks outside of those areas, just as Claude disallowed cybersecurity tasks. Obviously this would never happen, but the theft of the training data wouldn’t matter as much if the usecases were less sinister.
In my experience, the science/math/code crowd don't care about copyright/plagiarism as much as the video/art/literature crowd, so the first crowd turns a blind eye to most of the latter crowd talks about.
Code can be art, and copyright/plagiarism is real. It sort of boils down to how much it bothers us.
I don't think it is likely that they could get enough data without stealing. It would be incredibly costly to have to pay artists to church out art just to train an AI.
This is a pretty solid argument against people who argue that LLMs are more than just (very massive) next token predictors.
If there was any thought or underlying thought going on here not putting a signature (at least a real one) would be the right move, despite it being less likely. It would realize, while generating the pixels that eventually became a signature, that it shouldn't do that.
> This is a pretty solid argument against people who argue that LLMs are more than just (very massive) next token predictors.
This quote is a pretty solid argument that you need to understand the technology you’re trying to criticize better. This issue has nothing to do with LLMs. LLMs are not image generation models.
If you do not want to be called a duck, it would help if you stopped quacking like one. Maybe you aren't a duck, but you aren't helping your case with stories like this about how AI generates images.
A LLM, at least, prompted and served this image. LLMs can ingest images. They actually can generate them as well but that probably wasn't done here.
In the course of the conversation a with chatgpt, this image was generated and served by an LLM. It clearly shouldn't have been by any sort of reasoning.
Of course it does this. The training data is full of examples that associate e.g. "New Yorker-style cartoon" with Loper's signature in the corner, because Loper's signature is in the corner of a lot of them. There's nothing to make it treat the signature as anything special by default; that would have to be trained in explicitly.
One would hope that the person prompting ChatGPT would notice this sort of thing and do something about it before sharing it publicly, out of a genuine desire not to cause confusion etc. But I guess that's way more personal responsibility than we can expect average people to take on nowadays.
The problem is not that ChatGPT is doing that, the problem is that it's not being sued into oblivion after.
>it's not being sued into oblivion after.
Article said
>she wrote she had simply asked ChatGPT to make “a New Yorker-style cartoon.”
A "style" can't be copyrighted, at least in US law. They might have a stronger case of trademark/likeness infringement, but the fact that the person knew it was AI generated would make that difficult. Of course they knew it wasn't made by Brendan Loper. Of course, if they then published it, the other people viewing it might not know this, but who published it?
The signature can certainly be copyright protected, I would think?
Wikipedia doesn't think so, citing the US copyright office
https://commons.wikimedia.org/wiki/Commons:When_to_use_the_P...
This is true, it'd be a trademark issue if anything, not a copyright one. From this page:
> it may be reproduced, as long as the reproduction cannot be mistaken for an authentic signature.
>as long as the reproduction cannot be mistaken for an authentic signature.
Which seems applicable in this case, because the image is clearly generated by AI (at least to the guy who prompted it).
Visual artists have a right of attribution via the Visual Artists Rights Act (17 U.S.C. § 106A). This includes protection against misattribution.
There's a difference between reproducing a signature in an encyclopedia or in some way that makes it clear that you are recording the thing as it is.
Putting a signature on a work is forgery and in most jurisdictions charged as fraud.
If you produce an artwork in the style of someone and then clone the signature of someone who produces art in that style, there is a reasonable case for fraud.
When you are no-one, it’s fraud. “When you're famous they let you do it”
Why would ChatGPT be sued exactly? They didn’t publish the picture, they did what the user asked. The user is responsible because they directed the creation and publishing. The user is the entity who should be sued. Or dmca’d. Or whatever.
Think if the user commissioned the art from an outsourced creative shop nobody has heard of. Then they published it. They wouldn’t go after the creative shop, they would go after the publisher.
(I am just addressing publishing here, training on the artist’s works is a different, well discussed issue)
Seems to me like producing an artist's stylised signature would be a trademark infringement.
OpenAI give lip service to the idea of not producing others' intellectual property - go ask it to explicitly make a picture of the genie from Aladdin.
Plagiarism as a Service.
We can all try really hard to pretend that's not the business model, but that's totally the business model.
What Microsoft's Director of Applied Science called the, "largest theft of labor in human history".[1]
1. https://storage.courtlistener.com/recap/gov.uscourts.nysd.64...
All of these people need to go to jail after what happened to Aaron Swartz
If you think Aaron Schwartz would not have fought for exactly the freedom of information that would enable AI training, you misunderstand Schwartz and the nature of information freedom.
I continue to find that people strongly advocate for justice on exactly opposing sides, depending on who they have been told to think the “bad guy” is.
I think you misunderstood what parent was saying
The same orgs that harassed and antagonized Aaron Schwartz are not going after AI for the same thing at a much much larger scale.
What's different? The size of their bank accounts.
> What's different?
One, time went on. The prosecution of Schwartz was seen as an overreach, as demonstrated by Ortiz’s failed political career thereafter.
Two, Schwartz’s charges were only ever charged. No jury or judge signed off on them.
Three, context changed. Schwartz wasn’t 5% of GDP. For better or for worse, that matters to voters.
Voters might feel differently once they wake up and realize that it's 5% of GDP because of the anticipation that so many voters will be thrown out of work.
> once they wake up and realize that it's 5% of GDP because of the anticipation that so many voters will be thrown out of work
But it's not. The only layer anyone has a handle on is consumers and companies paying AI ridiculous sums for AI. Some of those users are probably justifying it with labour replacement. But a lot may not be. So far, we haven't seen the employment effect outside recent college graduates at enterprise companies.
> as implemented in the United States it's a malignant tumor, a theft of labor by capital, and it must be (minimally) adddressed with confiscatory taxes applied to all those involved with it's creation and operation
It's also a godsend of economic growth. Growth other countries who are trying to balance their books would kill for. Without AI, we'd be in a failure state. Maybe we are, if this is all a bubble. But as it stands, there is paper wealth that can and has–limitedly–been taxed. That gives everyone options.
> If a single AI billionaire exists in the year 2030, then the US is a failed state
This is silly and projecting a narrow view of the world onto a larger voting population. Voters don't care so much that there are billionaires as that living standards haven't kept up with the rate at which they're being minted. Double tax brackets, add more on top, raise the minimum wage, raise Social Security taxes and benefits, expand Medicare, beef up antitrust, establish a progressive property tax on wealth that starts at 1,000x the median American's wage (about $65mm) and billionaires are fine.
> All of these people need to go to jail after what happened to Aaron Swartz
Their wealth has exceeded an escape velocity beyond which they won't be put in prison (or if they are they would quickly be pay-for-play pardoned) unless they are seen as a threat to even wealthier people.
See, for example: Devon Archer, Jason Galanis, Benjamin Delo, Arthur Hayes, Samuel Reed, Trevor Milton, Carlos Watson, Paul Walczak, Todd and Julie Chrisley, Lawrence Duran, Marian Morgan, Imaad Zuberi, Changpeng Zhao (CZ), Joseph Schwartz, et al.
Some of these people are broke bitches compared to the group of people you're talking about now, and yet still hit the threshold of being above the law as long as they play the corruption game.
Nobody seems to care about AI automating coders out of a job.
If anyone can just prompt all their basic “information needs” however how sloppy, then what remains of the economy? Health care, child care, handyman?
Most people won’t even pay for ad free YouTube. I don’t think any software business can survive AI as a substitute good even if it’s inferior (and it might not be).
> Nobody seems to care about AI automating coders out of a job.
Have people ever really broadly cared about the IT professionals behind their devices?
IT professionals have been automating people out of jobs for years. Why would anyone worry about them?
Wait, were we the bad guys all this time by making things objectively better?
This attitude is why society has turned on the tech bros.
Meanwhile housing, health care, food, energy, and things like college all get more expensive while wages are stagnant (and falling when inflation adjusted). Now on top of that we toss a new jobs disrupting technology into the mix.
You’re making a lot of things objectively better, but none of them are the essentials that people need.
I’m not saying it’s bad to make AI bots or video games or network apps. I’m saying that the blanket statement “we make things better” is oblivious to a lot of realities.
After decades of enshitification, I have trouble reading this as "objectively better for those made wealthy by working in or investing in tech."
AI is making things objectively better too :)
Nope, because there's always been an executive to shaft said engineers out of success. But they went to business school, so I guess what's yours is theirs. I think that's biz 101.
You may not like AI but their business model is a silly twitter-worthy dunk. This is obviously not their business model. I didn't know I could've plagiarized all this code myself the entire time!
This is one of the reasons I'm extremely unimpressed by complaints from openai and anthropic that other labs are "distilling" their models based on them... Basically: "You're training your model by running it again our own model which is itself a gargantuan copyright and content violation of a scale never seen before by humankind, HOW DARE YOU"
Whatever is necessary for the functioning of the (democratic) society. That would be legal.
btw, why isn't Greta Thunberg standing up in front of the UN delivering a "How dare you" speach about AI ?
Only if it's intentional, which it's clearly not. So chill.
We live in confusing times.
Steal one mp3 and you might get fined thousands, steal a book from your local shoppe and the police would come visit you. Forge a signature and you would also be in trouble. Hack a government website and you will have to answer some questions.
Steal all the books in the world, forge millions and this story begins to tell and nothing happens.
After all that has been said and written on this topic, people still manage to conflate theft with intellectual property violation, real or imagined.
The part of the AI generating the mashup doesn’t really understand what the signature means, or how it may be interpreted, it’s just a visual part of a New Yorker cartoon. Elsewhere in ChatGPT there is plenty of information about signatures and what they mean, and what plagiarism is.
It’s not organized like a human brain, it shouldn’t be surprising that unusual results occur. They are approximating human intelligence from a different angle. It’s interesting to see the improvements in areas like this that require introspection that isn’t fully wired up yet.
[edit] I should add that a human making a New Yorker cartoon is extremely iterative and introspective. Current generative AI is meant to push it out, and you can do the iteration and introspection yourself.
I have used a simple feedback loop to generate AI art: 1) the initial prompt 2) ask for criticism of the generated image 3) apply those changes and generate another image 4) repeat the criticism / fix cycle again
Isn't that somewhat introspective?
I'm certain if I drew a cartoon and used the signature that looks just like a real cartoonists', I would get sued and liable for the damage.
The same should apply to LLM vendors.
>I'm certain if I drew a cartoon and used the signature that looks just like a real cartoonists', I would get sued and liable for the damage.
Would you? Always? Suppose I hate Obama drone striking people, so I made a satirical cartoon of him signing an executive order to "bomb brown people" or whatever, affixing his signature[1] to that image. Would that get me in trouble, even if the image was clearly satirical? What if someone takes that, then passes it as non-satire, either intentionally or unintentionally?
[1] https://en.wikipedia.org/wiki/File:Barack_Obama_signature.sv...
That sounds like satire, as you suggested. Are you really unsure?
No, see: https://news.ycombinator.com/item?id=49973037
There's nothing illegal unless someone seriously thinks Obama signed it. The easiest example would be his signature on his wikipedia page. Clearly he didn't sign that page, nor (probably) did he authorize it.
I don't understand your argument. Wikipedia and a satirical cartoon are both things clearly not made by Obama, nobody in their right mind would be "fooled" by the presence of his signature.
These cartoons are intentionally drawn in the style of a cartoonist and feature their signature. The goal is forgery. The whole point of these AI-generated images is to look like the real thing.
wow. what an incredible way to miss the point. bravo.
Laws are for poor people
Why are you certain? All your time in the courtroom arguing copyright and trademark cases?
This has been a perennial problem with my own generated comics with both Nano Banana Pro and ChatGPT (all generations). I often have to put in an extra edit to erase the false signature. It is annoying and I'm unsurprised most users don't bother.
>I'm unsurprised most users don't bother.
And then people wonder why the default mood of AI is so pessimistic. It's just revealing all of society's broken windows and adding a few more in the process.
Despite what all the clickwrap warnings and "AI can make mistakes" subtitles might lead you to believe, the service offering of AI is explicitly designed to be as "one and done" as possible. The inherent nature of these tools is to service laziness, and disincentivize too much scrutiny.
> my own generated comics
For certain values of "my own".
Well, see for yourself: https://gwern.net/comics
"I made this"
https://lemmy.ca/post/63438408
[delayed]
Engineering manager at my company put a comic at the end of our sprint demo that was signed bloper. Except it wasn’t funny at all, and kind of weird. I asked him, sure enough it was ChatGPT and he didn’t notice the signature.
Artists should be able to personally sue OpenAI for libel every time they forge an artists signature.
The copyright washing machine strikes again.
That’s not a copyright issue, it’s a trademark issue.
this is trivial to reproduce ever since early stable diffusion models, it's not really an oai exclusive issue.
for example if you ask any of these models (just about any image-gen) to produce Japanese ukiyo-e art they will almost always produce it with a hanko[0] that has been seen a lot in historical art pieces, usually having nothing to do with the era or style of the replica but seen so often in 'Japanese artwork' that it's just permanently tokenized into it as a defining characteristic.
[0]: https://theartofzen.org/the-hanko-in-japanese-art-and-ukiyo-...
Nice. This demolishes the "LLMs can reason" (but not enough to avoid this sort of basic error) and "humans make mistakes too" (not like this) talking points from the LLM promoters.
> This demolishes the "LLMs can reason"
This is completely silly. If you don’t think LLMs can reason, you’ve either never used them to do tasks that require reasoning, or you don’t understand enough to recognize what’s involved in the responses you get.
In this case it’s clearly the latter, because you’re confusing image generation models with LLMs. There are very big differences between the two. No-one is claiming that image generation models are capable of reasoning.
LLMs can very much reason and they do it very well.
I remain unconvinced.
The thing about a generative language model that’s trained from a massive but unknown corpus is, it’s practically (if not theoretically) impossible to evaluate the extent to which data leakage contributes to any particular output.
But I would argue that, as things currently stand, “sophisticated engine for approximately querying a pastiche of the results of human reasoning that comprise its training corpus” remains a more parsimonious explanation than “it’s doing actual reasoning” for how this neural network architecture produces the phenomena we’ve been observing.
>sophisticated engine for approximately querying a pastiche of the results of human reasoning that comprise its training corpus
Well if the thing can find and fix bugs in something that is using non-mainstream stuff that is surely not in it's training dataset, that's better than a rubber duck already. Whether it has soul is a different question of course.
“A soul”?
As popular as I know the rhetorical tactic is on both sides of these discussions about LLMs, I’d still thank you not to strawman me.
Don’t even bother. These people almost always have some goofy ass, non standard, fluid definition of “thinking” or “reasoning” that cannot ever be met.
There's a lot of space between:
- LLM's can reason
- everything an LLM does is the result of reasoning
This demolishes only the latter point, which as far as I know has no supporters.
This reminds me my early attempts to use GitHub Copilot when it just straight added some guy's name in a javadoc copyright note in the code it generated.
signatures in, signatures out
> “It’s like somebody attributed a quote to me that I didn’t say.”
> Katzenstein considers the reproduction of his signature by ChatGPT to be more than just a violation of intellectual property; to him, it’s closer to false impersonation. “[ChatGPT] is attaching my name to work that I do not endorse or like. It’s slop, and unlike the other slop that I’ve encountered, this is slop that’s pretending to be me.”
> “I’ve had people hack my credit card,” said Joe Dator, a New Yorker contributor for the past 20 years. “That feels like less of a violation than this. When they hacked my credit card, they didn’t dress up like me.”
So this has morphed from plagiarism and copyright infringement (bad) to impersonation (also bad, arguably worse, and maybe more provable in court). It’s chilling to think of the implications of having one’s signature attached to a document or to words that are not one’s own.
This is lawsuit material.
And I think maybe it's time for that. People need to learn that there's real, expensive legal liability for doing stuff like this. And AI companies the same.
I am very much not an advocate of "sue everybody for everything". This is major enough that it clears my threshold.
It seems like all the criticisms of Gen AI and LLM seems to concentrate on OpenAI and their products over products from anthropic and others. I am pretty sure that this faking of signature can be done by Gemini, claude as easily as chatgpt.
Claude doesn’t do image generation?
No image Gen, trying to get the pope to make Claude a real boy, selling users out to the police, Anthropic looking more shit every day
Not putting signatures in art unless otherwise requested should become a default in AI instances.
Whatever the original intention, this is clearly a bug and should be fixed. But should ChatGPT sign its cartoons with its own name or leave them unsigned?
No one should be allowed to claim they drew something when they didn’t draw any part of it and LLM’s are not people/can’t work without a person. We don’t credit pens and paintbrushes after all.
One could argue nobody should be allowed to claim it. It just exists.
But it didn't just blink into existence.
Even if the person can't claim the copyright of the image produced they ARE responsible for the use of their tools and what they do with the output.
In this case, they released an image with someone else's signature on it. That is wrong, the person should take the blame for that.
The person releasing the image may take it up with the AI service that their tooling led them into making such a mistake. But good luck with that in court...
i’m not saying it’s airtight, this is a pretty unbaked idea. But I think it pretty clearly has some legs to stand on. Can’t be any worse of an argument than somebody prompting an AI for a (facsimile of a) photograph and going “I made a photo.” And mind you I’ve heard people argue that that absolutely constitutes taking a photograph (which I wholly disagree with) here on HN.
Somebody “made something.” But just because you do something doesn’t mean you get to claim whole ownership of it and get to sign it with your name. Plenty of examples in life.
I mean, the bug is, "it generated the most likely cluster of pixels in the corner of a New Yorker cartoon".
I think what you're seeing is the probability of a particular signature or style of signature appearing on a particular style of cartoon, not an intent to sign.
This. It's also while you'll sometimes get a mangled Getty Images watermark on some image generations, or a logo in the bottom left corner. If it's a prominent feature in the training dataset it'll show up, exactly how these models are supposed to work.
The 'bug' here is whatever post-processing step or system prompt is in place to steer the model away from doing this.
Yes, but that's just the pixel generation layer. The harness or whatever infrastructure around could nudge it towards common sense.
I do many voice transcriptions. Many times I've had empty silence at the end of recordings transcribed as "Thank you" or even "Like and subscribe".
Why is it a bug? If other parts of the generated illustration are similarly taken from an artist, why not the signature as well? Why is a signature crossing the line but the rest of the image isn't?
For the same reason I'm allowed to draw, paint, or write things very similar to what others have drawn, painted, or written but I have to sign my own name not theirs.
You are a person, LLMs are not. You know this, which is why you know that if you signed someone else's name it would be forgery, but when you see the machine do it you call it a bug.
If the machine is like you, the machine is a forger. The machine is not like you, it is simply blending the work of others to order. Adding someone else's signature is simply part of that statistical process.
Because it removes plausible deniability, duh
We call it a "watermark" when a machine "signs" its work. And I wouldn't be surprised if its required as a part of AI disclosure in the coming years.
SynthID watermark but no visible signature
Makes sense. Pretty much every example it is trained on has a signature.
I've seen photorealistic generations add (distorted but still somewhat recognisable) watermarks too, because that's what the training data had.
Everything is a derivative work, and always has been. AI is just making that salient fact so much more visible, and now everyone who believes in the delusion of Imaginary Property is scared at that truth revealing itself.
Incidentally, this is also what young humans learning to draw will do. They start by copying what they've seen.
Hey anybody remember what happened to Aaron Swartz?
This is why I laugh when the companies cry about model distillation "attacks".
"you're trying to kidnap what I've rightfully stolen"
Let's see the prompts, without that this discussion is meaningless.
Man, forget the debate about AI being dangerous. The real reason all these companies need to be shut down is copy right infringement. None of this is "fair use." Does fair use imply doing things that actively harm the creators of works? Because training models to be able to replicate works only makes their skills less valuable...
Honestly, you all kind of mind fuck me that you're not more pissed off about AI basically replicating a large portion of your skills. This effects so many professions now and it's only going to get worse. I would expect a far greater outcry from software engineers trying to organise to ban this shit. But its like none of you even care?
I follow anti-LLM discourse quite a lot, and across the main bulletpoints: energy/carbon emissions, content worker harm, job displacement, deskilling, mental health effects, and copyright/plagiarism, the plagiarism one seems to have the most attention, and it's also the most solvable, if there were only more serious effort on ethically sourced models that can actually do the real science / math / code work that is what LLMs are best at. The whole world of LLMs to create videos/books/art/literature is where most of the offense is (the video/imagery side of it is where most of the energy/carbon emissions problems are too. and content worker harm).
I really wish there'd be a split among these disciplines (science/math/code vs. videos/art/literature) - one is vastly more problematic than the other.
i think math is close to art (just to be a contrarian, but kind of really)
its somewhat funny that math people are in a conundrum as to support or not support but this might partially be because some wish to believe that math itself is and can be useful and therefore accelerating is good
but the art people have no such delusions so they’re just strictly against
imo proof writing is more akin to art than coding/tech but…
Yep. I’d probably be a lot less chastised in some circles for using Claude Code at work if it wasn’t misconstrued as being in support of, I don’t know, encroaching on the hypothetical commissions of a chronically online instagram furry artist or something.
It is very tiring to say “I don’t necessarily disagree with you about AI ‘art’, but in my field—which you do not understand, and in which the underlying build process is often not the creative output—AI presents very real productivity gains” for the umpteenth time.
I am skeptical of there being sufficient data to build “ethical” training datasets, and I’m confident that much of the same contingent will (somewhat rightfully) argue that ‘second-generation’ copyrighted AI material has already irreversibly made its way into every modern dataset.
> has already irreversibly made its way into every modern dataset
The "gray goo" scenario finally happens... for AI. That's actually the good ending for humanity. I love it! Poetic and believable. Data doesn't "heal" like nature. :D
Yeah and it only took stealing all the intellectual output of everyone ever made.
But sure, there's um, an ethical way of doing that?
In theory, sure.
1. Only use open source/CC compliant assets.
2. Acquire rights/licenses to any datasets that do not fit #1. e.g. the Google deal with Reddit for 60m/yr.
3. Offer programs to have creatives willingly submit their data, with some sort of residual output based on the number of times their assets are sampled.
4. If all that is still not enough, hire creatives to create assets for you. This is something Spotify did recently with "ghost artists"[0]. The intentions here are suspect, but a non-consumer facing artist providing work for an LLM wouldn't have the same ethical dilemmas
5. Lastly, if all that still isn't enough: governmental programs to either provide grants, subsidies, or more outreach to get the ball rolling.
Would this cost tens, hundreds of billions of dollars? Yes. But clearly, that was not a barrier to entry for the industry anyway. So we can chalk this down to the personality of leadership or the wider culture of modern big tech
[0]: https://harpers.org/archive/2025/01/the-ghosts-in-the-machin...
>“I don’t necessarily disagree with you about AI ‘art’, but in my field—which you do not understand, and in which the underlying build process is often not the creative output—AI presents very real productivity gains”
Being able to prove such gains in better products would be a start. And an emphasis on how it assists existing engineers/mathmaticians/researchers, not that any accomplishment made with AI assistance is "AI solves problem".
I don't know whatever happened to "words are cheap". I guess it literally made money to say words, so that adage is false for the time being.
>I am skeptical of there being sufficient data to build “ethical” training datasets
Well if all those scam job ads paying 100/hr to create AI training content was not a scam and instead the approach from the start, there may have been a chance to bridge that gap ethically. The industry chose to break things and is trying to act mad that people are mad at all the broken stuff.
These results are entirely a consequences of the actions chosen. And I don't believe there was ever an honest consideration of there being ethical training datasets. They just thought they could brute force society with fearmongering and bribes. The BOTD was already low in the beginning but completely gone now.
I think you can train on math /science using synthetic generation to a significant extent. Training for coding requires more of the "scraping github / stackoverflow" angle but IMO that's a shallower hill to climb than scraping copyrighted art and literature.
There are actual models trained on ethical datasets but they are obviously not very high powered. If companies with the resources of an anthropic or openai were doing it (ha) it would be more feasible
> It is very tiring to say “I don’t necessarily disagree with you about AI ‘art’, but in my field—which you do not understand, and in which the underlying build process is often not the creative output—AI presents very real productivity gains” for the umpteenth time.
That’s not a justification. If a company were poisoning the water to your home as a byproduct, would you be satisfied if they told you “we don’t necessarily disagree with you about polluting the water, but in our field—which you do not understand, and in which the underlying build process is often not the water pollution—what we’re doing presents very real productivity gains”?
> I am skeptical of there being sufficient data to build “ethical” training datasets
Then you don’t build any. What fucked up world we live in where people think it’s OK to be unethical because they want something and can’t think of any other way to do it. What monumentally selfish rotten babies.
A reasonable next move, if the US was interested in acting like a democracy, would be legislation forcing this decision:
- prove that you had the rights for all of your training data
- open source the model
Give the labs a 3 month grace period in which to comply, so competition can persist even with dubiously sourced data, but the people can't be locked away from derivatives of their contributions for any significant amount of time.
Models aren't in school so plagiarism doesn't really apply
I can't tell if you're serious or not.
> I really wish there'd be a split among these disciplines (science/math/code vs. videos/art/literature) - one is vastly more problematic than the other.
I disagree that they can be separated. Practically, I think they can't. Because the mere invention of new tools inspires even more AI advancement and that in turn will cause the other side (artistic side) to degenerate even more.
I'm anti-LLM all the way, 100%, no exceptions. Zero tolerance.
So you recognize that discovery cannot be cut off from other discovery, that it's never just one or the other, and your solution is to say shut down all discovery?
I don't really see the difference, code is protected by copyright (or copyleft) as much as art is, and yet the LLM scrapers use it without scruples. Same goes for math and science publications.
They can't be ethically sourced and good at the same time.
The current models intelligence depends on massive training dataset of essentially stolen data
openai and anthropic trained on actually stolen data since it was pirated datasets.
google OTOH already had a lot of this dataset in their possession (e.g. Google Books etc), still questionably licensed for how they used it, but not quite as bad. They did apparently break through NYT paywalls and stuff like that though, still theft.
While that is true, theoretically a regulation could be enacted that output tokens must focus on STEM research and other practical tasks and the LLM must refuse tasks outside of those areas, just as Claude disallowed cybersecurity tasks. Obviously this would never happen, but the theft of the training data wouldn’t matter as much if the usecases were less sinister.
In my experience, the science/math/code crowd don't care about copyright/plagiarism as much as the video/art/literature crowd, so the first crowd turns a blind eye to most of the latter crowd talks about.
Code can be art, and copyright/plagiarism is real. It sort of boils down to how much it bothers us.
I hope thats possible but I am not sure if proper knowledge of science can be had without also learning other literature (and vice versa).
I don't think it is likely that they could get enough data without stealing. It would be incredibly costly to have to pay artists to church out art just to train an AI.
This is a pretty solid argument against people who argue that LLMs are more than just (very massive) next token predictors.
If there was any thought or underlying thought going on here not putting a signature (at least a real one) would be the right move, despite it being less likely. It would realize, while generating the pixels that eventually became a signature, that it shouldn't do that.
> This is a pretty solid argument against people who argue that LLMs are more than just (very massive) next token predictors.
This quote is a pretty solid argument that you need to understand the technology you’re trying to criticize better. This issue has nothing to do with LLMs. LLMs are not image generation models.
>LLMs are not image generation models.
If you do not want to be called a duck, it would help if you stopped quacking like one. Maybe you aren't a duck, but you aren't helping your case with stories like this about how AI generates images.
Quiz: What does the second "L" in "LLM" stand for?
Hint: it isn't "image".
A LLM, at least, prompted and served this image. LLMs can ingest images. They actually can generate them as well but that probably wasn't done here.
In the course of the conversation a with chatgpt, this image was generated and served by an LLM. It clearly shouldn't have been by any sort of reasoning.
meh... who cares