Nightshade: Make Images Unsuitable for Model Training

> We’ve seen this arms race before and know who wins. It’s all snake oil imo

It's kinda funny in a way because effectively they're helping iron out ways in which these models "see" differently to humans. Every escalation will in the end just help make the models more robust...

That they are disclosing the tools rather than e.g. creating a network service makes this even easier.

Idk. Perhaps this technique doesn't work, but if someone comes up with a working system, and LLMs start using techniques to counter it, artists might have a leg to stand upon, as the use of the counter-technique makes clear that the scraper never had any intention of respecting terms of use.

>> We’ve seen this arms race before and know who wins. It’s all snake oil imo

I haven't and I don't know who wins. Who wins?

Adversarial examples aren't snake oil, if that's what you meant. There's a rich literature on both producing and bypassing them that has accumulated over the years, but while I haven't kept abreast with it, my recollection is that the bottom line is like that for online security: there's never a good reason not to make sure your system is up to date and protected from attacks, even if there exist attacks that can bypass any defense.

Where in this case attack and defense can both describe what artists want to do with their work.

Doesn't mean artists should make it easy for these AI companies to steal artist IP. It doesn't take long to do and seems effective enough from what I've seen. BTW This is how cybersecurity works (cat and mouse etc)

AI model makers win, luddites lose.

Never mind that the more people try to corrupt a model, the more likely that future models will catch these corruption attempts as security and trust/safety issues to fix and work around.

The next Nightshade will eventually be viewed as malware to a model and then worked around, reconstructing around the attempt to break a model.

Isn't there a huge cost imbalance? As in easy to add some noise, difficult to remove reliably, so that even if it gets removed, it could still be counted as a partial win defending against unwanted AI scraping.

As with most things like this, it is a cat and mouse game. On the one hand, I am annoyed, because I am personally rather firmly on the side of 'why are we spending time trying to prevent people doing this somewhat cool thing?', but at the same time, just like with drms, copy restrictions and all that idiocy, it raises a new line of kids with something to rebel against. So I guess it serves a purpose. On a third hand, can you imagine those minds being able to focus on something else?

I dunno about this one, but I remember the previous versions suffered from visible artifacts to the point most artists elected not to use them as they made the output look bad.

It's also not obvious to me what happens with cartoon style art. Something that looks like white noise might be acceptable on an oil painting but not something with flat colors and clean lines.