AI-generated child sexual abuse images could flood the internet. A watchdog is calling for action : technology

[–] artaxadepressedhorse@lemmyngs.social 40 points 11 months ago* (last edited 11 months ago) (4 children)

I am sort of curious, bc I don't know: of all the types of sexual abuse that happens to children, ie being molested by family or acquaintances, being kidnapped by the creep in the van, being trafficked for prostitution, abuse in church, etc etc... in comparison to these cases, how many cases deal exclusively with producing imagery?

Next thing I'm curious about: if the internet becomes flooded with AI generated CP images, could that potentially reduce the demand for RL imagery? Wouldn't the demand-side be met? Is the concern normalization and inducing demand? Do we know there's any significant correlation between more people looking and more people actually abusing kids?

Which leads to the next part: I play violent video games and listen to violent aggressive music and have for many years now and I enjoy it a lot, and I've never done violence to anybody before, nor would I want to. Is persecuting someone for imagining/mentally roleplaying something that's cruel actually a form of social abuse in itself?

Props to anybody who asks hard questions btw, bc guaranteed there will be a lot of bullying on this topic. I'm not saying "I'm right and they're wrong", but there's a lot of nuance here and people here seem pretty quick to hand govt and police incredible powers for.. I dunno.. how much gain really? You'll never get rights back that you throw away. Never. They don't make 'em anymore these days.

[+] raccoona_nongrata@beehaw.org 26 points 11 months ago (2 children)

[deleted]

[–] artaxadepressedhorse@lemmyngs.social 10 points 11 months ago (1 children)

How often does tracking child abuse imagery lead to preventing actual child abuse? Out of all the children who are abused each year, what percentage of their abusers are tracked via online imagery? Aren't a lot of these cases IRL/situationally based? That's what I'm trying to determine here. Is this even a good use of public resources and/or focus?

As for how you personally feel about the imagery, I believe that a lot of things humans do are gross, but I don't believe we should be arbitrarily creating laws to restrict things that others do that I find appalling.. unless there's a very good reason to. It's extremely dangerous to go flying too fast down that road, esp with anything related to "terror/security" or "for the children" we need to be especially careful. We don't need another case of "Well in hindsight, that [war on whatever] was a terrible idea and hurt lots and lots of people"

And let's be absolutely clear here: I 100% believe that people abusing children is fucked up, and the fact that I even need to add this disclaimer here should be a red flag about the dangers of how this issue is structured.

[+] raccoona_nongrata@beehaw.org 6 points 11 months ago* (last edited 11 months ago) (3 children)

[deleted]

[–] artaxadepressedhorse@lemmyngs.social 5 points 10 months ago

I appreciate you posting the link to my question, but that's an article written from the perspective of law enforcement. They're an authority, so they're incentivized to manipulate facts and deceive to gain more authority. Sorry if I don't trust law enforcement but they've proven themselves untrustworthy at this point

[–] CorruptBuddha@lemmy.dbzer0.com 5 points 10 months ago (1 children)

Okay... So correct me if I'm wrong, but being abused as a child is like... one of the biggest predictors of becoming a pedophile. So like... Should we preemptively go after these people? You know... To protect the kids?

How about single parents that expose their kids to strangers when dating. That's a massive vector for kids to be exposed to child abuse.

[–] PelicanPersuader@beehaw.org 4 points 11 months ago

It already is outlawed in the US. The US bans all depictions precisely because of this. The courts anticipated that there would come a time when people could create images which are indistinguishable from reality so allowing any content to be produced wasn't permissible.

[–] Nollij@sopuli.xyz 7 points 11 months ago (1 children)

Of all the problems and challenges with this idea, this is probably the easiest to solve technologically. If we assume that AI-generated material is given the ok to be produced, the AI generators would need to (and easily can, and arguably already should) embed a watermark (visible or not) or digital signature. This would prevent actual photos from being presented as AI. It may be possible to remove these markers, but the reasons to do so are very limited in this scenario.

[+] raccoona_nongrata@beehaw.org 8 points 11 months ago (1 children)

[deleted]

[–] Nollij@sopuli.xyz 4 points 11 months ago (1 children)

I was actually specifically avoiding all of those concerns in my reply. They're valid, and others are discussing them on this thread, just not what my reply was about.

I was exclusively talking about how to identify if an image was generated by AI or was a real photo.

[–] abhibeckert@beehaw.org 3 points 11 months ago* (last edited 11 months ago) (1 children)

I was exclusively talking about how to identify if an image was generated by AI or was a real photo.

These images are being created with open source / free models. Whatever watermark feature the open source code has will simply be removed by the criminal.

Watermarking is like a lock on a door. Keeps honest people honest... which is useful, but it's not going to stop any real criminals.

[–] evranch@lemmy.ca 6 points 11 months ago (1 children)

In this specific scenario, you wouldn't want to remove the watermark.

The watermark would be the only thing that defines the content as "harmless" AI-generated content, which for the sake of discussion is being presented as legal. Remove the watermark, and as far as the law knows, you're in possession of real CSAM and you're on the way to prison.

The real concern would be adding the watermark to the real thing, to let it slip through the cracks. However, not only would this be computationally expensive if it was properly implemented, but I would assume the goal in marketing the real thing could only be to sell it to the worst of the worst, people who get off on the fact that children were abused to create it. And in that case, if AI is indistinguishable from the real thing, how do you sell criminal content if everyone thinks it's fake?

Anyways, I agree with other commenters that this entire can of worms should be left tightly shut. We don't need to encourage pedophilia in any way. "Regular" porn has experienced selection pressure to the point where taboo is now mainstream. We don't need to create a new market for bored porn viewers looking for something shocking.

[–] abhibeckert@beehaw.org 3 points 10 months ago* (last edited 10 months ago) (1 children)

The real concern would be adding the watermark to the real thing, to let it slip through the cracks. However, not only would this be computationally expensive if it was properly implemented,

It wouldn't be expensive, you could do it on a laptop in a few seconds.

Unless, of course, we decide only large corporations should be allowed to generate images and completely outlaw all of the open source / free image generation software - that's not going to happen.

Most images are created with a "diffusion" model where you take an image, and run an algorithm that slightly modifies it. Over and over and over until you get what you want. You don't have to (and commonly don't - for the best results) start with a blank image. And you can run just a single pass, with the output being almost indistinguishable from the input.

This is a hard problem to solve and I think catching abuse after it happens is increasingly going to be more difficult. Better to focus on stopping the abuse from happening in the first place. E.g. by flagging and investigating questionable behaviour by kids in schools. That approach is proven and works well.

[–] evranch@lemmy.ca 2 points 10 months ago

The image generation can be cheap, but I was imagining this sort of watermark wouldn't be so much a visible part of the image, but an embedded signature that hashes the image.

Require enough PoW to generate the signature, and this would at least cut down the volumes of images created, and possibly limit them to groups or businesses with clusters that could be monitored, without clamping down on image generation in general.

A modified version of what you mentioned could work too, but where just these specific images have to be vetted and signed by a central authority using a private key. Image generation software wouldn't be restricted for general purposes, but no signature on suspicious content and it's off to jail.

[–] ConsciousCode@beehaw.org 4 points 11 months ago

I respect your boldness to ask these questions, but I don't feel like I can adequately answer them. I wrote a 6 paragraph essay but using GPT-4 as a sensitivity reader, I don't think I can post it without some kind of miscommunication or unintentional hurt. Instead, I'll answer the questions directly by presenting non-authoritative alternate viewpoints.

No idea, maybe someone else knows
That makes sense to me; I would think there would be a strong pressure to present fake content as real to avoid getting caught but they're already in deep legal trouble anyway and I'm sure they get off to it too. It's hard to know for sure because it's so stigmatized that the data are both biased and sparse. Good luck getting anyone to volunteer that information
I consider pedophilia (ie the attraction) to be amoral but acting on it to be "evil", ala noncon, gore, necrophilia, etc. That's just from consistent application of my principles though, as I haven't humanized them enough to care that pedophilia itself is illegal. I don't think violent video games are quite comparable because humans normally abhor violence, so there's a degree of separation, whereas CP is inherently attractive to them. More research is needed, if we as a society care enough to research it.
I don't quite agree, rights are hard-won and easy-lost but we seem to gain them over time. Take trans rights to healthcare for example - first it wasn't available to anyone, then it was available to everyone (trans or not), now we have reactionary denials of those rights, and soon we'll get those rights for real, like what happened with gay rights. Also, I don't see what rights are lost in arguing for the status quo that pedophilia remain criminalized? If MAPs are any indication, I'm not sure we're ready for that tightrope, and there are at least a dozen marginalized groups I'd rather see get rights first. Unlike gay people for instance, being "in the closet" is a net societal good because there's no valid way to present that publicly without harming children or eroding their protections.

load more comments (2 replies)

[–] Draedron@lemmy.dbzer0.com 26 points 11 months ago (4 children)

Isnt it better the are AI generated than real? Pedophiles exist and wont go away and no one can control it. So best they watch AI images than real ones or worse

[–] python@programming.dev 14 points 11 months ago (1 children)

AI needs training data.

[–] skullgiver@popplesburger.hilciferous.nl 35 points 11 months ago* (last edited 9 months ago) (1 children)

[This comment has been deleted by an automated system]

[–] barsoap@lemm.ee 2 points 11 months ago (4 children)

Images, yes, but mixing concepts is a mixed bag. Just because the model can draw, say, human faces and dog faces doesn't mean it has the understanding necessary to blend those concepts. Without employing specialised models (and yes of course the furries have been busy) the best you'll get is facepaint. The pope at a beach bar doesn't even come close to exercising that kind of capability: The pope is still the pope and the beach bar is still the beach bar, and a person is still sitting there slurping a caipirinha.

[–] amju_wolf@pawb.social 10 points 11 months ago (1 children)

I mean if you train a model on porn with adult actors and on regular photos with children, it shouldn't be hard to generate the combination.

You probably wouldn't even need any fancy training data but if you really wanted you could pick adult actors that look young or in other ways similar to the children to help the process.

[–] barsoap@lemm.ee 3 points 11 months ago (5 children)

Knowing what a nude adult looks like doesn't mean that the model knows what a nude child looks like. I'm quite sure it's easy to generate disturbing images like that, but actual paedophiles I think won't be satisfied with child faces on small adult bodies.

Ordinary deepfakes actually have a very similar problem: Sure you can take a picture of a celebrity and tell the AI to undress them -- but it won't be their actual body. The AI is going to be able to approximate their overall build but it's going to be a generic adult body, not the celebrity's body. Or, differently put, AI models aren't any better at undressing people with their eyes than teenagers.

[–] amju_wolf@pawb.social 3 points 11 months ago (1 children)

I see where you're coming from but that's a technical issue that will probably be solved in time.

It's also really not a black and white; sure maybe you can see it isn't perfect but you'd still prefer it to content where you know no one was actually harmed.

Despite what reputation people like that have (due to the simple fact of how reporting works), most are harmless like me and you and don't actually want to see innocent people suffer and would never act on their desires. So having a safe and harmless outlet might help.

[–] barsoap@lemm.ee 3 points 10 months ago* (last edited 10 months ago) (1 children)

I see where you’re coming from but that’s a technical issue that will probably be solved in time.

You cannot create information from nothing.

So having a safe and harmless outlet might help.

Psychologists/Psychiatrists are still on the fence on that one, I wouldn't be surprised if it depends on the person. And yes the external harm produced by AI images is definitely lower than that produced from actual CSAM, doubly so newly produced CSAM, but that doesn't mean that therapy, even in its current early stages, couldn't do even better.

Differently put: We may be again falling into the trap of trying to find technological solutions to societal problems (well, this is /c/technology...). Which isn't to say that we shouldn't care at all about models trained on CSAM, but that's addressing symptoms, not causes. Ultimately addressing root causes is more important: The vast majority of paedophiles are not exclusive paedophiles, often they're not even really attracted to kids at all beyond having developed a fetish, they're rapists focussing on the most vulnerable, often due to having been victims of sexual abuse themselves.

[–] amju_wolf@pawb.social 2 points 10 months ago (1 children)

You cannot create information from nothing.

Arguably that's exactly what generative AIs do. Which is not what you meant, but yeah. I was going more for like "given current progress and advancements in how we curate datasets and whatnot, there is no reason to believe that we won't have 100% undistinguishable AI-generated pictures eventually".

We already know that you don't need to have stuff in the training dataset to have it show up meaningfully in the output.

Psychologists/Psychiatrists are still on the fence on that one, I wouldn’t be surprised if it depends on the person. And yes the external harm produced by AI images is definitely lower than that produced from actual CSAM, doubly so newly produced CSAM, but that doesn’t mean that therapy, even in its current early stages, couldn’t do even better.

100% agree there. What I would like to see is more research, but that's currently kinda impossible with CSAM being as criminalized as it is. Which is kinda sad.

Therapy seems to work on most help-seeking people (and there are studies proving that), so this should be a last ditch effort.

The rest of your post I don't agree with. It isn't really (definitely not exclusively) a societal problem - some people's brains are simply wired in a way that's just bad and there isn't much you can do with it, and either these people suffer by living with it, or they cause harm to others because of it. Both is bad.

The vast majority of paedophiles are not exclusive paedophiles, often they’re not even really attracted to kids at all beyond having developed a fetish, they’re rapists focussing on the most vulnerable, often due to having been victims of sexual abuse themselves.

Do you have any statistics proving this? It's exactly the bias that already makes non-acting pedophiles unlikely to seek help. Obviously these kinds of people are the ones you hear most about, but I wouldn't be so sure that they're the majority (even if they're most of the problem).

My point is that if you take it as people who need help and actually manage to provide it, you should be able to get the number of abuse down overall except for the people who truly can't be helped. And it really doesn't matter much how you provide that help, even if it's morally questionable like using artificially generated CSAM.

load more comments (1 replies)

load more comments (4 replies)

load more comments (2 replies)

[–] coffeejunky@beehaw.org 12 points 11 months ago (3 children)

Yeah exactly, I don't want to see it but the same goes for a lot of weird fetishes.

As long as no one is getting hurt I don't really see the problem.

[–] BarryZuckerkorn@beehaw.org 15 points 11 months ago (1 children)

As long as no one is getting hurt I don’t really see the problem.

It'd be hard to actually meet that premise, though. People are getting hurt.

Child abuse imagery is used as both a currency within those circles to incentivize additional distribution, which means there is a demand for ongoing and new actual abuse of victims. Extending that financial/economic analogy, seeding that economy with liquidity, in a financial sense, might or might not incentivize the creation of new authentic child abuse imagery (that requires a child victim to create). That's not as clear, but what is clear is that it would reduce the transaction costs of distributing existing child abuse imagery, which is a form of re-victimizing those who have already been abused.

Child abuse imagery is also used as a grooming technique. Normalization of child sexual activity is how a lot of abusers persuade children to engage in sexual acts. Providing victimless "seed" material might still result in actual abuse happening down the line.

If the creation of AI-generated child abuse imagery begins to give actual abusers and users of real child abuse imagery cover, to where it becomes more difficult to investigate the crime or secure convictions against child rapists, then the proliferation of this technology would make it easier to victimize additional children without consequences.

I'm not sure what the latest research is on the extent to which viewing and consuming child porn would lead to harmful behavior down the line (on the one hand, maybe it's a less harmless outlet for unhealthy urges, but on the other hand, it may feed an addictive cycle that results in net additional harm to society).

I'm sure there are a lot of other considerations and social forces at play, too.

[–] amju_wolf@pawb.social 10 points 11 months ago (1 children)

I mean you could also go with a more sane model that still represses the idea while allowing some controlled environment for people whom it can really help.

You could start by not prosecuting posession, only distribution. So it would still be effectively "blocked" everywhere like it's (attempted to be) now, but distributing models for generation would be fine.

Or you could create "known safe" (AI generated) 'datasets' to distribute to people, while knowing it was ethically created.

is used as both a currency within those circles to incentivize additional distribution, which means there is a demand for ongoing and new actual abuse of victims

A huge part of the idea is that if you create a surplus of supply it cannot work as a currency and actual abuse material will be drowned out and not wort it to create for the vast majority of people - too risky and irrelevant if you have a good enough alternative.

You're definitely right though that there would have to be more considerations.

[–] ParsnipWitch@feddit.de 7 points 11 months ago (1 children)

You seem to think it's some kind of human right and people are entitled to have fapping material provided for them. No one is hurt if people don't have fapping material.

[–] amju_wolf@pawb.social 6 points 11 months ago* (last edited 11 months ago) (3 children)

There is an argument to be made that allowing people with unhealthy desires a safe and harmless outlet, they will be less compelled to go with the harmful option.

And, actually, I kinda want to disagree with the premise too. Even if it was provably true that noone gets hurt if there wasn't porn, you can flip the question; why should it be banned if it doesn't hurt anyone? Do you want to live in a world where anything that's perceived as bad is just outright banned without much thought?

load more comments (3 replies)

[+] trevron@beehaw.org 4 points 11 months ago (3 children)

[deleted]

[–] fox_the_apprentice@lemmynsfw.com 9 points 11 months ago* (last edited 11 months ago) (7 children)

A lot of the comments in here seem a little bit too sympathetic

It is a mental illness. If fake images result in less real-world abuse then that's a good thing.

load more comments (7 replies)

[–] amju_wolf@pawb.social 6 points 11 months ago (1 children)

A "weird fetish" is, quite literally a paraphilia, just like pedophilia. We only care about the latter because it has the potential to hurt people if acted upon. There's no difference, medically speaking.

A lot of the comments in here seem a little bit too sympathetic.

When you want to solve an issue you need to understand the people having it and have some compassion, which tends to include stuff like defending people who didn't actually do anything harmful from being grouped with the kind who do act on their urges.

[–] artaxadepressedhorse@lemmyngs.social 3 points 10 months ago

Humans also tend to possess an abusive tendency, where, once they can justify labeling somebody as "bad" they can justify being cruel to them. I see people doing it all the time.

[–] CorruptBuddha@lemmy.dbzer0.com 2 points 10 months ago

It doesn't seem right to me to prosecute someone for computer generated images.

[–] hh93@lemm.ee 2 points 11 months ago

I guess it depends on what pedophilia is in the end of how it's developed.

If it's more like a sexual preference then it's probably there already when someone is born and not changeable, but if it's more like a fetish then those are (afaik) related to experiences and exposures while growing up and actually can change and develop over time - and in that case it could be really dangerous to have that kind of material floating around freely.

load more comments (1 replies)

[–] tesseract@beehaw.org 14 points 11 months ago

Now that CSAM content is generated by bigcos with deep pockets, politicians don't want to scan their servers or take any other action. These are the same demagogues who wanted to kill end-to-end encryption and scan ordinary people's devices in the name of CSAM. Greedy and hypocritical vermin.

[–] NatoBoram@lemm.ee 5 points 11 months ago (1 children)

It's already happening on Pixiv...

[–] SmoochyPit@beehaw.org 4 points 11 months ago (1 children)

I can’t believe how hard it is to avoid drawn or generated cp on there— and you can only ignore one tag without premium, so it’s not viable to manually make a blocklist :(

[–] NatoBoram@lemm.ee 2 points 9 months ago

That only-one-ignore-without-premium thing is really asshole design, though

[–] autotldr@lemmings.world 2 points 11 months ago

🤖 I'm a bot that provides automatic summaries for articles:

Click here to see the summary

NEW YORK (AP) — The already-alarming proliferation of child sexual abuse images on the internet could become much worse if something is not done to put controls on artificial intelligence tools that generate deepfake photos, a watchdog agency warned on Tuesday.

In a written report, the U.K.-based Internet Watch Foundation urges governments and technology providers to act quickly before a flood of AI-generated images of child sexual abuse overwhelms law enforcement investigators and vastly expands the pool of potential victims.

In a first-of-its-kind case in South Korea, a man was sentenced in September to 2 1/2 years in prison for using artificial intelligence to create 360 virtual child abuse images, according to the Busan District Court in the country’s southeast.

What IWF analysts found were abusers sharing tips and marveling about how easy it was to turn their home computers into factories for generating sexually explicit images of children of all ages.

While the IWF’s report is meant to flag a growing problem more than offer prescriptions, it urges governments to strengthen laws to make it easier to combat AI-generated abuse.

Users can still access unfiltered older versions of Stable Diffusion, however, which are “overwhelmingly the software of choice ... for people creating explicit content involving children,” said David Thiel, chief technologist of the Stanford Internet Observatory, another watchdog group studying the problem.

Saved 78% of original text.

Technology