No one’s ready for this

Sarah Jeong's recent article for The Verge notes how the trend of on-phone AI photo manipulation tools are not just risking a cavalcade of convincing fakes but also our trust in photography at general. She makes a convincing argument, and a somewhat terrifying one, that when we move our understanding of photos from assumed truth to assumed falsehood it means having to do difficult litigation over every image.

The next Abu Ghraib will be buried under a sea of AI-generated war crime snuff. The next George Floyd will go unnoticed and unvindicated. [...]

as photographs become little more than hallucinations made manifest, the dumbest shit will devolve into a courtroom battle over the reputation of the witnesses and the existence of corroborating evidence.

Other good conversations on this:

• Robin Rendle: No One's Ready for This

this stuff right here—adding things that never happened to a picture—that’s immoral because confusion and deception is the point of this product. There are only shady applications for it. Looking at a lot of the examples here I can’t tell what’s real without inspecting them—the crashed motorcycle has a bicycle tire for example but man I would never look this closely in most situations.

So right now I think this stuff should be straight up illegal.

• Ben Werdmuller: No One's Read for This

My take is that the cat is out of the bag. The societal implications aren't good - at all - but I don't think banning the technology is practical. So, instead, we have to find a way to live with it.

I do think we are leaping headfirst into a full on crisis of ground truth through AI photo manipulation. Sadly I think both Robin and Ben are correct: this should be illegal, but the cat is out of the bag. Jeong's conclusion makes this clear as well.

We briefly lived in an era in which the photograph was a shortcut to reality, to knowing things, to having a smoking gun. It was an extraordinarily useful tool for navigating the world around us. We are now leaping headfirst into a future in which reality is simply less knowable. The lost Library of Alexandria could have fit onto the microSD card in my Nintendo Switch, and yet the cutting edge of technology is a handheld telephone that spews lies as a fun little bonus feature.

We are fucked.