Technically Sweet

When you see something that is technically sweet, you go ahead and do it and you argue about what to do about it only after you have had your technical success. That is the way it was with the atomic bomb.
— J. Robert Oppenheimer

I am not the first to link this Oppenheimer quote to recent developments in AI but it seems quite apt. It is striking how quickly this era of generative AI has come about. The landmark paper presenting the theoretical architecture (Attention Is All You Need) behind large language models (i.e. ChatGPT and its relatives) was published in 2017. ChatGPT itself was released in November 2022, scaling up in complexity from the prototype model presented in the Attention paper by a factor of about 800.

The arrival of generative AI for images and video is another case of rapid evolution, well presented in a Stephen Welch YouTube video on the theory behind these technologies. Today there are numerous systems for generating video from text descriptions, but it took several mathematical breakthroughs in the past five years to get to these. For instance in February 2021 research was published describing a training method for placing images and their text descriptions in the same high-dimensional numerical space, but that was just the initial step in image generation, let alone video creation.

But the model built for the 2021 research was trained on 400 million pairs of images with corresponding text, scraped from we don’t know where. This week one of the big AI companies, Anthropic, settled out of court a major copyright class action lawsuit concerning the company’s use of millions of pirated books. Also this week, a wrongful death lawsuit was filed against the company OpenAI detailing how ChatGPT coached a teenager in committing suicide. Meanwhile, it has become clear that large language model-driven systems have security flaws that one can drive proverbial trucks through. And it has become incredibly easy to use text-to-image AI systems to create fake photographs for propaganda purposes. Pursuing the technically sweet has gotten well ahead of ethics. Again.