For more on Kate Crawford and Trevor Paglen's research checkout their online essay excavating.ai on "the politics of images in machine learning training sets". For a critical response to Refik Anadol's MoMA piece "Unsurpervised" checkout this review from e-flux.
Making Art with AI
(high-level to low-level)
- AI as muse: Google experimented with Language Models and writers in this "Digital Muse" project. That was 3 years ago, the language models we have today are already orders of magnitude larger (in size/scope/etc), but the principle is still the same: use AI as part of your creative process (brainstorming, concepting, etc). The tools (apps) we build around these AI models might encourage this way of working, for example Google's TextFX made in collaboration with the rapper Lupe Fiasco, or might not (ex: ChatGPT). The same can be said for image generating models like Dall-E, Stable Diffusion, midjourney, etc as well as music tools like Jukebox and the various Magenta projects.
- AI as studio assistant: the quality of writing/sounds/images that these models (like the Dall-E, ChatGPT, etc) are starting to produce are good enough these days to make it to the final product (edited or even readymade), we've been seeing a lot of that form "AI artists" using prompt-based systems to create NFTs for example, but we’re also starting to see new AI-powered studio tools (think Adobe apps), like NVIDIA Canvas, RunwayML, EbSynth, descript and so many others starting to come out.
- AI powered art: new media artists and creative technologists are starting to make work which incorporates pre-existing AI models, like this playful mashup of GPT-4's vision model and ElevenLab's voice models made by Charlie Holtz or this dance project by Bill T. Jones in collaboration with Google is maybe a less obvious example.
-
AI as an artwork: some artists have been training their own modesl, like Holly Herndon who trained an AI of ver voice called Holly+ (she’s got a great TED talk/performance on it) or Refik Anadol's image generator trained on the MoMA's collection.
This might sound hard at first, but there's actually a few ways to do this:
- You can take an existing model (pre-trained AI) and build on top of it. You can learn the basics here: https://teachablemachine.withgoogle.com I think this is also a very accessible demo we can use to better understand how these models are “trained”, what their limits are, how bias enters the picture, etc. Also check out the ml5.js library and the corresponding tutorial series A Beginner's Guide to Machine Learning in JavaScript from the Coding Train.
- You can gather your own data (or use publicly available datasets like those found on kaggle.com or elsewhere) and use existing neural network architecture to train your own models from scratch. This typically takes a lot of data and processing power (so it can be take a while to create), artist Gene Kogan has some great resources for this called Machine Learning for Artists.
- Or you could create your own architecture and train your own models “from scratch”, or rather, using now industry standard Machine Learning libraries like Tensor Flow (see Machine Learning for Web Devs and Creatives by Google)
To experiment with this sort of AI right now, check out this netnet sketch which uses an AI "pose detection" model to track your position via your webcam and places an animated Gif over your face, or this technically (in terms of the code) similar example, where rather than using pose detection to move a gif around the screen, we're using it to control a digital theramin made with Tone.js.
Making Art about AI
The coded gaze reflects the priorities, preferences and prejudices of those who have the power to shape technology
Joy Buolamwini
AI isn't just changing our field, it's reshaping so many different aspects of our lives. I also think it's important to engage with some of these technologies not necessarily so to AI-art (or AI-aided art) but work in any medium about this present moment.
One example is Kate Crawford and Vladan Joler's 2018 piece Anatomy of an AI System (seen above), a detailed diagram, with an accompanying essay, of the human labor, data and planetary resources used in Amazon Echo's AI system. The work itself is a PDF (often printed in large format for exhibitions) with an accompanying essay, which is to say that it is not "digital art" (in terms of the medium), but the work is clearly informed by the artists's deep research into the subject matter of AI.
Another example might be Tom White’s The Treachery of ImageNet (seen above), a reference to Magritte’s “this is not a pipe” painting, these prints clearly don’t look like the thing they are not to us (humans) but if you point an AI image classifier at it, it will think it is the thing that it is not. White’s work is an interesting conceptual response to this tech, but there is also loads of political work in this area as well. For example, artists who've worked on "datasets" as art, these aren't necessarily meant to be functional ML training datasets (like the kind you find on kaggle.com) but rather are works aimed at invoking discussions around the role curated data plays in this new world:
Black Health by Bomani Oseni McClendon
Feminist Data Set by Carolin Sinders
The Library of Missing Datasets by Mimi Ọnụọha
Of course, a fundamental understanding of how this tech works and more importantly, whose in control of this tech and how is it being used in the world as well as what the implications are, are prerequisites for making meaningful work in this regard. You can find more of my notes on Bias in AI here.