how and why machines are getting into art

Roughly a week ago, the internet once again opened its maw of madness and showed me what happens when someone asks a computer what would happen if the Minions were designed not by a French studio aiming to create annoyingly cute cash cows but were instead spawned from the mind of H.R. Giger. Note the emphasis on asking a computer because the nightmarish little collage in the link was created not by a talented human imitating the famous gothic artist, but an artificial intelligence model called Midjourney.

Hold on a minute, you might say, automation isn’t just taking over factory jobs, mid-level office work, making entire careers obsolete, and radically restructuring our economy and society, but is now going after something as uniquely human as art? Yes, that’s right. Even human models aren’t safe from the things we can now do with massive artificial neural networks running on vast server farms across the world. Just a few high-quality reference pictures can be virtually stitched into a photorealistic 3D model which can now be used for just about anything.

Armed with more processing power than ever, computers are even taking on animation, using keyframes and detailed character designs to generate motion based on detailed parameters to control expression and degrees of movement. Even writers aren’t safe as creatives struggling to keep up with schedules imposed by the new economics of creative fiction imposed by Amazon and Wattpad are turning to AI platforms able to generate entire passages in the blink of an eye given enough information about mood and characters.

how is all this creative a.i. possible?

But wait, you might say, artificial intelligence is just bits and bytes and statistics. How do text prompts or images go in and art comes out? Is there something happening beyond just the numbers? Not at all. It’s just that instead of having humans constantly correcting a carefully curated library of inputs and outputs, we’re letting computers train each other and upped the number of possible inputs and outputs. Able to both grab and spit out more information, the models are now able to produce more, and more complex iterations of what we requested.

Here’s how it works. Rather than simply training one neural network until its output satisfies a human, we pit two models against each other. The first model, the generator, produces a rapid-fire stream of images or text. The second model, the discriminator, armed with a library of high-quality reference material, grades how well the first model does and rejects anything deviating too far from the real thing. This design is called Generative Adversarial Networks, or GAN, and its success and widespread adoption is what’s fueling these new creative platforms.

GAN is not a new idea on the scene, but with only 100-pixel wide squares or a paragraph to transform, there wasn’t much they could do. Today’s networks with billions of input neurons, however, allow them to digest ultrahigh-resolution images and entire books in one gulp. On top of that, access to cheaper processing power across millions of linked servers means they have billions of chances to turn terabytes of data into something really interesting, which, thanks to the laws of math, they’re eventually bound to do.

why is use of artistic software exploding?

All right, so now you know not only that computers are making art but how they’re doing it, which brings us to the question of why exactly we’re trying to automate creative pursuits. Wouldn’t we want to keep art intrinsically human? Well, one would think so, and you could argue that computer-generated art will never have the deliberateness of a human who knows what they’re drawing and why, not just moving data around until a piece of software approves based on whether that data is close enough to something similar created by humans.

But at the same time, consider what’s being automated. Illustrations for quick news pieces. In-between frames for animation. Passages meant to get overwhelmed and tired writers on tight deadlines past their writer’s block. Modeling shots reused to show off different clothes, so an army of photographers doesn’t have to do constant reshoots as fast fashion styles erupt from factories. Commercial visuals for a world hungry for an endless stream of novelty, rather than unique pieces meant to showcase talent or make a statement.

We also don’t pay very much for the vast majority of such art. Pennies for stock photos and illustrations. A few dollars per web novel chapter. And animators and VFX artists are almost always poorly paid and overworked. Why wouldn’t we, and they, turn to automating as much tedious, cheap technical work as possible, just like we already do with just about everything else? We like to create, but we hate when creating is a slog, so it makes sense that our urge to free ourselves from busywork would extend to every aspect of our lives, including art.