AI Sculpting

A co-creation process with artificial intelligence

What happens when AI becomes the creator? How can human creativity and machine-made output feed each other? Curious about co-creating with AI, we initiated an AI sculpting process in 2020. The human interest in manipulation of objects for an aesthetic purpose is very antique. By experimenting with different tools and techniques the old masters studied how material can be shaped into form. Transferring this curiosity to the age of technology, we engineered a machine-learning process — reinforcement learning to be precise — to observe the evolution of an AI learning how to sculpt a 3D model. To best experience AI as the creator and to learn from it became our main pursuit. In the course of the process we were equally surprised by the strategies and the output the AI developed. Inspired by the process and the visual variety, we curated some of the examples and built an aesthetic environment in 3D to orchestrate them.

The Concept: What happens when AI becomes the creator?

When it comes to studying a basic artisan technique humans are mostly motivated by curiosity, while the machine is driven by efficiency — an interesting tension we found worth exploring. With its underlying setup, the machine-learning process is related to the studies ancient sculptors undertook in the very beginning. Reminiscent of the helpers assisting them in the old workshops, the AI developed different strategies seeking constant improvement on its way to form a given object. By feeding it with different tools, rules and rewards through reinforcement learning, we were steering the process, but not the outcome. The core aspect of this co-creation process is that we — to an extent — let go of control. We were seeking an equilibrium of setting a frame, giving instructions and observing the outcome: a simple cube being transformed into an increasingly recognizable shape with every iteration.

The Training: Reinforcement Learning

The agent, which is defined as a certain machine-learning model, learned to be seeking maximum reward with the given goal to sculpt a 3D model. In the voxel-based environment infinite data and a clear reward structure were provided for the agent to move through. The starting state of the environment was one big cube. Out of this, the agent needed to remove mass to get closer to a predefined target state. With each step, the agent could decide where to go and if to remove a mass of voxels around itself and how. To enable its learning, it was rewarded in a specific way: when extraneous mass was removed it was rewarded and it was given a penalty when mass that ought to be part of the final sculpture was removed. All the technical ML parts were implemented in Unity3D and Unity’s ml-agents library was used for implementing the reinforcement learning.

The Curation: Artificial Intelligence & Human Creativity

By studying the visual results along the process, our role as a designer became one of a curator: we selected a number of outcomes, which we defined as sculptures to present them as digital artworks. This is where human creativity came to play again: what do we consider as a presentable manipulation of the cube? How do we orchestrate the machine-made output? Which sculptures do we choose and what kind of environment do we create in order to exhibit them? In a scenario like this, we collaborate with AI. The outcome is not a reproduction of our creativity, it is unforeseen. We receive unprecedented results, which re-inspire our way of creating, highlighting the importance of exploring the concept of co-creating with AI.

The Conclusion: Always Learning, Always Evolving

What have we learned by observing the machine learning process? With AI as a creator, we have not only gained new inspiration, but also new perspectives on the relationship between man and machine in digital art. Above all, we have also questioned our own role as creators. Rather than leaving creation to AI, we need to find ways to integrate it into the creative process. We take technology as a starting point, a tool, a source of inspiration and a creative partner. The human aspect is quite clear in this: We choose the rules and define the approximate output. However, in the end it was the interplay between our human choices and the agent’s ability to find the best solutions and occasionally surprise us. This made the process rewarding to us and shows the true potential of an AI based co-creation process.

By understanding how the machine works, we can reinterpret it, rethink it, and develop new strategies or systems to implement it, enriching our creativity and the creation process on many levels. In the age of artificial intelligence, creative concepts and ideas become more relevant again as we determine and actively shape the environment and process and thus the impact of this AI evolution.

Creative Direction: Cedric Kiefer
Production: Tobias Ziegler
ML / Reinforcement Learning: Mark Tensen
3D Artist: Alexander Hahn
Code & Design: Norman Wassmuth, Piotr Rymer, Bernd Marbach