This series is intended as a guide through the field of computational creativity for practitioners and executives in the creative industries. Over four installments, I’m going to provide an overview of the current state-of-the-AI in different creative domains, outline challenges, and present realistic near- to mid-term scenarios for business application.

For a couple of years now, artificial intelligence and machine learning have been among the biggest trends in technology. Across many industries, people are worried that rising machines are going to replace humans. Naturally, this also applies to the creative sector. These days, AIs make music, paint pictures, develop games, create ads, write fiction, non-fiction, movie scripts and produce trailers. That raises questions. How are algorithms going to shape the creative industry? Is the next generation of pop cultural icons going to consist of AIs?

Imagine if music labels wouldn’t have to deal with eccentric artists but could ask a software to produce the next summer hit. Imagine if a book publisher could just ask an algorithm to write the next bestseller instead of having to sort through thousands upon thousands of manuscripts. Imagine a games publisher whose AI just spits out game after game, a newsroom without journalists but full of powerful computers, or a movie studio that produces animation flicks by pressing a button.

On first sight, it would make a lot of sense — at least from a pure business standpoint. The biggest cost factor in all creative industries is creation. Replacing the people it takes to create a creative good with powerful computers that achieve the same results sure sounds like a good business model: You minimize the high marginal costs that are usually associated with creative industries, replace them with fixed, depreciable cost and send your algorithmic creators to work.

But is that a realistic scenario?

Series Outline

To answer the question above, we need to answer a few other ones first:

• What is the current state-of-the-art of creative computing in different domains?
• How close are computers to achieving the same result as human creators?
• What are the conceptual problems that computational creativity faces?

So, this is precisely what I’m going to look at in this series. Of course, there isn’t a one-size-fits-all answer to these questions across all creative domains. Hence, I’ll look at different creative domains/industries. Namely:

1. Music
2. Writing (creative & journalism)
3. Painting & Fine Arts
4. Advertising
5. Video & Movies
6. Games

I’ll give you an overview of the current state of creative AI in those domains and point out industry-specific challenges and limitations. Also, I’ll present you the artificial intelligence scenarios which I deem most likely in the respective industries short-to-mid-term.

And — as I’m a firm believer in the idea that it takes at least a foundational understanding of how a given technology works before one can judge its potential in a business context — we’ll also take a look at some of the most important techniques that creative algorithms use.

All told, this series has the following structure:

Part I: General Introduction to Computational Creativity

Part II: An Overview of the Current State-of-the-AI in Six Creative Domains

Part III: Conceptual Problems, Challenges & Limitations of Creative AI

Part IV: Realistic AI Scenarios in Six Creative Domains

A technical note: In order to avoid the overly extensive use of links in the text, I collected (the online part of) my sources on Refind. Of course, I’m going to link to the most important material when referencing it. I’ll also provide a further reading list at the end of the series.

General Introduction to Computational Creativity

As my aim with this series is to evaluate the technology from the practical perspective of creative businesses, I’ll keep theoretical, philosophical and deep technological issues to a minimum. However, they can’t be avoided entirely because they matter (or should matter) to practitioners when it comes to evaluating the technology’s potential for practical application.

Therefore, let’s start with an examination of what computational creativity is. The term refers to a field of research as well as to the application of algorithmically enabled technology to create artifacts. In simplified terms, we are talking about software and robots that create art. But a somewhat more precise definition would be helpful. The Association for Computational Creativity, a non-profit organization and research platform, defines it like this:

Computational creativity is a multidisciplinary endeavour that is located at the intersection of the fields of artificial intelligence, cognitive psychology, philosophy, and the arts.

The goal of computational creativity is to model, simulate or replicate creativity using a computer, to achieve one of several ends:

– to construct a program or computer capable of human-level creativity

– to better understand human creativity and to formulate an algorithmic perspective on creative behavior in humans

– to design programs that can enhance human creativity without necessarily being creative themselves

While there are several problems with that definition, as Anna Jordanous pointed out, it’s still helpful; it illustrates that the field ranges from rather narrow applications to the idea of building autonomous creative agents.

It is, basically, the same segmentation that is known in the broad artificial intelligence field. There, people differentiate between strong and weak AI. The former is a (hypothetical) general-purpose, omniscient AI, the latter a narrowly focused application that uses artificial intelligence techniques like machine learning. With regards to computational creativity, a strong creative algorithm would be an autonomous artificial artist. A weak creative algorithm has a narrow purpose or use case.

The Creativity Problem

Little surprisingly, given that computational creativity is closely related to artificial intelligence, there is another similarity between the two. At the core of either discipline is the complicating factor that they try to model and replicate something that isn’t fully understood in its original form: thinking in the case of artificial intelligence and creativity in the case of computational creativity.

We don’t have a single, clear definition for them because we don’t fully understand either. In his very read-worthy article The Myth of a Superhuman AI, Kevin Kelly explains why it is so complicated to get the full picture of human (and, in extension, organic) thinking: What we refer to as thinking is a complex affair, not a singular act. He writes:

Human minds are societies of minds, in the words of Marvin Minsky. We run on ecosystems of thinking. We contain multiple species of cognition that do many types of thinking: deduction, induction, symbolic reasoning, emotional intelligence, spacial logic, short-term memory, and long-term memory. The entire nervous system in our gut is also a type of brain with its own mode of cognition. We don’t really think with just our brain; rather, we think with our whole bodies.

These suites of cognition vary between individuals and between species. A squirrel can remember the exact location of several thousand acorns for years, a feat that blows human minds away. So in that one type of cognition, squirrels exceed humans.

The very same thing applies to creativity (a concept closely intertwined with human intelligence, to be sure). It’s another term which humans intuitively grasp but that refers to a variety of acts and processes in different contexts. Inventing a new musical genre is certainly a creative act, but so is finding an unconventional way to settle a conflict between two colleagues at the office. Once you have to formalize creativity — as is the prerequisite to designing algorithms that model it— this poses a challenge (particularly when the goal is strong artificial creativity).

Typologies of Creativity

As a result, research into creativity has come up with several models to describe different kinds of creative behavior. For one, there is exploratory creativity. You are looking for a novel solution to a known problem by exploring an “established conceptual space”. It’s probably the least exciting but most common variety of creativity. It describes most incremental innovation in business and probably even many pop cultural artifacts: Once a couple of rappers and producers made trap songs (the modern dubstep-influenced trap with heavy auto-tune use), the concept of trap became generally understood and a kind of formula manifested itself. Every other rapper started making trap songs.

Another form of creativity is combinatorial creativity. It essentially describes the novel combination of existing concepts or ideas to create something “new”. The creation of modern trap from above fits that model. 808 drums, rap, auto-tune, dubstep basslines and all other ingredients were known but hadn’t been combined in the way trap did.

Since basically any innovation today is created on the foundation of human history and our accumulated knowledge, it’s likely fair to argue that any creativity has a combinatorial element to it. In human creativity, however, this process often happens unconsciously. It is simply a byproduct of our continuous experience and perception of the world around us.

Transformational creativity refers to a creative process that transforms or transcends the boundaries of the conceptual space. When Elon Musk thought about making space travel more affordable, then figured it might take reusable rockets, and went on to build them — which was formerly thought of as impossible —we witnessed an example of that type of creativity.

All of these models are useful descriptions of creative processes. None, however, grasps creativity in its entirety. They formalize the how of creativity. But they don’t consider the why — the motivation — of creativity. Also, they look at creativity as a means to solve a problem. In reality, however, many creative acts aren’t solution-oriented. Often, they are almost aimless exploration. Creativity, particularly in the arts, is often an act of personal expression, driven by emotion rather than reason.

Another issue all three models have in common: They focus on a singular creative act. A human creator, however, draws from the accumulated experiences s/he had during her lifetime. Me writing this piece might be a creative act of the exploratory nature. Yet, it is grounded on more than just a few days of research into computational creativity. It is influenced by 20 years of writing, by my interest in music and arts, by my decision to start attentionecono.me, by working as a digital business consultant for a decade and so forth.

I won’t go into further detail. For our purpose here — to evaluate the near-to-mid-term potential of AI in creative industries — it’s simply important to note: We have a limited understanding of human creativity. By definition that’s reflected in the design of creative algorithms. That isn’t necessarily a bad thing. It’s totally feasible that computer creativity turns out to be quite distinct from human creativity but still is very useful.

How Computers Become Creative

To conclude the first part of the series, let me give you a brief (and as non-technical as possible) overview of the most important types of algorithms for computational creativity.

Machine Learning & Neural Networks

The recent AI boom has been largely driven by machine learning and related techniques. I explained it last year in Artificial Stupidity, so I’ll just quote myself:

The discipline’s basic approach is to create self-learning algorithms. Developers no longer tell the machine how to solve a problem. Instead, the algorithm is designed in a way that it learns to approximate the best solution by testing data in an iterative manner and thereby detecting patterns. In order for it to do so, the machine needs to be fed with a lot of data.

A typical use case for machine learning is object recognition in pictures or videos. In order to detect a cat, you have to feed the algorithm thousands upon thousands of pictures (usually labeled data, though Google already has algorithms that don’t rely on this). After a huge amount of iterations, the machine learns to understand which visual features define a cat.

A typical application in the field of computational creativity is to feed a machine learning system with some kind of artistic input, from which it distills certain stylistic attributes and then creates a novel output by combining the attributes in a new way. This is how Sony’s AI famously composed a Beatles-style song last year (the actual production and recording were done by humans).

If we stick with music as an example, the algorithm might figure out that songs written in a certain key, at 102 bpm, that use no more than 20 notes (arranged in a certain way and played by at least 4 instruments plus a rhythm section), and featuring a deep female voice sell particularly well in Eastern Europe. Based on this, it could, in theory, create something that we haven’t heard before.

You can also use feedback during training to influence the creations. That approach is called reinforcement learning. Since creative products need to meet people’s taste, it makes sense to incorporate their feedback I suppose.

Evolutionary & Genetic Algorithms

Glossing over all mathematical details and nuances, the basic idea behind these types of algorithms is continuous optimization by using mutation and selection. Genetic algorithms are used to find solutions to optimization and search problems by creating random system states which are then evaluated against feedback (the so-called fitness function). A simple example is NEvAR, an algorithm that generates 3D images which get better after each round of assessment.

The catch here, of course, is that the criteria against which you assess the output are part of said fitness function. As judging creative work is often highly subjective, this is tricky, to say the least. Some creative algorithms of that type, therefore, use feedback from human observers (which only leads to reliable results at scale).

Rule-Based Algorithms

The old-school algorithm. I almost skipped it but it’s my understanding that many artificial artists use a combination of several algorithms and rule-based ones haven’t vanished yet. They are pretty much what the name suggests: developers write a set of rules within which the algorithm operates. It’s the classic if this, then that logic. The quality of the output, of course, comes down to the quality of the rules. If you had a secret formula to write Top 10 hits, such an algorithm would certainly be useful. Absent one, those algorithms are unlikely to create any kind of artifact that is “groundbreaking” or of the transformational category.

So much for an introduction. In part two I’ll look at the current state-of-the-art of computational creativity in six creative domains. If you don’t want to miss it, follow attentionecono.me or subscribe to my newsletter.