Player Responses to a Live Algorithm: Conceptualising computational creativity
without recourse to human comparisons?
Oliver Bown
Design Lab
University of Sydney
NSW, 2006, Australia
oliver.bown@sydney.edu.au
Abstract
Live algorithms are computational systems made to perform
in an improvised manner with human improvising
musicians, typically using only live audio or MIDI
streams as the medium of interaction. They are designed
to establish meaningful musical interaction with
their musical partners, without necessarily being conceived
of as “virtual musicians”. This paper investigates,
with respect to a specific live algorithm designed
by the author, how improvising musicians approach and
discuss performing with that system.
The study supports a working assumption that such systems
constitute a distinct type of object from the traditional
categories of instrument, composition and performer,
which are capable of satisfying some of the expectations
of an engaging improvisatory performance
experience, despite being unambiguously distinct from
a human musician. I investigate how the study participants’
comments and actions support this view. Specifically:
1) participants interacting with the system had
a stronger sense of the nature of the interaction than
when they were passively observing the interaction; 2)
participants couldn’t tell what the “rules” of the interactive
behaviour were, and didn’t feel they could predict
the behaviour, but reported this as being a positive,
engaging aspect of the experience. Their actions implied
that the improvisation had purpose and invited engagement;
3) participants strictly avoided discussing the
system in terms of virtual musicianship, or of creating
original output, and preferred to categorise the system
as an instrument or a composition, despite describing
the interaction of the system as musically engaging; 4)
participants felt the long-term structure was lacking.
Such results, it is argued, lend weight to the idea that
as CC applications in real creation scenarios grow, the
creative contribution of computer systems becomes less
grounded in comparison with human standards.
Introduction
Live algorithms (Blackwell, Bown, and Young, 2012) are
software systems designed to autonomously perform music
with live musicians, typically in an improvised music format.
There has been a great deal of activity in this area recently,
owing to the increasing ease with which artist programmers
can put together powerful realtime systems incorporating
machine listening, realtime synthesis and patterning,
and forms of adaptive behaviour. Recent concerts, attached
to electronic arts and music conferences such as the
International Symposium on Electronic Arts (ISEA) 2013,
and New Interfaces for Musical Expression (NIME) 2014,
have demonstrated the diversity of approaches to live algorithms
(see Bown et al. (2013) for a discussion of these concerts).
As in all aspects of computational creativity, the question
of evaluation in live algorithms requires detailed consideration,
as there are no simple, objective measurables that indicate
when computer generation of output has been creatively
successful. Two issues are important: how system
output is evaluated by humans, and the extent to which we
can attribute the creative component of the output to the system,
rather than to its maker or to the ‘inspiring set’ (Ritchie,
2007): the set of all examples given to the system.
In live algorithms, the creative process is somewhat different
from many instances of automated creative generation,
since the output is always the result of the interaction
between a human and a computer system. It is an interactive
creative scenario. This muddies the issue of the attribution
of the creativity further, but at the same time presents alternative,
more tractable questions regarding the success of the
system in its collaborative, improvisatory role.
Whilst it should be borne in mind that such questions
regarding the interactive experience of live algorithms are
separate from the core questions of computational creativity
evaluation, there is still much to be learnt from such an
analysis. In Bown (2014), I argue that a human-focused,
qualitative, and strongly context-aware approach to studying
computational creativity systems is important to advancing
evaluation. In the case of an artistic robot, for example, one
should begin by examining the full set of interactions between
the system, its maker, its operator, its audience and
so on, before deciding how one should frame questions of
creative ability. This is to avoid the danger of inappropriately
framing the activity and the agency of participants in
that activity. How is the creative attribution divided between
these actors? How do people perceive the system, not only
in terms of good or bad output, but in terms of the way in
which the system’s activity is presented in a social context?
Others, particularly Colton (e.g., (Colton et al., 2014)) have
emphasised the management of the social interactive context
in computational creativity, presented as a means to enhance
Proceedings of the Sixth International Conference on Computational Creativity June 2015 126
the perception of creativity, rather than as a means to better
understand interaction with creative systems to improve
their design and efficacy in areas of application.
Such developments point to the possibility that any proposed
measure of the creativity of a system is significantly
less fruitful than a rich description of the system as an agent
with creative affordances described by its networks of interaction.
This neutralises the crisis of working out how
to score creativity, and provides simple practical analysis
which can support real applications in the way that human
computer interaction (HCI) and interaction design does to
great success. Thus a qualitative descriptive approach is pursued
in the present research, in order to build a rich descriptive
understanding of human-machine creative interactions
in practice in the context of live algorithms.
A central motivation for conducting the following study is
to conduct computational creativity research that is more focused
on the details of a participant’s interaction with a creative
system, involving a number of dimensions of experience
that are relevant to creativity, and in doing so contribute
to an understanding of how such systems work in practice in
real creative contexts.
In this paper I study the responses of improvising musicians
to Zamyatin, a live algorithm system that I have developed
and worked with artistically since 2010. Zamyatin
has performed with a wide variety of musicians. It is conceptually
speaking a very simple system as far as creative
systems go, in terms of the generation of original content on
its own. Specifically it is less driven by the use of musical
intelligence than by an interest in low-level gestural interaction.
But in light of the value of diverse approaches to
computational creativity, I view the system as a useful experiment
in computational creativity in that it is successful
in establishing an autonomy of behaviour, both conceptually
and as perceived.
The questions the study looks at are focused on the ways
in which participants experience and benefit from the creativity
of a system: (1) how effective the system is at contributing
to an effective performance; (2) the extent to which
the participant experiences the system as autonomous, and
also human-like, and how this influences other aspects of
the perception of the system, and; (3) whether the participant
experiences the system as originating novel output, and
how this influences (and is possibly influenced by) the general
perception of the system.
These are issues that we must clearly gain an understanding
of as part of a body of knowledge in applied computational
creativity. The computational creativity literature
remains lacking in work that formally studies these basic
forms of interaction and experience using qualitative methods.
The first question has self-evident value, and in one form
or another is naturally asked in the course of creating any
musical system. A challenge for a more experience- and
interaction-focused computational creativity research program
is to balance this goal with that of advanced computational
generative sophistication. The perception of autonomy
addressed in the second question is an important topic
for the study of computational creativity. Autonomy is a
critical component in the making of creativity: a system can
only be called creative insofar as it possesses some degree of
autonomy in the output it creates. Perceived autonomy may
not be actual autonomy and vice versa, and actual autonomy
anyway lacks a robust applicable definition. The distinction
between software autonomy in general and human-like autonomy
is one that will need to be unpacked further as we
witness computationally creative systems at play in real interactive
scenarios, and it is important to understand how
individuals experience that autonomy and how that influences
their behaviour towards the system and their own activities.
Finally, in the context of interactive music creation
we are interested in how the system can drive surprise and
intrigue in the co-performer, and under what circumstances
the performer acknowledges something as either creative, or
in terms that connote creativity. Here it is particularly interesting
to look at the language used, as this is an area where
the anthropomorphism of cognition comes up easily.
I begin by describing the motivations behind the design
of Zamyatin in the following section, before moving onto
describing the study and results.
Zamyatin
Zamyatin is a software system in ongoing development since
2010 (Bown, 2011). Before describing the design of Zamyatin,
it is necessary to explain some of the design considerations,
including a number of aesthetic decisions. An earlier
description of Zamyatin’s design is given in Bown (2011).
One of Zamyatin’s main goals was to emphasise the experience
of interacting with something that ‘felt’ autonomous
and engaged in interaction, even if, it does not make sophisticated
use of musical knowledge. For this, the free improvised
mode provides a context that allows one to explore behaviour
in a more abstract way than is afforded by many musical
genres. Improvising software agents are a longstanding
area of activity. George Lewis’ Voyager system(Lewis,
2000) is a widely known example, and uses a hand-coded
complex of interacting generative elements to create rich,
diverse and musically responsive behaviour. Musicians performing
with Lewis’ system can be seen deeply engaged in
the musical interaction as if performing with another human
improviser. The use of a Disklavier (an acoustic piano that
can be controlled by MIDI via mechanical actuators) limits
the sense of a computer being involved. Artists such as
Lewis have reported the responses of musicians performing
with their systems, but such reports increasingly show that
it is hard to pin down exactly how musicians think about,
understand and evaluate such systems, suggesting the need
for studies that get into more detail about the conceptual
language and approaches used. Banerji (2012), for example,
takes an anthropological approach, with a strong focus
on working in real contexts, and looking as much at how
the system influences the performer’s behaviour as at how
the performer judges the system. Other projects such as the
work of Plans Casal and Morelli (2007), focus strongly on
using low-level realtime audio analysis and resynthesis to
give the performer a strong sense that the software acts as
a responsive agent, through interactive immersion. Pachet’s
(2004) approach to establishing engagement is to mimic the
Proceedings of the Sixth International Conference on Computational Creativity June 2015 127
style of the improviser in a call and response fashion. Similarly
with Blackwell and Young (2004) and Brown, Gifford,
and Voltz (2013), who draw on a style analysis and resynthesis
of the performer’s input to establish a strong sense of engagement.
Although these projects report on user-responses,
further research is needed to determine whether these are
indeed effective strategies for creating desirable interactive
musical experiences.
A common challenge for the makers of generative systems
is how to endow the system with autonomous behaviour
that transcends the rules put into it by the programmer.
That is, if your system is a collection of procedural
instructions defined by the programmer, then even if the specific
behaviour of the system is original, being some possibly
unexpected product of the interacting rules, the general
nature of the system’s behaviour remains down to the programmer,
since no new knowledge has been gained by the
system.
There are three commonly cited ways around this problem
(Todd and Werner, 1999). The first is already implied
above: if the set of rules I provide are complicated enough,
then from the interaction of these elements there will emerge
new, higher-level behaviours that were not anticipated. The
classic example is flocking behaviour, where the programmer
defines the behaviour of individual ’boids’ (Reynolds,
1987), but nowhere dictates that the system should start
forming oscillating blobs on a macroscopic scale. Classical
work from the generative art canon also highlight the value
of this approach. Both Harold Cohen’s celebrated AARON
system (McCorduck, 1990) and George Lewis’ Voyager system
(Lewis, 2000) consist of complex rule sets that result in
outcomes even their makers find surprising. In this case, it is
perhaps wrong to describe what emerges from these systems
as new knowledge.
The second approach is that the system learns. This is easily
understood by analogy with how humans acquire knowledge
that they are not born with. A large number of systems
use learning to build musical knowledge, and famous examples
include David Cope’s EMI (Cope, 1996) and Franc¸ois
Pachet’s Continuator (Pachet, 2004). In these cases, the input
knowledge now comes from a body of input musical
data as well as the programmer. One problem then is how to
avoid the system becoming just a copycat. The system needs
not only to learn the style but to learn how to produce new
material in that style. Current systems have yet to show how
the learning itself can perform this extrapolation.
A third approach uses targeted evolution or another form
of optimisation, dictated either by a measurable target behaviour,
or user-feedback applied to a population of evolving
behaviours. The rationale goes that a target behaviour itself
does not contain the knowledge about how to achieve that
behaviour, but running an evolutionary system to achieve
that target can discover novel solutions which themselves
constitute knowledge. Experiments in artificial evolution
have shown the discovery of such solutions. For example,
the coevolution of predator and prey systems reveal the
emergence of specific hunting or hiding techniques (Cliff
and Miller, 1995). Here the knowledge is produced through
interaction, or learning-by-doing. Thus by specifying a target
behaviour in the form of an evolutionary goal, one can
drive a system to discover component behaviours that are
not specified in that goal.
Unlike the majority of live algorithm approaches to deriving
behaviour, Zamyatin is not a corpus or machine-learning
based system, and employs this third approach to achieving
autonomy. I draw on Blackwell and Young’s PfQ framework
to describe the system (Blackwell, Bown, and Young,
2012). Passing from the input (P) layer to the inner ‘patterning’
(f) layer are low-level feature values derived from
the input musical data of the system. Passing from the inner
layer to the ‘instrument’ or ‘sounding’ layer (Q) are control
signals. These can be thought of as the equivalent to
the human physical control ‘signals’ applied to a musical instrument,
i.e., the movement of the hands, feet, breath, etc.,
although the object being controlled might involve its own
generative elements. In Zamyatin, the inner patterning system
is a type of decision tree, coupled with a internal array
of states, that together feedback on themselves. This inner
patterning system is connected to the outside world though
the input layer and output layer. Somewhat like a traditional
feedforward multilayer neural network, the connections between
these layers flow in the forward direction only.
A decision tree is a binary tree that propagates a decision
making iteration from the root of the tree to one of the leaves
(leaves represent decisions), at each junction choosing to go
one way or the other based on whether a single numerical
value is above or below a single threshold. Decision trees
are used commonly as efficient classifiers. The internal state
array is simply an array of floating point values in the range
[0,1]. In addition to the internal state, the system is constantly
being fed an input state derived from low-level features
of the incoming audio. Decisions at each node in the
decision tree are made based on either the current state of
the low-level audio features being passed into the system, or
the internal state array. A leaf in the decision tree contains a
list of actions which include passing on control commands
to the musical system (Q) and also updating the state array.
In this way, the decision tree and state array form a feedback
system that can exhibit complex dynamics in the absence of
any input, and can also be driven by changes to the input.
Previous work (Bown and Lexer, 2006) has looked at the
musical use of neural networks with similar properties.
An evolutionary approach is applied to the design of the
decision tree, including the architecture of the tree (which
can grow or shrink over time), the parameters of each decision
node (which value to query and what threshold to apply)
and the (variable length) list of actions to perform at
each leaf. Actions control how the internal state array is updated,
applying simple arithmetical operations to the state
values.
The inner layer updates at a ‘control rate’ of around 20hz.
It outputs two forms of control data at each update: a single
integer, representing its current decision state, and an array
of floating point values in the range [0-1], representing its
internal state. Both are actually used to control the musical
output. In evolving the system behaviour, a fitness function
is hand-coded, that takes into account the pattern complexity,
and other patterning properties such as degree of variProceedings
of the Sixth International Conference on Computational Creativity June 2015 128
ability and repetition, of the system’s output under various
input conditions. Different variant fitness functions are used
to create large populations of decision tree candidates, which
are then creatively explored during the preparation of musical
work.
Like procedural systems, Zamyatin does not draw on a
corpus of musical knowledge, but instead attempts to establish
novel behaviour through the interplay between the
programmer specifying a behavioural target and the system
evolving novel behaviours that achieve that target. The target
behaviour does not describe the final musical output,
but the output of the nested control system (f) that operates
a number of virtual musical instruments. This target
behaviour is defined by the programmer and the selection
or definition of different target behaviours to suit the performing
musician becomes part of the creative process of
preparing Zamyatin for each new performance.
Musical Study
Three improvising musicians (P1, P2, P3) were invited to attend
a focus group to investigate musician responses to Zamyatin.
The goal was not to set up a musical Turing test:
there was no attempt to conceal the computational status of
the system. Instead, the study looked at questions of engagement,
experience and perception in improvised interaction.
The study was set up as a focus group in order to stimulate
interaction between the participants, to look at the way they
discussed musical interaction, and to get them to observe
each other playing.
Participants were first shown a video recording of the system
performing with a musician and asked questions about
how they perceived the interaction with the musician. They
were then played an audio recording of an earlier manifestation
of the system performing with another musician and
asked similar questions. They were then asked to perform
with the system and develop their responses to it.
The author initially did not explain the design of the system,
but later answered questions and provided more context
as the study progressed, in response to the participants’
questions.
Several other interviews with performers conducted prior
to the focus group have influenced the expectations of the
author in approaching the focus group. These will be reported
in full in a forthcoming journal paper.
Results
Three main results are considered here:
1. Participants interacting with the system had a
stronger sense of the nature of the interaction than
when they were passively observing the interaction.
During the initial observation of the pre-recorded concert,
all three participants said that they did not see any clear
clues as to how the system was responding, what information
it had access to from the musician, and what the
interaction paradigm was. This was manifest largely in
the sense of uncertainty surrounding the interaction. The
musicians had no way of identifying clear paths of causality
from the musician to the system.
Of the system in general, P2 says the following:
The system of interaction is not obvious to me. . . . I
can’t tell. At times [the musician] is loud and I don’t
think the software’s responding, or vice versa, and
then sometimes the two things are loud or the two
things are soft. The obvious parameters you can
sample and listen to are like dynamics and pitch, timbral
stuff . . . There doesn’t seem to be any clear oneto-one
relationships with what the software does, or
it changes over time? Sometimes it reacts in a particular
way and sometimes it doesn’t.
In performing with the system the musicians’ responses
shifted from this ambiguity to a greater sense of awareness
that the system was responding to their playing. A
good deal of uncertainty remained about precisely how
the system behaviour was influenced by the musician, and
as discussed below this is a theme in itself.
After watching P1 performing, P2 says:
It was way more dynamic than it comes across in the
flat stereo recordings, it was actually really good. It
surprised me a few times how loud it was prepared
to go and transgressive of the duo in a way . . . mainly
with dynamics but sometimes placement too . . . it
did some bizarre things and you go “oh that’s cool”.
. . . but when it’s compressed . . . you don’t understand
the dynamics that much. . . .
(Participant was asked to explain ‘transgressive’) It
did naughty things, to do with timbres and placement.
If it was someone playing that material you’d
go, they’re being a bit upfront, kicking the thing
along a bit, putting provocations in. I like that.
P3 adds:
That’s the weird thing about it; you can really sense
that something’s happening but I can’t tell what it is.
Interestingly, also, the critical analysis of the system naturally
extended to the performing musician as well. The
evaluation of the improvisation by the participants naturally
applied as much to the performers as to the systems.
This may be more their habit, but of some relevance,
Banerji (2012) has proposed looking at the impact
on musicians’ playing as a form of ethnographic approach
to studying the qualities of live algorithms.
P3: One thing I found that the second musician
wasn’t interacting with the software at all. I felt
like maybe they were just playing. I didn’t hear too
much active listening, they were obviously playing
with it, but didn’t really feel like they were kind of
. . . that level of interactivity wasn’t really there from
their performance. . . . It was a real contrast of style
I thought. I thought the first guy was really overtly
interacting with it to quite a large extent, and the second
I thought wasn’t. But it’s hard to say what the
agenda is. . . . It’s not that I enjoyed the first one better.
It’s more that if someone told me if the second
player was in another room not being able to hear the
performance I could believe you.
Proceedings of the Sixth International Conference on Computational Creativity June 2015 129
2. Participants couldn’t tell what the “rules” of the interactive
behaviour were, and didn’t feel they could
predict the behaviour, but reported that they did experience
the behaviour as interactive, and presented
this uncertainty as being a positive, engaging aspect of
the experience. Their actions implied that the improvisation
had purpose and invited engagement.
In discussing what if any cues revealed the nature of interaction
between system and human performer during the
video playback section of the study, participants noted
that any candidate explanations they developed for the
behaviour of the system were frustrated by its seemingly
changing interactive behaviour. For example, one participant
began thinking that the system was matching the intensity
of the performer’s behaviour, but then found that
the oppose suddenly occurred.
During interaction with the system, performers remained
unsure about exactly what the responsive behaviour consisted
of, but reported that they did feel that there was
some sort of complex interaction taking place, and finding
this particularly engaging, owing to the uncertainty of
the system’s behaviour.
P2 (describing performance with P1): That started
off with a noisy atmospheric tone. P1 came in and
it maintained its thing, it kept its thing for a while.
which kinda surprise me. I thought the introduction
of a strong tone would shift it, but it didn’t shift it
and I thought “that’s cool”. . . . the fact that it doesn’t
jump the whole time makes it worth listening to. If it
was jumping the whole time with your stimuli, with
the distinction from the live instrument to a clear distinction
from that it would drive you crazy.
This uncertainty was also described as potential source
of frustration. Equally, the stability of the system over the
long-term was described as a potential source of boredom.
But on the whole participants agreed that the balance between
uncertainty and predictability was well measured to
create an effective sense of engagement for the musician.
P1: To begin with, and that’s the same with the other
ones I saw, it takes the musician to initiate the interaction.
. . . it was playing a long granulated tone, I
came on top of that with a between note, probably to
create some symbiosis with what it was doing. Then
I found as I went into it that I wanted to find out that
it reacted to what I was doing, and this was less clear.
Sometimes it did and sometimes it didn’t.
P3: There was a really loud section with no stimulus
behind it, and its like, where did that come from,
but I’m getting closer to seeing [the relationship]
. . . actually I’d find it quite stressful to perform with.
3. Participants strictly avoided discussing the system in
terms of virtual musicianship, or of creating original
output, and preferred to categorise the system as an
instrument or a composition, despite describing the interaction
of the system as musically engaging.
The participants were clear explicitly – in response to direct
questions about it – and implicitly – in the way they
described the interaction with the system – that they felt
no compulsion to see the system as a ‘performer’, preferring
instead to view it as a form of complex instrument,
or interactive score. However, the participants equally
acknowledged that the behaviour of the system made it
stand out from other types of digital interactive systems
or instruments, particularly in terms of the autonomy of
behaviour. To some extent this afforded the use of terms
such as a perceived volition, that are arguably not normally
associated with machine behaviour.
As an example of a clear shortcoming, P2 states:
It seemed a bit confused with the very high frequencies
. . . I felt that it kind of suddenly went “I can’t
actually see you” . . . It was quite interesting. If it
was another player you’d go, ok, that’s working.
They elaborate on their perception of the system in terms
of humanness:
I’ve steered clear from [referring to] anything to do
with a performer because it doesn’t feel like a human
being at all, but it feels interesting, you’ve set up a
compositional tool that’s not momentary predictable
but in the long term its predictable.
The participant describes this engagement further as follows:
It was good, it was something I wanted to do listening
to the other things: give it its own space, do its
thing. It’s an intriguing notion that you didn’t play
for a while and then it comes up with something else.
It kinda lets the audience know. It’s not some sort of
stupid device, something of its own volition.
When asked how it compared to ‘mere tape’, P2 elaborates:
I think audiences are pretty smart, they understand
what tape is, what predetermination is and what liveness
is, and if the audience were sitting there knowing
that it’s a live system and it seems to have some
initiative without the player, I think that’s an interesting
moment. . . . But I’d say the choosing the samples
becomes this overridingly important compositional
decision. . . . I feel that with this, whatever
samples you put into the composition . . . the machine
has some sort of ability to stop and start things.
4. Participants felt the long-term structure was lacking.
It was widely agreed that the system did not convincingly
deal with long term structural management of the performance.
P2: Listening to both of those things a lot of the activity
is very much less than 3 seconds, so there’s a
lot of active many-events-per-minute sort of feel to
it, and because it goes on for some time in that way
it then has a sort of strange flatness as a result, and
after a while you settle into the fact that there aren’t
going to be any super-long events, and so in a sense
it kind of flattens the whole thing down and makes it
kind of amorphous.
Proceedings of the Sixth International Conference on Computational Creativity June 2015 130
The participant frames this in the context of contemporary
improvised music:
It’s a subject right at the heart of what’s going on
in improvised music. Probably always has been, but
seems really central to it these days. It feels to be
generational as well. The older generation may feel
that they’re not interacting and reacting (themselves)
but they tend to more than younger generations. . . . I
feel like there’s players around now who work in
much longer structures and they don’t want to have a
dialogue which is over some 10 second framework.
Discussion
The results of this study go some way to confirming existing
assumptions and findings about evaluation of musical systems.
The first result affirms a general principle that certain
knowledge is better acquired through active participation.
Interacting with a system tells you more about its interactive
capacity than watching an interaction with a third party.
This may not be manifest in the form of a expressible understanding
of what the system is doing, as was the case in
the present study, but nevertheless the participant in the interaction
gains a direct sense of the interactive nature of the
system, that may be obscure from outside.
This has implications for the audience experience of the
work. They may not be fully aware of the experience of the
musician during the performance. On the other hand, the
expression and observed response of the musician can be
important to an observer understanding the interaction, and
may indirectly reveal the experience. Pachet has shown how
the video footage of participants, or simply composers engaged
with the treatment of their own work, can do a fantastic
job of revealing basic facets of user-experience (Addessi
and Pachet, 2006; Pachet, 2014).
Related to this, a common theme in the evaluation of autonomous
music and art system is the question of making
use of a Turing-style test (e.g., (Ariza, 2009; Bishop and Boden,
2010; Pease and Colton, 2011)). Results such as those
of Moffat and Kelly (2006) show that positive results can
be easily achieved in situations where people try to guess
whether artefacts were computer or human generated, i.e.,
the system generated output can pass as human. However,
without involving any form of probing or interaction with
the system, the test in this form doesn’t really tell us anything
about the system, its intelligence or creative capacity
(Pease and Colton, 2011). Despite what is said about the
great communicative power of art and music, these artefacts
form a poor window onto their creators.
Nevertheless, it is still reasonable to expect that in general
there are cues in creative outputs which reveal aspects
of the nature of the system producing them, and which may
be identified in interactive scenarios, but also possibly without
the need for interaction. These cues may not be reliable
identifiers of whether or not the system is computer or
human, and should be better understood as contributing to
a qualitative evaluation of creative or interactive behaviour.
More generally, we may talk of the character of the system
and how it contributes to or stimulates a productive musical
process.
The musicians participating in the study did develop a
sense of the cues that indicated Zamyatin’s responsive behaviour
in certain ways, sometimes, but without certainty.
This led them to feel that the system was nontrivial and invited
an engagement with the behaviour of the system.
Related to this are the other two results. The character of
the system is one in which an actively obscure relationship
between performer action and system result is sought. To
this end the evolutionary strategy has proven to be a convenient
approach to relieving the system designer from the task
of dictating the system’s response directly, working around
the “Lovelace objection” that a computational system might
only do what it has been programmed to do.
Finally we come to the issue of whether the system was at
all perceived as bearing the qualities of a human performer.
The response was resoundingly negative in answer to this
question. Whilst, as stated in the introduction, the aim of the
system design was never to simulate or mimic human behaviour,
a stated goal has been to explore the middle ground
between inanimate objects that do not exhibit adaptive or
proactive behaviours, and sentient humans, or other creatures.
The participants unambiguously placed the system
in the category of objects, as opposed to any sort of ‘performer’,
equating it either to an instrument or composition.
It does not follow that they perceived this object as dumb or
lacking lifelike properties.
Scoring Zamyatin
From these results we can consider the questions posed at
the beginning of the paper:
1. (Q1) How successful is it as at creating effective performances
with improvising musicians? The participants’ responses
give enough support to a positive answer to this
question, specifically in terms of the interesting dynamics
produced by the system’s interactive behaviour.
2. (Q2) To what extent do performers conceptualise of and
perceive Zamyatin as autonomous, as well as human-like,
and how does this influence other aspects of the perception
of the system? The responses are more ambiguous
with respect to this issue, not least because the definition
of autonomy is itself hard to pin down in application. Because
of this, participants were not asked to discuss autonomy
directly, but we may make inferences based on
their responses. Significantly, they perceived the system
as being both (i) not passive in the form of its responses
to input, and (ii) able to drive the performance through
spontaneous action that appeared to come from nowhere.
These support a technical definition of autonomy as behaviour
that is not entirely determined from outside of the
system, and the varying nature of the system’s predictability
supports an information theoretic form of this. However,
there are other senses of perceived autonomy that
could be achieved. Future studies could work towards understanding
in greater detail the space of possible types of
autonomy (for example as discussed in Eigenfeldt et al.
(2013)) that might be perceived in a system.
Proceedings of the Sixth International Conference on Computational Creativity June 2015 131
3. (Q3) Does Zamyatin actually originate novel responses
as far as the performers are concerned? The predominant
response to this from participants was ‘no’. They quickly
perceived that the system worked within strict limits, with
the musical style and much of the content (e.g., choice of
sounds, pitch sets, etc.) dictated by the system designer.
But equally the participants responses indicated that they
did attribute actions to the idiosyncratic nature of the
system, which was described to them as having resulted
from an evolutionary search that went beyond the input
of the designer. For example, on numerous occasions
the behaviour of the system was described by participants
as surprising, and not like anything a human would do,
coupled with value judgements ranging from this being
highly engaging, to it being frustrating. We could claim
that a surprising and valued response is technically speaking
creative, according to the most commonly agreed definition
of the term. This would be a generous interpretation,
since many ‘dumb’ processes might achieve some
such level of surprise in interaction. If instead we were to
apply Colton’s ‘creativity tripod’ of imagination, skill and
appreciation (Colton, 2008), we would have to accept that
at best only skill could be claimed (I would claim that the
system can appear skilful in the complex manipulation of
electronic sound). An open question is what kinds of systems
stimulate ‘perceived imagination’ and ‘perceived appreciation’,
and whether these are in fact always relevant
in contexts such as this: is it important to the perception
of musical creativity that such elements are perceived?
Conclusion
The evaluation of systems from computational creativity, using
qualitative analysis grounded in specific contexts of creative
interaction, is an important part of the emerging suite
of research methods we use to discover and understand how
systems can act successfully to support creativity or act as
creative agents. This paper has attempted to dig deeper into
how improvising musicians, presented with a live algorithm
system, approach, interpret and engage with that system in
an applied context.
The results suggest ways in which the system, Zamyatin,
could be improved to create more compelling improvised
musical experiences. Good long-term structure is a
challenging area that this system could improve upon. The
results appear to affirm the value of exploring forms of
software-based musical agency that does not conform to human
modes of behaviour but that still produce engagement.
This could be developed further by categorising these kinds
of behaviour.
The relationship between the behaviour of the system and
the engagement of the performers could be developed by
improving the user-interface to the underlying evolutionary
techniques, possibly using interactive evolutionary techniques,
so that there is a real capacity for a musician to feedback
on and modify the behaviour of the agents. It was also
apparent from the study that certain traits of the system, such
as the degree of uncertainty of its behaviour, could be explicitly
recognised as adding to the musicality of the system,
and could be codified into future fitness functions. An immediate
goal for Zamyatin is to create a modular system that
can be easily incorporated into live performance sets by nonprogrammer
musicians, and these ideas can be incorporated
into that design.
In addition, the view held by this author is that questions
of computational creativity are now shifting towards more
applied areas where the comparison with human creative activity
is less of a concern than a more open-ended understanding
of how machines may act creatively. This is weakly
supported by the research in this paper, in which a failure to
stand up to any sort of Turing-style test does not diminish
the discussion of the creative potential of the system. This
is a perspective that warrants further study across a range of
systems.
<references_biblio/>
References
Addessi, A. R., and Pachet, F. 2006. Young children confronting
the continuator, an interactive reflective musical
system. Musicae Scientiae 10(1 suppl):13–39.
Ariza, C. 2009. The interrogator as critic: The turing test
and the evaluation of generative music systems. Computer
Music Journal 33(2):48–70.
Banerji, R. 2012. Maxine’s turing test - a player-program as
co-ethnographer of socio-aesthetic interaction in improvised
music. In Proceedings of the Artificial Intelligence
and Interactive Digital Entertainment (AIIDE’12) Conference.
Bishop, M., and Boden, M. A. 2010. The turing test and
artistic creativity. Kybernetes 39(3):409–413.
Blackwell, T., and Young, M. 2004. Self-organised music.
Organised Sound 9(2):137–150.
Blackwell, T.; Bown, O.; and Young, M. 2012. Live algorithms.
In McCormack, J., and d’Inverno, M., eds.,
Computers and Creativity. Springer-Verlag.
Bown, O., and Lexer, S. 2006. Continuous-time recurrent
neural networks for generative and interactive musical
performance. In Rothlauf, F., and Branke, J., eds.,
Applications of Evolutionary Computing, EvoWorkshops
2006 Proceedings.
Bown, O.; Eigenfeldt, A.; Martin, A.; Carey, B.; and
Pasquier, P. 2013. The musical metacreation weekend:
challenges arising from the live presentation of musically
metacreative systems. In Proceedings of the musical
metacreation workshop, AIIDE conference, Boston.
Bown, O. 2011. Experiments in modular design for the
creative composition of live algorithms. Computer Music
Journal 35(3).
Bown, O. 2014. Empirically grounding the evaluation
of creative systems: incorporating interaction design.
In Proceedings of the Fifth International Conference on
Computational Creativity.
Brown, A.; Gifford, T.; and Voltz, B. 2013. Controlling interactive
music performance (cim). In Proceedings of the
Fourth International Conference on Computational Creativity,
221.
Proceedings of the Sixth International Conference on Computational Creativity June 2015 132
Cliff, D., and Miller, G. F. 1995. Tracking the red queen:
Measurements of adaptive progress in co-evolutionary
simulations. In Advances In Artificial Life. Springer. 200–
218.
Colton, S.; Cook, M.; Hepworth, R.; and Pease, A. 2014.
On acid drops and teardrops: observer issues in computational
creativity. In Proceedings of the 7th AISB Symposium
on Computing and Philosophy.
Colton, S. 2008. Creativity versus the perception of creativity
in computational systems. In AAAI Spring Symposium:
Creative Intelligent Systems, 14–20.
Cope, D. 1996. Experiments in Musical Intelligence. Madison,
WI: A-R Editions.
Eigenfeldt, A.; Bown, O.; Pasquier, P.; and Martin, A. 2013.
Towards a taxonomy of musical metacreation: Reflections
on the first musical metacreation weekend. In Ninth
Artificial Intelligence and Interactive Digital Entertainment
Conference.
Lewis, G. E. 2000. Too many notes: Computers, complexity
and culture in voyager. Leonardo Music Journal 10:33–
39.
McCorduck, P. 1990. AARON’s Code: Meta-Art, Artificial
Intelligence, and the Work of Harold Cohen. W. H. Freeman
and Co.
Moffat, D. C., and Kelly, M. 2006. An investigation into
people’s bias against computational creativity in music
composition. Assessment 13:11.
Pachet, F. 2004. Beyond the cybernetic jam fantasy: The
continuator. IEEE Computer Graphics and Applications
24(1):31–35.
Pachet, F. 2014. Non-conformant harmonization: The real
book in the style of take 6. In Proceedings of the Fifth
International Conference on Computational Creativity.
Pease, A., and Colton, S. 2011. On impact and evaluation in
computational creativity: A discussion of the turing test
and an alternative proposal. In Proceedings of the AISB
symposium on AI and Philosophy.
Plans Casal, D., and Morelli, D. 2007. Remembering the future:
An overview of co-evolution in musical improvisation.
In Proceedings of the 2007 International Computer
Music Conference.
Reynolds, C. W. 1987. Flocks, herds and schools: A
dstributed behavioural model. In Computer Graphics
(SIGGRAPH ’87 Conference Proceedings), volume 21,
25–34.
Ritchie, G. 2007. Some empirical criteria for attributing
creativity to a computer program. Minds and Machines
17(1):67–99.
Todd, P. M., and Werner, G. 1999. Frankensteinian methods
for evolutionary music composition. In Griffith, N.,
and Todd, P. M., eds., Musical Networks: Parallel Distributed
Perception and Performance. Cambridge, MA:
MIT Press/Bradford Books. 313–339.
Proceedings of the Sixth International Conference on Computational Creativity June 2015 133