Learning how to reinterpret creative problems
Kazjon Grace
College of Computing and Informatics
University of North Carolina at Charlotte
Charlotte, NC, USA
k.grace@uncc.edu
John Gero
Krasnow Institute
for Advanced Study
George Mason University
Fairfax, VA, US
john@johngero.com
Rob Saunders
Faculty of Architecture, Design and Planning
Sydney University
Sydney, NSW, Australia
rob.saunders@sydney.edu.au
Abstract
This paper discusses a method, implemented in the domain
of computational association, by which computational
creative systems could learn from their previous
experiences and apply them to influence their future behaviour,
even on creative problems that differ signifi-
cantly from those encountered before. The approach is
based on learning ways that problems can be reinterpreted.
These interpretations may then be applicable to
other problems in ways that specific solutions or object
knowledge may not. We demonstrate a simple proof-ofconcept
of this approach in the domain of simple visual
association, and discuss how and why this behaviour
could be integrated into other creative systems.
Introduction
Learning to be creative is hard. Experience is known to be
a significant influence in creative acts: cognitive studies of
designers show significant differences in the ways novices
and experts approach creative problems (Kavakli and Gero,
2002). Yet each creative act is potentially so different from
every other act that it is complex to operationalise the experience
gained and apply it to subsequent acts of creating.
Systems that can, through experience, improve their own
capacity to be creative are an interesting goal for computational
creativity research as they are a rich avenue for improving
system autonomy. While computational creativity
research has coalesced over the last decade around quanti-
fied ways to evaluate creative output, there have been few
attempts to imbue a system with methods of self-evaluation
and processes by which it could learn to improve. This research
presents one possible avenue for pursuing that goal.
A distinction should be drawn between learning about the
various objects and concepts to be used in particular creative
acts, which serves to aid those acts specifically, and learning
about how to be a better creator more broadly. Knowledge
about objects influences future creative acts with those objects,
but the generalisability of that knowledge is suspect.
One example of where this learning challenge is particularly
relevant is analogy-making, in which every mapping
created between two objects is, by the definition of an analogy
as a new relationship, in some way unique. Multiple
analogies using the same object or objects are not guaranteed
to be similar. This makes it very difficult to generalise
knowledge about making analogies and apply it to any future
analogy-making act.
We propose to tackle this problem of learning to be (computationally)
creative by learning ways to interpret problems,
rather than learning solutions to problems or learning
about objects used in problems. These interpretations can be
learnt, evaluated, recalled and reapplied to other problems,
potentially producing useful representations. This process is
based on the idea that perspectives that have been adopted
in the past and have led to some valuable creative output
may be useful to adopt again if a compatible problem arises.
While even quite similar creative problems may require very
different solutions, quite different problems may be able to
be reinterpreted in similar ways. We discuss this approach
specifically for association and analogy-making but it may
hypothetically apply to other components of computational
creativity. We develop a proof-of-concept implementation in
the domain of computational association, and outline some
ways in which this learning of interpretations could be more
useful than object- or solution-learning in creative contexts.
Models for how previous experiences can influence behaviour
could be a valuable addition to learning in creative
systems. A computational model able to learn ways to approach
creative problems would behave in ways driven by
its previous experiences, permitting kinds of autonomy of
motivation and action currently missing from most models
of computational creativity. For example, it would be possible
to develop a creative system that could autonomously
construct aesthetic preferences based on what it has (or has
not) experienced, or to learn styles by which it can categorise
the work of itself and others, such as described in (Jennings,
2010). A creative system capable using past experiences to
influence its behaviour is a key step towards computationally
creative systems that are embedded in the kind of rich historical
and cultural contexts which are so valuable to human
artists and scientists alike.
Learning interpretations in computational
association
We have previously developed a model of computational association
based on the reinterpretation of representations so
as to render them able to be mapped. Our model, along
with an implementation of it in the domain of ornamental
Proceedings of the Fourth International Conference on Computational Creativity 2013 113
design, is detailed in (Grace, Gero, and Saunders, 2012).
We distinguish association from analogy by the absence of
the transfer process which follows the construction of a new
mapping: analogy is, in this view, association plus transfer.
Interpretation-driven association uses a cyclical interaction
of re-representation and mapping search processes to
both construct compatible representations of two objects and
produce a new mapping between them. An interpretation is
considered to be a transformation that can be applied to the
representations of the objects being associated. These transformations
are constructed, evaluated and applied during the
course of a search for a mapping, transforming the space of
that search and influencing its trajectory while the search
occurs. This differs from the theory of rerepresentation in
analogy-making presented in Yan, Forbus and Gentner 2003
as in our system representations are iteratively adapted in
parallel with the search for mappings, rather than only after
mapping has failed. This permits interpretation to influence
the search for mappings, and for mapping to influence the
construction, evaluation and use of interpretations in turn.
The implementation of this model preented here explores
the process of Interpretation Recollection, through which interpretations
that have been instrumental in creating past associations
can be recalled to influence a current association
problem. This process occurs in conjunction with the construction
of interpretations from observations made about
the current problem.
In the model interpretation recollection is a step in the
iterative interpretation process in which the set of past, successful
interpretations is checked for any interpretations appropriate
to the current situation. These past interpretations
will then be considered for application to the object representations
alongside other interpretations that have previously
been constructed or recalled. A successful interpretation
– one that has previously led to an association – can
thereby by reconstructed and reapplied to a new association
problem. In this paper we demonstrate that this feature
of the interpretation-driven model leads to previous experiences
influencing acts of association-making, and claim that
this is promising groundwork for future investigations into
learning in creative contexts.
In the implementation described in this paper we use simplified
approaches to determining the relevance of previously
successful interpretations and reapplying them to the
current context. The metric for determining appropriateness
is straightforward: any previous interpretation which has a
non-zero effect on a current object representation is determined
to be capable of influencing the course of the current
association problem and included. This simplifies the notion
of ”appropriate for future use” and leads to an obvious
scalability issue, but we demonstrate that this very simple
approach influences behaviour. More sophisticated methods
for determining when and how known interpretations should
be reapplied are an area of future investigation.
Experimenting with learnt interpretations
As a preliminary investigation into the potential of
interpretation-based creative learning, we will demonstrate
that the approach we have developed permits previous experience
to influence the behaviour of an association system.
To illustrate this we will prime the system to produce
different results after having experienced different histories.
In our system previously constructed associations can influence
new association problems through interpretation learning;
past associations can act to “prime” the system to produce
particular results on future associations. By demonstrating
that an association system’s experience with one
pair of objects can influence its behaviour associating different
objects, we show the advantage of interpretation-based
approach to learning. Comparatively an object-based approach
to learning would not have permitted generalisation
to an unfamiliar pair of objects.
In our experiments the system is exposed to a particular
stimulus (either a simple unambiguous association problem
or nothing in the case of the control trial) and then attempts
to solve an ambiguous association problem that is the same
between all trials. Our association system produces many
different mappings between any two objects, so changes in
the distribution of mappings produced on the second problem
is used as an indicator of priming effects.
Three trials were conducted. In the first trial no priming
association was performed, in the second trial a priming
association between Objects 1 and 2 of Figure 1 was performed,
and in the third trial a priming association Objects
1 and 3 of Figure 1 was performed. In each trial an association
between Objects 4 and 5, depicted in Figure 2, followed
the priming stage. Each trial was performed 100 times, with
the system being re-initialised (and re-primed) between each
one so that the histories are identical for every association.
A distribution of the results of the association between Objects
4 and 5 was produced. All trials were conducted using
three relationships: relationships of the relative orientation
of shapes, such as ‘⇠45" difference in orientation’; relationships
of the relative vertical separation of shapes, such as
‘⇠3 units of separation in the Y axis’; and simple binary relationships
when two shapes share vertices.
(c): Object 3
(a): Object 1
(b): Object 2
Figure 1: The three objects used in the priming associations.
An association between either Objects 1 and 2 or Objects 1
and 3 is used to prime the interpretation system.
The two associations used for priming are designed to repeatably
produce a predictable association based on a predictable
interpretation - making them well suited to testing
the impact of priming an association system with that
Proceedings of the Fourth International Conference on Computational Creativity 2013 114
interpretation. The system perceives Objects 1 and 2 and
constructs a simple association based on equating the pattern
of relative rotations between features in Object 1 with
the pattern of shared vertices between features in Object 2.
In the other trial, the system perceives Objects 1 and 3 and
constructs another simple association, this time equating the
pattern of relative rotations in Object 1 with the pattern of
relative vertical positions in Object 3. These associations
are depicted in Figure 3, with the thick dashed lines between
features within the objects denoting relationships that were
mapped, while solid lines between features joining the two
objects denote which features were mapped to each other.
(d): Object 4
(e): Object 5
Figure 2: The two objects used in the test association of
all three trials, which is used to measure the effects of the
priming associations.
These simple associations effectively prime the system
with an interpretation which will predictably bias the dependent
association between Objects 4 and 5. This bias provides
a proof-of-concept test of experiential influence. Future
studies are needed to determine the scope of influences
which historical context can exert in creative systems.
The post-priming association problem used in all three trials
is designed to have two dominant solutions. Over many
runs the system will produce many other associations in addition
to these two, but these will occur relatively often. The
two associations can be seen in Figure 4, with association (a)
being between the radial arrangement of shapes in Object 4
and the similar arrangement of touching shapes in Object 5,
and association (b) being between the same arrangement in
Object 4 and the vertically spaced shapes in Object 5.
It is hypothesised that when the system is first primed with
the association in Figure 3(a) the solution in Figure 4(a) will
be more common (than when unprimed), and that when the
system is first primed with the association in Figure 3(b)
the solution in Figure 4(b) will instead be more common
(than when unprimed). This outcome would demonstrate the
feasibility of using interpretation-based learning to enable a
creative system’s experiences to influence its actions.
Experimental results
The distribution of associations produced in each trial can
be seen in Figure 5. Each of the three bars represents one
i:
~45° Δrot = shared vertex
(a): Trial 2 priming association
~45° Δrot shared vertex
Object 1 Object 2
Rv
(O1
) Rv
(O2
)
i:
~45° Δrot = ~3.0 ΔY
(b): Trial 3 priming association
~45° Δrot
~3.0 ΔY
Object 1
Object 3
Rv
(O1
)
Rv
(O3
)
Figure 3: The solutions to the association problems used to
prime the system in the second and third trials. These simple
problems predictably influence the experiential component
of the creative system in ways that can then be measured.
trial, and each of the three different shading tones represent
a different result, with the darkest tone representing the solution
seen in Figure 4(a), the middle tone representing Figure
4(b), and the lightest tone representing all other solutions.
The latter category included fragmented mappings (those for
which the system could not find a complete mapping of all
the shapes in Object 4) based on relationships such as 90!
and 135! orientation differences as well as similar varieties
and combinations of vertical separation and vertex sharing
relationships. Although they are irrelevant to this investigation
of priming effects, at present this implementation has
no way of evaluating associations other than the number of
features which are mapped. See (Grace, Gero, and Saunders,
2012) for a discussion of the evaluative capabilities of
this model and its current implementation.
It is clear from Figure 5 that priming the association system
with previous problems that rely on compatible interpretations
leads to a significant influence on the outcome of
the association process. Trial 1, in which no priming is performed,
serves as a control against which the frequency of
different associations can be compared. In Trial 2 the system
is primed with a problem that relies on the adoption
of an interpretation equating a pattern of rotational relationships
with a pattern of shared vertices. The result for Trial
2 clearly shows that the frequency of solutions relying on
this interpretation (such as the one seen in Figure 4(a)) has
Proceedings of the Fourth International Conference on Computational Creativity 2013 115
Object 4
Object 5
(b): Trial 3 example dependent association
i:
~45° Δrot = ~3.0 ΔY
~45° Δrot ~3.0 ΔY
Object 4
Object 5
(a): Trial 2 example dependent association
i:
~45° Δrot = shared vertex
~45° Δrot
shared vertex
shared vertex
Rv
(O4
)
Rv
(O5
)
Rv
(O4
)
Rv
(O5
)
Figure 4: Two of the possible solutions to the dependent
association performed in each trial. Solution (a) uses the
same interpretation as used in Figure 3(a), while solution
(b) uses the one found in Figure 3(b).
increased significantly, from 17% in the control to 63% in
Trial 2. In Trial 3 the system is primed with a problem that
relies on equating the same pattern of rotational relationships
with a pattern of vertical separation, shown in Figure
4(b). The result for Trial 3 shows a similarly significant increase
in frequency, with 36% frequency for the primed trial
compared to only 3% in the control.
The difference in absolute frequency of the two associations
shown in Figure 4 can be explained by the underlying
graph structures and the process for searching them
used in our model. The association primed for in Trial 2
is based on the “shared vertex” relationship, which is 50%
more common in Object 5’s graph representation than the
“3.0 difference in the Y axis” relationship used in the interpretation
primed for in Trial 3. For information on how
our system automatically extracts these and other relationships
from vector representations of the objects see (Grace,
0 10 20 30 40 50 60 70 80 90 100
Trial 3: O1
 -> O3
Trial 2: O1
 -> O2
Trial 1: Unprimed
~45° rot -> shared vertex
~45° rot -> ~3.0 Y
Other interpretations
Solutions using each interpretation (percentage)
Trials
Interpretation priming results
Figure 5: The distribution of association results in each trial,
showing the influence of the priming in Trials 2 & 3.
Gero, and Saunders, 2012). The commonality of that relationship
makes mappings that involve features connected
by that relationship similarly more common, which makes
it more likely to be utilised by both the mapping and interpretation
processes. This bias makes the vertex-sharing
relationships much more likely to feature in associations,
but priming the system towards a less common result largely
overrides it. This can be seen in the twelve-fold increase in
the likelihood of the less-common association as compared
to the only three-fold increase in the more common one.
These results show that it is possible for the learning
of interpretations to influence the behaviour of a creative
system, and demonstrate our model of association’s capacity
for interpretation learning and experiential influence on
behaviour. While the influence on behaviour produced in
this implementation is limited, these experiments demonstrate
that interpretation learning can influence behaviour on
problems significantly different than those previously experienced.
This shows the potential for more general learning
than is possible by solution- or object-based methods, making
this approach a valuable building block for modelling
learning computational creativity.
Discussion
The experiments described in this paper are a demonstration
that the behaviour of creative systems can be influenced
by storing and reusing ways to interpret creative problems.
This section discusses the impact on creativity of drawing
from experience to reinterpret a problem and the ways interpretation
can influence creative acts. For a more general
discussion of our model and how it compares to other models
see Grace et. al (2012).
Re-using interpretations for creativity?
There is an intuitive objection to the idea of re-using elements
of a previous creative process: that process, or at least
that element of the larger creative process, cannot by definition
be p-creative. While the process may go on to produce
p- or h-creative outputs, it will at least partially be based on
things that have been experienced previously.
The p-creativity, or lack thereof, of any element of the
creative process does not imply any impact on the creativity
of the final product, but the objection bears discussion:
if drawing on experience will only reduce the creativity of
a process, what is its value? Investigations of the diversity
Proceedings of the Fourth International Conference on Computational Creativity 2013 116
of solutions both with and without priming show that there
is no significant reduction in the breadth of solutions produced,
only in the order in which the system produces them.
This is due to the novelty-favouring behaviour of our model,
which over time discounts and eventually discards solutions
to a particular problem which have repeatedly arisen. Such
intrinsic motivations towards novelty are a necessary component
of learning creative systems, balancing the desire to
repeat the familiar against the desire to explore the new.
(Suwa, Gero, and Purcell, 1999) propose a third element
to Boden’s categorisation of creativity (1992), ‘situational’,
or s-creativity, to describe when an object or process is not
absolutely new to an agent, but is new within the current
situation. This occurs when a familiar idea is considered for
the first time in an unfamiliar context, a common outcome
of analogy-making and a potent component of experiential
learning. This is particularly applicable to the notion of reusing
interpretations, which have the potential to transform
the solution space of the current problem despite not being
a novel process to the agent in question.
Kinds of interpretation and their influence
In the system presented here interpretations are simple transformations
that are stored and re-applied verbatim. However,
the notion that interpretations can influence future acts
does not require that the previously useful interpretation be
literally re-applied to the new context. It would be possible
to develop a system in which exemplary, prototypical or
generalised interpretations could be reconstructed from experience
and applied to the current context.
We define interpretations as a transformations applied to
the objects being associated, but this need not be a direct
transformation of the object representations used by the system.
Other elements of the model could be transformed,
such as evaluative processes, which would change not the
information being used in the creative process but its value
metrics. This could lead to experiential influence on aesthetic
judgement, similar to the idea of autonomously derived
aesthetics proposed by Colton (2011). Alternatively,
representational processes of the model could be transformed,
for example relaxing thresholds for categorisation
or similarity. This could lead to behaviours like satisficing,
a common behaviour of human designers in which requirements
are changed during the creative act (Simon, 1957).
Conclusions
This paper proposes the notion of interpretation-learning –
the storage and recollection of ways to transform problems
– as a complement to more familiar models of object- or
solution-learning. Interpretation-learning is hypothesised as
being of particular utility in creative contexts as each creative
problem is unique in its solutions, but potentially not
in the ways it can be perceived. These remembered interpretations
can be thought of as granting a creative system more
autonomy over its decision making than other means of deciding
how to interpret problems such as provided heuristics
or stochastic processes. We present a simple implementation
of a creative system in which past experiences influence
behaviour through interpretation, to serve as a proofof-concept
of the notion of interpretation-learning. With
this approach demonstrated as feasible and promising, future
work can explore its efficiency and effectiveness.
Incorporating learning is emerging as an important component
of computational creativity due to growing prominence
of desired behaviours like surprise (Maher, 2010),
appreciation (Colton, Goodwin, and Veale, 2012) and autopoeisis
(Saunders, 2012), which necessarily involve past
experience. Learning about specific objects or outcomes
is of limited utility in computational creativity, as creative
problems are by definition unique. However, learning and
recalling different perspectives through which to view objects
is one process by which learning in creative contexts
could be modelled.
<references_biblio/>
References
Boden, M. 1992. The Creative Mind. London: Abacus.
Colton, S.; Charnley, J.; and Pease, A. 2011. Computational
creativity theory: The face and idea models. In
Proceedings of the 2nd International Conference on Computational
Creativity. 90–95.
Colton, S.; Goodwin, J.; and Veale, T. 2012. Full-face poetry
generation. In Proceedings of the 3rd International
Conference on Computational Creativity. 95–102.
Grace, K.; Gero, J.; and Saunders, R. 2012. Constructing
computational associations between ornamental designs.
In Proceedings of the 17th International Conference on
Computer Aided Architectural Design Research in Asia,
37–46.
Jennings, K. E. 2010. Developing creativity: Artificial barriers
in artificial intelligence. Mind and Machines 20:489–
501.
Kavakli, M., and Gero, J. S. 2002. The structure of concurrent
cognitive actions: A case study of novice and expert
designers. Design Studies 23:25–40.
Maher, M. L. 2010. Evaluating creativity in humans,
computers, and collectively intelligent systems. In DESIRE10:
Creativity and Innovation in Design.
Saunders, R. 2012. Towards autonomous creative systems:
A computational approach. Cognitive Computation, Special
Issues on Computational Creativity, Intelligence and
Autonomy 4:216–225.
Simon, H. 1957. Models of Man - Social and Rational. New
York: John Wiley and Sons.
Suwa, M.; Gero, J.; and Purcell, T. 1999. How an architect
created design requirements. In Goldschmidt, G., and
Porter, W., eds., Design Thinking Research Symposium:
Design Representation, volume 2. MIT Press. 101–124.
Yan, J.; Forbus, K.; and Gentner, D. 2003. A theory of
rerepresentation in analogical matching. In Alterman,
R., and Kirsch, D., eds., Proceedings of the Twenty-Fifth
Annual Meeting of the Cognitive Science Society, 1265–
1270. Mahwah, NJ: Lawrence Erlbaum Associates, Inc.
Proceedings of the Fourth International Conference on Computational Creativity 2013 117