Representational affordances and creativity in association-based systems
Kazjon Grace
Faculty of Architecture, Design and Planning
Sydney University
NSW, Australia
kazjon@arch.usyd.edu.au
John Gero
Krasnow Institute
for Advanced Study
George Mason University
Fairfax, VA, US
john@johngero.com
Rob Saunders
Faculty of Architecture, Design and Planning
Sydney University
NSW, Australia
rob.saunders@sydney.edu.au
Abstract
This paper describes ongoing research into associationbased
computational creative systems. The necessary
components for developing association-based creative
systems are outlined, and the challenges in measuring
the creativity of such a system are discussed.
An approach to operationalising creativity metrics for
association-based systems based on representational affordances
is described. This approach is then demonstrated
through an analysis of results produced by a
system for constructing associations between visual designs.
Association-based creative systems
Association, or the construction of a new relationship between
two concepts or ideas, is a cognitive process at the
heart of many creative endeavours. Its presence is most obviously
felt in analogy and metaphor, but associative reasoning
is also a component of complex similarity judgement,
recognition and simplification tasks (Markman and Gentner
1993; Balazs and Brown 1998; Goel 2008) that are critical
to the appreciation of creative works. Given the creative potential
of analogical processes (Goel 1997; Hofstadter and
the FARG 1995; Kuhn 1962) and the importance of understanding
and appreciating creative works (Jennings 2010;
Wiggins 2006; Colton 2008) to a computational creative system,
it is clear that an operationalised understanding of the
process of association that underlies these and other acts is
of value to the field of computational creativity. Furthering
that understanding is a twofold endeavour: computational
models of association that are general, extensible and powerful
must be developed, and metrics by which the creativity
of those models can be assessed must be devised.
Association involves representing objects in a manner that
enables a new relationship between them. A mapping is then
constructed between the objects which embodies that relationship.
These two component processes - representation
and mapping - cannot be modelled serially or discretely, as
representation depends on mapping and mapping depends
on representation (Kokinov 1998). This complex relationship
between the mapping and the representations used in
mapping creates a ‘chicken-or-egg’ problem that must be
addressed by any computational model. Not only must a
computational model of association possess representational
flexibility, but the search for representations must be informed
by feedback from the ongoing search for mappings,
just as the search for mappings is influenced by the construction
of new representations.
Notably the process of association does not incorporate
the use or evaluation of the relationships that it constructs.
This is the primary addition of processes like analogy that
extend association - analogy adds the transfer of knowledge
between the associated object, the use of that knowledge
to achieve some goal, and the evaluation of the analogy
in terms of its utility at achieving that goal (French 2002).
Association-based similarity judgement also extends association,
in this case by evaluating mapped and unmapped
attributes to construct a notion of similarity between the objects
and then using that similarity in some categorisation
or comparison task (Markman and Gentner 1993). Models
of association must be capable of supporting this variety of
applications.
This research has developed the notion of interpretationdriven
search as a general framework for computational association.
We investigate this approach for its potential to
exhibit creative behaviours.
Interpretation-driven association
Donald Schon (1983) proposed a theory, ‘reflection-in- ¨
action’, to explain the cyclical interactions of evaluation and
synthesis processes that had been observed in studies of designers.
Schon suggests that designers change the design ¨
representations with which they are working, then observe
and reflect on the effects of those changes. As a result of
that reflection, the designer again acts to change the emerging
design representation. This iterative interaction is enabled
by the designer’s ability to interpret a representation
in a new way after it has been produced. Schon posits that ¨
the designer’s ability to see things in an emerging design
that were not consciously put there is the core of the creative
design process.
The framework for computational association developed
in this research draws a parallel between Schon’s theory of ¨
creative design and Boden’s (1990) notion of creativity as
exploring (and potentially transforming) a conceptual space.
The actions taken by a designer to modify their design may
translate that design to a new position within the designer’s
conceptual space, or they may transform the space itself, reInternational
Conference on Computational Creativity 2012 195
formulating the designer’s understanding of the problem and
producing a novel and surprising design. The genesis of both
exploratory and transformative reformulations is the reconceptualisation
of the representation the designer had constructed
previously. This produces a new interpretation of
the design on which previously impossible actions are rendered
possible.
Schon sees the process of reflection-in-action as itself ¨
being based on analogical reasoning (Schon and Wiggins ¨
1992), but this research inverts that relationship, putting forward
a framework for association that is based on Schon’s ¨
iterative cycle of reflection and action. This framework is referred
to as interpretation-driven search. While inspired by
the design process, the interpretation-driven search approach
can be generalised beyond design tasks to any domain in
which potentially creative associations are constructed.
Interpretation-driven association uses iterative transformation
and exploration of the objects being associated to
produce a representation that enables a new mapping to be
constructed. An interpretation is a transformation of the
representation of the objects being associated. These transformations
affect the object representations and enable potential
mappings between them to be explored. In this approach,
interpretations are explicitly represented elements of
system knowledge, allowing them to be constructed, evaluated,
stored and retrieved. The interpretation process iteratively
interacts with the process of searching for mappings
and operates in parallel with it. Interpretation influences
mapping search and mapping influences the construction,
application and evaluation of interpretations.
A model of association that implements these principles
can broadly be viewed a consisting of three processes: Representation,
Interpretation and Mapping. Representation
produces the ‘original’ representations of the objects that
are then iteratively searched, transformed and mapped by
the Interpretation and Mapping cycle. This framework can
be seen in Figure 1. The benefits of this parallel, interactive
approach are discussed in Grace et. al. (2012), along with a
more detailed elaboration of the framework.
Representation
Interpretation
Mapping
mappings
objects
association graph representation
interpretations
Figure 1: Interpretation-driven search, a high-level framework
for computational association.
The creativity of associations
The definition of, and criteria for, creativity have been the
subject of considerable debate. One broad definition that has
attained some consensus is that of creativity as the union of
novelty and value (Sternberg and Lubart 1999). Novelty is
a metric based on the difference between the artefact and
other, existing artefacts in the same domain. Value is a metric
based on the artefact’s performance at whatever tasks to
which it is applied, when compared to the performance of
existing artefacts. Both of these qualities are highly contextualised,
as novelty can only be assessed from the perspective
of a viewer and usefulness can only be assessed in the
context of an application.
Some challenges arise in applying this pair of creativity
criteria to the domain of computational association. Firstly,
the novelty of an association is on some level guaranteed, as
by definition an association must be a new relationship that
did not exist previously. Recalling a relationship of which a
system was already aware is a memory task, not an association
one. This makes an association always at least P-novel
(novel to the system itself, as defined by Boden (1990) ).
A significant challenge in applying the “novelty and
value” framework for evaluating creativity to a model of
association is in assessing an association’s value. Association
does not necessarily incorporate an evaluative component
and it is not necessary that an association be constructed
to serve some purpose. We refer to this goal-agnostic form
of association as ‘free’ association, which may incorporate
evaluative components but in which the associations are not
used to accomplish some purpose. Evaluation and purposefulness
are instead features of association-derived processes
that incorporate additional components. This does not mean,
however, ‘free’ association has no effect on the system that
constructed it, and therefore an alternative assessment for
value can be derived. Different associations produce different
transformed and mapped representations of the associated
objects, and their value can be assessed based on the
degree to which those representations go on to affect the
system. This research focusses on this representational affordance
model of association value as a way by which the
model of association that has been developed could be further
developed into an association-based creative system.
Representational affordance as a utility metric
The “affordances” of an object or environment were first de-
fined by the psychologist James Gibson (1979), referring to
the opportunities it offers to a user. As applied to the design
of objects (Norman 2002) affordances refer to the possibilities
for action that a user perceives when interacting with an
object. Affordances do not require instruction, they emerge
implicitly from the interaction of an object, its user and the
situation (Maier and Fadel 2009).
A representation is an internal surrogate that encapsulates
knowledge about an entity, enabling the agent or system to
reason about that thing (Davis, Shrobe, and Szolovits 1993).
In any system that permits the construction of different representations
of an object, those representations will facilitate
the performance of different actions by that system.
Different representations of objects within a system open
up different action possibilities for that system. Gero and
Kannengeisser (2012) refer to this as representational affordance:
the cognitive actions that are enabled by a representing
an object in a particular way. During the design process
a representation may afford the construction of a new representation
with its own, different set of affordances. Gaver
International Conference on Computational Creativity 2012 196
(1991) refers to this as “sequential affordance” and it is consistent
with the notion of reflection-in-action (Schon 1983). ¨
Representations can provide affordances based on their
syntax or based on their semantics. The structure of a graph
representation can provide syntactic affordances, such as
path following or matching. However, a representation can
also provide semantic affordances based on how its content
can interact with other system knowledge. This paper focusses
on semantic representational affordances.
In modelling the creativity of an association, a key question
arises: what is the value of a new mapping and the representations
that underlie it? We define an association’s value
in terms of what activities the possession of that association
enables the system to do. An association can be said to
be of value if the interpretation of the associated objects it
contains provides the system with different representational
affordances than it previously possessed. Furthermore, associations
can be compared and contrasted by the affordances
they provide.
Value can be defined using representational affordances
in the absence of any specific objectives or purpose of the
association construction process, making it apt for use in
a general model of creative association. In the case of an
analogy-making system built on a model of association, the
affordances that would be most relevant would be those that
enable acts of knowledge transfer between the object domains.
By contrast, in a model of design style the most relevant
affordances would be those that permitted the detection
of new patterns that connect stylistically similar objects.
A model of ‘free’ association that does not extend the process
to incorporate a use for the mappings it constructs can
also be assessed using the representational affordance metric
for value. If the goal of a free association system is to
construct as many different associations as possible, then
valuable associations are those that afford the possibility of
future, different associations. This kind of sequential affordance
of association is made possible by association models
that incorporate the effects of a system’s past experiences in
constructing associations into new association tasks.
In this research we use the notion of representational affordances
as a value metric for association models to discuss
the potential creativity of results from a computational
model of association.
Experimenting with interpretation-driven
association
A computational model based on the interpretation-driven
framework for association has been developed. An implementation
of that model which constructs ‘free’ associations
(in that the associations it constructs are not used for any explicit
goals) between ornamental designs is described here.
The structure of the model and its prototypical implementation
are presented here, along with selected association results
produced by the system. The potential representational
affordances of the results presented are discussed as a first
step towards extending this model towards an associationbased
creative system.
Computational model
Interpretation-driven search builds on the model of analogy
as Structure Mapping (Gentner 1983), in which the relationships
within two objects are mapped, rather than their features.
The search for these relationship mappings is integrated
with an iterative process of re-representation.
The model of interpretation-driven association (see Grace
et. al. (2012) for a detailed description) is comprised of
five processes. The first three processes: concept formation,
relation formation and graph construction collectively
form the “representation” process of the interpretation-based
framework, while the latter two processes, mapping and interpretation,
are direct implementations of that framework.
The system begins with an image-based representation
of the objects, extracts a set of features to describe them
and then categorises those features into concepts. Relationships
between these features within each object are then constructed
based on both topological information (such as relative
size, bearing or symmetry) from the feature sets and
typological information from the conceptual categorisation
(such as conceptual similarity or conceptual sameness). The
features and relationships are then compiled into a graph
representation that serves as the basis for the iterative mapping
and interpretation. The mapping process then searches
these graphs for subgraphs that contain common edge labels.
These subgraphs represent regions of the two images
that possess a consistent relational structure.
The transformations that are applied by the interpretation
process affect the structure or content of the object graph
representations. Implementations of this model could utilise
a variety of transformational approaches, such as transforming
the graph objects directly, transforming the features or
concepts directly and then re-constructing the graphs, or
even transforming the process by which one or more representational
stages are constructed.
At any given time, a single transformation is applied to the
graph representations, this is referred to as the ‘current’ interpretation.
This interpretation changes the structure of the
graphs, altering the trajectory of the mapping search operating
on those graphs. The mapping search process produces
candidate mappings as it searches, and these are used to
construct new interpretations. New interpretations are constructed
by examining what features-to-feature mappings in
those candidates cannot currently be successfully mapped,
and extrapolating what transformations would be necessary
to cause those to be successful.
Implementation
The implementation of the model uses vector images as its
input, calculating object features from the minimal closed
shapes formed by vector lines. The kinds of relationship
implemented in the system are ‘same concept’, ‘similar
concept’, ‘relative scale’, ‘linear distance’, ‘horizontal distance’,
‘vertical distance’, ‘relative orientation’, ‘bearing’,
‘contains’, ‘reflection of’, ‘shared vertex’ and ‘shared edge’.
The implementation is provided with the knowledge necessary
to detect these relationships and categorise them into
groups such as ”slightly smaller than” or ” 120 degrees of
International Conference on Computational Creativity 2012 197
difference in orientation”. Instantiations of these relationships
form the edge labels on the graph representations of
each object being associated.
Mapping search is implemented as a genetic algorithm
that searches for subgraph isomorphisms between the graph
representation of each object. Each individual in the population
of the genetic algorithm is a set of mappings between
a feature in one object and a feature in the other. The fitness
for this algorithm is the largest contiguous subgraph
that can be constructed out of those feature-to-feature mappings
in both objects. This use of a powerful, general search
algorithm reflects the fact that we are not attempting to implement
association in a biologically or cognitively plausible
way, rather we are demonstrating the feasibility of the
interpretation-driven approach.
The interpretation process is implemented as the substitution
of relationships between features. Replacing relationships
effectively causes the system to perceive two disparate
relationships as being alike. Interpretations in this system
can be expressed as “in this situation, relationship X in the
first object is the same as relationship Y in the second object’.
An interpretation is therefore a set of rules for replacing
relationships, where relationships are represented as
edge labels in the graphs. Which interpretation is being applied
to the objects is able to change every iteration, providing
the parallelism between mapping and interpretation that
characterises the interpretation-based framework.
Methodology
A total of 31 ornamental designs were inputted into the system
as part of a series of experiments to demonstrate the application
of interpretation-based association. Objects were
drawn from a broad variety of design domains, including
symbols, architectural ornamentation and decorations and
object designs. These objects were drawn from a variety of
cultures and historical periods. From this library of designs
a subset of objects were selected for which interesting associations
could be produced and the capabilities of the system
could be documented.
A set of associations constructed by repeatedly associating
a single pair of objects is presented. These associations
are presented as a demonstration of the interpretationbased
model, but also as a starting point from which the
use of representational affordances as a metric for utility in
association-based creative systems can be discussed.
The two objects associated here are presented in Figure 2.
Object 1, on the left, is a Hittite sun symbol, while Object 2,
on the right, is a Japanese floral symbol. Both are vector line
drawings produced manually from black and white images
by the authors. For the purposes of this experiment the system
has been restricted so that the only type of relationship
which connects the features of these two objects is relative
orientation.
Results
Three associations between the two objects in Figure 2,
along with the interpretations used to produce them, are
shown in Figures 3, 4 and 5. All three associations constructed
between these two objects utilised the ‘relative ori-
(a): Object 1 (b): Object 2
f1
f2
f4
f5
f6
f8 f7
f9
f10 f11
f12
f13
f14
f15 f16 f17 f18 f19
f20
f21
f22 f23 f24
f25
f26
f27
f28
f36
f29
f30
f31
f32
f33
f34 f35
f3
Figure 2: The two objects used in the example associations.
The minimal closed shapes extracted from the images by the
system are numbered. The relative orientations of these features
are the relationships relevant to these examples. Designs
sourced from (Humbert 1970).
entation’ relationship type, but each involved a different interpretation.
These differing interpretations permitted the
construction of different mappings.
In each of these figures the associated objects are presented
side by side, with the features involved in the mapping
being highlighted. The mapping between features in
one object and features in the other is shown as solid lines
joining the two images. The common set of relationships
between the features within each object is shown as thick
dashed lines. Only the relationships that are used in the
mapping are shown, pairs of features can have many relationships
connecting them. Each of these relationships is labelled
with its uninterpreted description. Interpretations are
an imposed equality between different labels and are shown
at the bottom of each image. Mappings can be constructed
between sets of features that share patterns of relationships
after this interpretation is applied.
The first association, seen in Figure 3, is constructed without
the use of an interpretation. Within the representations of
the two objects there exists a pattern of seven nodes in each
that share a consistent pattern of relationships. All seven
objects in both objects, in the order indicated by the thick
dashed lines, are consecutively separated by approximately
150 degrees of orientation. This relationship is present between
every second point in the seven-pointed star in Object
1, starting from Feature f4 and proceeding twice around the
star to Feature f9. The same relationship of relative orientation
is present between every eighth petal in Object 2 - that
is between each petal and the petal one spot to its left in
the floret to its left, starting from Feature f35 and proceeding
in a spiralling pattern twice around the design to f29.
The ‘null’ interpretation i; is shown as this mapping can be
constructed from the base representations produced by the
system without any transformation.
This association is included to demonstrate the capability
of the representation construction and mapping search
elements of our model of association. Without the use of
interpretations the system is capable of producing representations
of visual objects comprised of networks of abstract
International Conference on Computational Creativity 2012 198
Object 1 Object 2
f4
f5
f6
f8 f7
f9
f10 f16
f19
f24
f27
f29
f32
f35
Interpretation:
i
0
Figure 3: An association constructed between the two objects without the use of an interpretation. All the relationships incorporated
into this mapping (depicted by thick dashed lines) are of the type “⇠150 degrees of difference of orientation”. These
relationships join the seven points of the star in Object 1 (by traversing the star twice) and seven of the petals in Object 2 (in
a spiral pattern joining each eighth petal). The empty or “null” interpretation means that these relationships are present in the
default object representations the system constructs.
relationships and features. These representations can then
be searched for common patterns of relationships, allowing
the features which those relationships join to be mapped.
Without the ability to transform representations, this association
(and others trivially different from it) are all that the
system can construct. With mappings limited to those relationships
already present in both objects, the potential for
constructing an association with relevant affordances is slim.
The only way to give a system with a single representation of
each object the ability to construct additional associations is
to incorporate more information into those representations.
In this case that information would take the form of additional
types of relationship between features other than differences
of orientation.
Utilising the capacity to reinterpret object representations,
the system is not limited to “literal” associations as seen in
Figure 3. Figure 4 shows the mapping produced by an association
between the same two objects that was produced
through interpretation-driven search. During the search for
mappings between the objects, the system constructed (initially
by chance) some fragment of a mapping similar to the
one shown in Figure 4. This mapping candidate would not
have been successful in the absence of an interpretation. The
mapping was selected by the interpretation construction process,
which reverse-engineered one or more interpretations
from it. Interpretations are generated that would improve the
size of the largest common subgraph of the mapping specified
by the candidate. That interpretation is then likely to
become the “active” one if the mapping search reaches a
point where the current interpretation is significantly outperformed
by the new interpretation. The search for mappings
influences the construction of interpretations and then
those interpretations in turn influence mapping.
The mapping expressed in Figure 4 is based on an interpretation
that effectively treats the orientation difference
between adjacent points on the star in Object 1 the same
as the orientation difference between adjacent points on the
star in Object 2. Thick dashed lines are shown representing
the “approximately 50 degrees of orientation difference”
relationship in Object 1 and the “approximately 20 degrees
of orientation difference” relationship in Object 2. The resultant
mapping connects each feature in the star in Object
1, starting with f4 and proceeding sequentially to f10 with
each feature in a floret in Object 2, starting with f14 and proceeding
sequentially to f20. This mapping was constructed
from low-level relationships extracted from a visual representation
of these objects and then interpreted to make those
relationships situationally alike.
The notion of representational affordances as a tool for
assessing value in association models can be applied to the
association shown in Figure 4. The interpreted representations
of the two objects are useful to the extent that the new
interpretations more aptly afford actions to the system. In
the case of a free association model like this implementation,
the only actions available are the construction of different
associations, and therefore the value of an association
can only defined by the degree to which it enables that.
In this system interpretations are remembered and re-used,
which causes the system’s past experiences to affect future
interpretations. This provides a mechanism by which this
association can guide the system’s future actions.
Representational affordances provided by the association
International Conference on Computational Creativity 2012 199
Object 1 Object 2
f4
f5
f6
f8 f7
f9
f10
f14
f15
f16 f17 f18
f19
f20
Interpretation:
~50° Δrot = ~20° Δrot
~20° Δrot
~50° Δrot
~50° Δrot
~50° Δrot
Figure 4: An association constructed between the two objects through the application of an interpretation which equates the
relationship “⇠50 degrees of difference of orientation” in Object 1 to the relationship “⇠20 degrees of difference of orientation”
in Object 2. These relationships join the adjacent points of the star in Object 1 with eight adjacent petals in one of the florets in
Object 2. The interpretation that enabled this association is shown in the box beneath the objects.
in Figure 4 could be used to define the value of that association
if the system were extended to perform purposeful
association construction. If associations were being constructed
for use by an ornamental design classification system,
then the transformed representation may afford the possibility
of classifying Object 2 as being based on radial adjacency,
which a naive classification system would have been
able to do. If instead associations were being constructed for
use in an analogy-based design system, then the mapping of
f4 through f10 to f14 through f20 may afford the transfer
of knowledge about f11 (the centre circle which all of the
mapped features in Object 1 touch) to f13, which could then
be considered the “centre” of Object 2. The actions permitted
by the representational affordances of the association are
used by the system to achieve its goals, so those affordances
can be said to be of value because of that use.
Figure 5 shows a different association constructed by the
system. This time the interpretation equates the difference in
orientation between every third point in the star in Object 1
with the difference in orientation between the edge petals in
Object 2. Viewing the “edges” of a compound object such
as Object 2 as being part of a rotating sequence may be a
valuable affordance for a creative system. The association
system implemented in this research was able to find a broad
variety of such associations using just two objects and considering
just one type of relationship. Different combinations
of the relationships that were mapped in Figures 4 and
5 were also found, such as mapping every adjacent point on
the star in Object 1 to the outermost petals of the three florets
in Object 2. Interpretation construction provided the system
with the ability to produce a variety of divergent mapping
from a single association problem.
Discussion
The experiments described here demonstrate that it is possible
to use the interpretation-driven search approach to construct
associations between real-world design objects. The
representation construction processes used to produce the
graphs on which the iterative mapping and interpretation
processes operate have been shown to be viable. Associations
were produced based on interpretations that were constructed
by the system that transformed graph representations
that had also been constructed by the system from features
and concepts that had been extracted from low-level
visual input. These results serve as an initial proof of concept
of the interpretation-driven model of association.
The associations presented in this paper could not have
been constructed from the information that was provided to
the system without the ability of the system to transform its
representations through the interpretation process. These associations
could be constructed without the use of interpretation
if the system were provided with additional information
about the relationships between the objects, but this reduced
representational autonomy would have a deleterious
effect on novelty. As assessed in the context of a hypothetical
society of individuals with access to the same information
and possessed of comparable perceptual abilities, associations
produced using interpretation will be P-novel to
any individual that has not constructed the same interpretation,
while associations produced using additional information
would be apparent to any other individual with access
to that information.
The P-novelty of an association is guaranteed as the system
by definition did not know of the relationship expressed
in an association before its construction. However, the interInternational
Conference on Computational Creativity 2012 200
Object 1 Object 2
f4
f5
f6
f7
f9
f10
f14 f20
f22
f28 f29
f35
Interpretation:
~150° Δrot = ~120° Δrot
~150° Δrot
~120° Δrot
Figure 5: An association constructed between the two objects through the application of an interpretation which equates the
relationship “⇠150 degrees of difference of orientation” in Object 1 with the relationship “⇠120 degrees of difference of orientation”
in Object 2. These relationships join every third point of the star in Object 1 with the two outermost petals in each
floret in Object 2. As there are only six such petals in Object 2, only six of the seven points in Object 1 have been mapped. The
interpretation that enabled this association is shown in the box beneath the objects.
pretations used in the model of association described in this
research may not themselves be novel as they can be learnt
and re-used. It is possible to apply a known interpretation to
a different object and still produce a novel representation, as
a known transformation can produce a novel result.
It is also possible that a P-novel association can be constructed
from a representation that has been used before. For
example, if the representation of Object 1 present in 4 (and
the interpretation used to produce it) were associated with
some other, different object, then the resulting association
would be P-novel. The known representation of Object 1 is
novel in the current circumstances, but is not novel to the
system as a whole. This is referred to as “situational” or “S”
novelty, after the definition of “S-creativity” in Suwa et al.
(1999).
Figure 3 demonstrates an inherent weaknesses of a model
of creative association that does not incorporate the ability
to reinterpret its object representations. The mapping it expresses
is quite likely H-novel, in that it is not expected that
the majority of human observers would have identified that
mapping. However, any sufficient pattern matching system
provided with the two graph representations used by the system
would come to the exact same conclusion. The mapping
used in this association may be H-novel in a society composed
of individuals with human perceptual biases, but to a
hypothetical society composed of individuals with a perceptual
system like that of this model, the mapping is obvious.
The use of an interpretation in the construction of associations
acts to redress the weakness present in models of creative
association based on static representations. The association
expressed in Figure 4 could not be constructed from
the graph representations the system built without the use
of a representation transformation process. In a hypothetical
society of individuals with the same perceptual biases as
this system, the mapping used in this association would be
P-novel to any individual that had not constructed the same
interpretation. If this same mapping were to be constructed
by incorporating a new relationship type in the default representations
that made the mapping possible without interpretation
- “radial adjacency” for example - then the resultant
mapping would, like the one in Figure 3, be trivially deducible
by any other pattern matching system with access to
the same information.
The utility component of evaluating creativity can be
aided by the use of representational affordances. In the
interpretation-driven approach to association, mappings are
produced between transformed representations that can reveal
structures and connections that were not apparent in the
original representations. The utility of those mappings can
then be assessed by what actions those new structures enable
the system to take. For example, the view of the outermost
petals in Object 2 as a sequence of rotated features seen in
Figure 5 was not expressed in the uninterpreted representation.
The notion of representational affordance also allows
the value of an association to be defined in the abstract for
models of association that do not use the associations they
construct to accomplish any objectives.
Maher (2010) frames the evaluation of a creative artefact
as requiring three criteria; not just novelty and value but also
unexpectedness. Unexpectedness (also referred to as ‘surprisingness’)
is a metric based on how different the artefact
is to what was expected to be the next artefact produced.
International Conference on Computational Creativity 2012 201
Unexpectedness, writes Maher, differs from novelty in that
it relates to the expected trajectory of the domain or field in
which the artefact is being produced, which is distinct from
the existing set of artefacts within that domain.
Assessing the unexpectedness of an association using Maher’s
criteria requires identifying abstract patterns in the sequence
of recently constructed associations that can be used
to project a trajectory of expected associations. Associations
can certainly be surprising in a variety of ways - much humour
depends on setting the recipient up to expect that a
certain association is being proposed, then subverting that
expectation and instead constructing a very different association.
However, unexpectedness as defined by Maher specifically
refers to identifying emerging trends in the output of
a system over multiple iterations. This is a challenging task
in the domain of association as it is difficult to define a similarity
metric that could then be used to find patterns in association
output.
The interpretation-based model of association could provide
a method by which the unexpectedness of associations
could be assessed. Interpretation-based associations could
be characterised by the interpretations used to construct
them, which are significantly more generalisable than the
mappings that comprise those associations. The explicit representation
of interpretations permits further investigation of
expectation and unexpectedness in computational models of
association. This would address a challenging issue in the
development of creative models of association and methods
by which they can be evaluated.
The representational affordance framework for assessing
the value of an association allows us to consider the associations
constructed by an interpretation-based system as a
creative artefact. The utility of that artefact is the degree to
which it has an effect on the system that constructed it. In
a “free” association system where experience plays a role
in future associations then that effect can be defined as the
influence an association has on the construction of future associations.
<references_biblio/>
References
Balazs, M. E., and Brown, D. C. 1998. A preliminary investigation
of design simplification by analogy. In AI in Design
Conference. Kluwer.
Boden, M. 1990. The Creative Mind. Abacus.
Colton, S. 2008. Creativity versus the perception of creativity
in computational systems. In Proceedings of the AAAI
Spring Symposium on Creative Systems.
Davis, R.; Shrobe, H.; and Szolovits, P. 1993. What is a
knowledge representation? AI Magazine 14(1):17–33.
French, R. 2002. The computational modeling of analogymaking.
Trends in Cognitive Sciences 6(5):200–205.
Gaver, W. 1991. Technology affordances. In Robertson,
S.; Olson, G.; and Olson, J., eds., Proceedings of the
SIGCHI Conference on Human Factors in Computing Systems:
Reaching through Technology, 79–84.
Gentner, D. 1983. Structure-mapping: A theoretical framework
for analogy. Cognitive Science 7:155–170.
Gero, J., and Kannengiesser, U. 2012. Representational
affordances in design, with examples from analogy making
and optimization. Research in Engineering Design (to appear).
Gibson, J. 1979. The Ecological Approach to Visual Perception.
Houghton Mifflin.
Goel, A. 1997. Design, analogy, and creativity. IEEE Expert
12(3):62–70.
Goel, A. 2008. Analogical recognition of shape and structure
in design drawings. Artificial Intelligence for Engineering
Design, Analysis and Manufacturing 22:117–128.
Grace, K.; Gero, J.; and Saunders, R. 2012. Constructing
computational associations between ornamental designs. In
CAADRIA 2012. to appear.
Hofstadter, D., and the FARG. 1995. Fluid Concepts and
Creative Analogies: Computer Models of the Fundamental
Mechanisms of Thought. New York, NY: Basic Books.
Humbert, C. 1970. Ornamental Design. Thames and Hudson.
Jennings, K. E. 2010. Developing creativity: Artificial barriers
in artificial intelligence. Mind and Machines 20(4):489–
501.
Kokinov, B. 1998. Analogy is like cognition: dynamic,
emergent, and context-sensitive. In Holyoak, K.; Gentner,
D.; and Kokinov, B., eds., Advances in analogy research:
Integration of theory and data from the cognitive, computational,
and neural sciences. NBU Press. 96–105.
Kuhn, T. 1962. The structure of scientific revolutions. University
of Chicago Press.
Maher, M. L. 2010. Evaluating creativity in humans, computers,
and collectively intelligent systems. In Proceedings
of the 1st DESIRE Network Conference on Creativity and
Innovation in Design. Lancaster, UK: Design Network.
Maier, J., and Fadel, G. 2009. Affordance based design: A
relational theory for design. Research in Engineering Design
20(1):13–27.
Markman, A., and Gentner, D. 1993. Splitting the differences:
A structural alignment view of similarity. Journal of
Memory and Language 32(4):517–535.
Norman, D. 2002. The Design of Everyday Things. New
York: Basic Books.
Schon, D., and Wiggins, G. 1992. Kinds of seeing and their ¨
functions in designing. Design Studies 13(2):135–156.
Schon, D. 1983. ¨ The Reflective Practitioner. Basic Books.
Sternberg, R., and Lubart, T. 1999. The concept of creativity:
Prospects and paradigms. In Sternberg, R., ed., The
Handbook of Creativity. Cambridge University Press. 3–15.
Suwa, M.; Gero, J.; and Purcell, T. 1999. How an architect
created design requirements. In Goldschmidt, G., and
Porter, W., eds., Design Thinking Research Symposium: Design
Representation, volume 2. MIT Press. 101–124.
Wiggins, G. 2006. A preliminary framework for description,
analysis and comparison of creative systems. Journal
of Knowledge Based Systems 19(7):449–458.
International Conference on Computational Creativity 2012 202