On the Impact of Chat Communication on Computer-
Supported Idea Generation Processes
Florian Forster1, Marc René Frieß1, Michele Brocco1
and Georg Groh1,
1 Technische Universität München, Lehrstuhl XI, Boltzmanstr. 3,
85748 Garching, Germany
{forster,friess,brocco,grohg}@in.tum.de
Abstract. It has been shown that traditional forms of communication negatively
affect idea generation processes. Creativity support systems can help to avoid
these impacts and provide alternative means of interaction for which the known
negative affects do not apply. In this paper, we investigate the impact of chat
communication on computer-supported idea generation processes. The results
show that idea quantity, quality and group satisfaction are not affected by the
option to use a chat as an additional communication channel. This implies that
positive as well as negative influence factors of communication described in
previous studies seem to offset each other in computer-supported idea
generation processes. We discuss several of these potential positive and
negative influence factors. Furthermore, we discuss why a chat feature can help
to lower acceptance barriers for creativity support systems.
1 Introduction
In the 1950s, Alex Osborn [1] proposed four central guidelines for groups to follow in
idea generation processes: Criticism is ruled out, freewheeling is welcome, quantity is
wanted and combinations / improvements are sought. As of today, brainstorming is
probably the most popular and most often applied (group) creativity technique for
idea generation processes. According to a study of Fernald and Nickolenko [2], 92%
of the American companies use the brainstorming technique in meetings.
Studies on the effectiveness of this technique consistently show that brainstorming
groups yield better results than groups conducting traditional meetings [3]. This is
mainly due to the strict separation between a divergent phase, where ideas are only
generated but not yet discussed, and a consecutive convergent phase, where ideas are
evaluated.
Taylor et al. [4] made the finding that for brainstorming, nominal groups1 are more
effective (in both quantity and quality of ideas) than groups where the participants
communicate. Conducting creativity techniques as a real group process seems to
imply some negative consequences that outbalance potential benefits. Three factors
can explain the decrease of efficiency for interacting groups [3]:
2 In nominal groups, participants work separately from each other and at the end their ideas are merged.
165
1. Group pressure: Fear of judgment by the other group members and power
misbalance (e.g. when hierarchies exist) inhibit participation and can lead to
unwanted conformity of idea proposals.
2. Social loafing: Social loafing describes the tendency of group members to do
less than their potential would allow them to do. It can occur either if a group
member feels isolated from the group or if he feels too submerged.
3. Production blocking: Diehl and Stroebe see production blocking as the
dominant factor for efficiency losses in group brainstorming processes [5].
Production blocking refers to the fact, that in interacting groups only one
member can speak at a time, while the others have to listen; hence all but one
member of the group are blocked and cannot work on their own ideas in this
time.
Computer support for idea generation can help to mitigate the negative effects of
interacting groups. Carte et al. [6] state that due to the fact that a creativity support
system (CSS) is able to anonymize the users and their contributions, the group
pressure on the participants is lowered. Shepherd et al. [7] have shown that by
improving the participation awareness in the process, social loafing effects can be
reduced. Finally, when the team members do not communicate verbally, they can
enter ideas parallely using keyboards and doing so are not interrupted by others.
That's why the production blocking does not occur in computer-supported idea
generation processes. All these factors lead to improved team effectiveness in idea
generation sessions [8].
The studies mentioned above clearly indicate that it is preferable to avoid than to
allow verbal communication in idea generation processes, which can be well
explained with the effect of production blocking. However, this does not imply that
other kinds of communication that enable direct communication between the
participants must have a negative impact on the team effectiveness. E.g. using
communication means that to allow parallel (non-blocking) communication such as a
chat could be an improvement, as suggested in recent studies on computer supported
group work (see section 2). In this paper, we want to present and discuss an empirical
study we conducted on the research question whether a chat is beneficial for computer
supported idea generation processes.
2 Communication in idea generation processes
An idea generation process is a series of activities leading to creative ideas.
According to Sternberg [9], an idea is creative if it is “both novel (i.e. original,
unexpected) and appropriate (i.e. useful, adaptive concerning task constraints)”.
Creativity techniques are guidelines that structure idea generation processes, e.g. by
defining distinctive phases or by regulating the participants’ behavior. These
restrictions often affect the communication between the group members, as is the case
for the brainstorming technique, where criticism is forbidden during idea generation.
The strong empirical evidence on the effectiveness and result quality of creativity
techniques with strong restrictions on communication (such as brainstorming)
166
indicates that direct communication may not be a positive or necessary factor for idea
generation processes.
Applegate et al. [10] investigated a group decision support system for idea
generation and issue analysis in organization planning. The participants could
exchange ideas using the system, but had no means of direct communication via the
system. Even though the participants were allowed to communicate verbally, the
authors observed that approximately 96,6 % of the participants' time was spent on
working on ideas using the system and only 3,4 % was used for non-electronic group
interaction and communication. From this non-electronic communication, a large
majority (47,54 %) was relatively short and technology-oriented. They also noted that
the majority of verbal interaction was directed to the session facilitator (57 %). Given
these small percentage values for "true" task-related direct group communication acts,
one could assume that the overall role of direct communication in the process is
neglectable.
Hilliges et al. [11] investigated the effects of using a tabletop interface in
combination with a large wall display for face-to-face group brainstorming. They
compared the results of this setting with control groups using the traditional paperbased
method. When counting the groups’ ideas for the analysis of the experiment,
they differentiated between new independent ideas, ideas that built on their own
earlier ideas, ideas that resulted from seeing somebody else write down an idea and
ideas that resulted from talking about an idea. 29 % of the ideas of the paper-based
groups and 26% of the ideas of the electronically-supported groups ideas emerged as
a result of a communication process. This implies that direct communication is the
source of a substantial percentage of ideas which (in contrast to the studies and
arguments presented above) may advocate the point of view that direct
communication is beneficial in creative processes.
Comparing more than 40 studies on teams that interact mainly or exclusively using
computer systems, Powell et al. [12] come to the conclusion that communication is
the key success factor in virtual team situations. They argue for a high media-richness
of the communication channels, which contrasts with the principle of creativity
techniques for idea generation processes that have the tendency to limit the
communication channels to allow only task-related idea exchanges and to avoid direct
communication between the team members.
Summarizing major findings in GSS reserach, Nunamaker et al. [13] point out that
support systems increase the number of ideas generated during a divergent
(generation) process. The participation tends to be more equally distributed than in
traditional meeting scenarios, which is mainly due to anonymity and the possibility to
input ideas parallely. While Nunamaker refers only to parallel input of the generated
ideas, we want to investigate the impact of a chat as a channel for direct
communication between the participants of an idea generation process.
In conclusion, theories and studies of creative processes give ambiguous signals
with respect to the question, whether direct communication, as provided by a chat
actually supports idea generation processes or not.
167
3 Study
3.1 Setting
In order to find out more about the impact of chat communication on computersupported
idea generation processes, we conducted an experimental lab study. As
participants we selected computer science students, mainly pursuing their Bachelor's
degree. A total of 60 students, divided into 18 different groups, took part in the
experiment.
The groups were composed randomly by picking the students who had signed up
for an experiment date and dividing them into groups of four persons each. This was
done in advance. Due to the fact that not all people showed up, the final groups split
up into twelve groups with three students each and six groups with four students.
Each participant had his own PC, and all PCs were set up with the same configuration
(Windows XP, Firefox web browser). Any kind of explicit (e.g. verbal, visual)
communication outside of the tool was strictly forbidden. A dedicated facilitator
monitored the strict adherence to this rule.
3.2 Experiment
The students used a creativity support system named IdeaStream to find ideas for the
given problem “For which purpose could the student fees be used?”. IdeaStream
allows teams to collaborate on a virtual whiteboard using a broad set of different
creativity techniques [14]. The user interface is shown in figure 1.
Fig. 1. Screenshot of the virtual whiteboard and the chat in IdeaStream.
There, people can work collaboratively on their ideas by creating new ones,
changing, moving or deleting them. All ideas on the whiteboard are publicly
168
accessible, but without any information on who created or changed them. An idea is
represented by a title and a set of components called aspects that represent pieces of
information that in turn compose the idea. These pieces of information consist of
texts, uploaded images or sketches. The participants used pseudonyms of the form
"user x" that were randomly assigned by the system.
In half of the sessions, the groups were able to chat with each other by using the
integrated chat-function. This set consisted of a total of 29 students partitioned into
seven groups of three students and two of four students. The other half (31 students)
had no means of direct communication at all. Those students were divided into five
groups of three and four groups of four.
At the beginning of each session, the facilitator explained the user interface and the
applied creativity techniques. After that, a 10-minute test case was played through, in
order to let the participants get familiar with the user interface and the features of the
tool. After this introduction the main part of the experiment took place.
The idea generation process of the experiment consisted of three connected phases,
each following a different creativity technique for idea generation (Brainwriting,
Unrelated Stimuli and Forced Combination, see [15] for detailed explanations of the
techniques). The duration of each phase was 10 minutes, so each group spent 30
minutes for idea generation, which is a typical period for idea generation sessions [18]
[19]. During those phases, all participants’ activities were logged. After completion of
all phases, the students evaluated their generated ideas with respect to creativity and
feasibility by using a score from 0 (worst) to 4 (best).
4 Results
While analyzing the results of the lab study, we were particularly interested in idea
quantity, idea quality and participant satisfaction. When describing the results, we
refer to the groups having a chat with “chat” and the groups having no chat with “nochat”.
4.1 Idea quantity
Chat produced a total of 172 ideas, while no-chat generated 215 ideas. In relation to
the number of participants, chat averaged 5.9 ideas per group member, being slightly
outscored by no-chat with 6.9 ideas per participant.
4.2 Idea quality
The members of chat rated their ideas with respect to creativity with an average of 2.2
of 4 points , with respect to feasibility with an average of 2.6 of 4 points. No-chat
evaluated their idea's creativity with an average of 2.0 and their idea's feasibility with
an average of 2.3. So for both criteria, chat assessed slightly higher scores than nochat.
For an external measure, we asked three researchers from our group to rate the
ideas as objectively as possible. In this external rating, chat scored 2.1 for both
169
creativity and feasibility, while the ideas of no-chat were rated with an average of 2.2
for creativity and 2.1 for feasibility. So the external rating showed no significant
difference in the quality of contributions between the two groups.
4.3 Participant Satisfaction
In a survey that was conducted immediately after the experiment, the participants
were able to suggest improvements for the IdeaStream application. In 7 of the 9
groups of no-chat, at least one member requested a chat. However, both groups
equally enjoyed working with the IdeaStream tool (4.6 of 6, where 0 means worst and
6 means best). No-chat rated their satisfaction with the group slightly higher than chat
(5.0 / 4.8 of 6).
Table 1. Results of experiment comparing groups with chat and without chat during computersupported
creativity techniques for idea generation.
4.4 Summary
Even though there are numerical differences in the means of most of the relevant
variables, t-tests showed that they are not statistically significant. Hence, the
hypothesis that chat communication affects a collaborative idea generation process is
not supported (H0 saying that there is no difference cannot be rejected). This is true
for idea quantity, idea quality (internal and external) and group satisfaction.
5 Discussion
The design of the IdeaStream application and its use and configuration in our
experiment provide means to counteract the three well-known negative influences that
are usually credited for negatively impacting collaborative creative processes. The
user awareness functions can help to inhibit social loafing. To prevent negative effects
from social pressure, the users had pseudonyms assigned. Hence, the fear of negative
judgment from others was lowered. Lastly, production blocking effects were
minimized, since the system allows parallel input of ideas (and, for the chat-groups,
of chat messages) and oral communication was strictly forbidden. In this way we
provided an environment in which the known negative influence factors on group
170
creativity are mostly suppressed. Based on this, we were able to investigate the
question if groups in idea generation processes can benefit from a chat as a channel
for direct communication besides the means to enter ideas and view the ideas of
others.
5.1 Impact on the process
Our experiment shows that neither overall quality nor quantity of the ideas that are
generated by groups in computer-supported creativity techniques are influenced by
the group having the possibility to chat. So, looking at the overall process results,
there is no net impact of this additional direct communication at all. However, this
does not imply that the process itself is not influenced by the communication. The
theories presented in the previous chapters provide reasonable arguments on these
effects and support the assumption that communication does actually affect the
creative group process. However, our experiment showed that different process
effects implied by communication definitely cancel each other out in creative group
processes, since both settings yielded the same results.
To be able to better support the creative process with creativity support systems, it
will be necessary to gain an understanding on the factors which influence
communication. As a starting point for future research, we summarized potential
positive and negative influence factors, which emerge from direct observations in the
experiment as well as personal considerations (see figure 2).
Fig. 1. Potential positive and negative influence factors of communication on
computer-supported idea generation processes.
Potential positive factors:
1. Information sharing: Letting others participate in your knowledge on a particular
subject may enable them to create new or improved ideas. The stimulus through direct
communication between the participants may be different from and additional to the
stimulus received by seeing other people's ideas. Furthermore, people may have an
information need with respect to the given problem that they can express via a
channel such as a chat. That may also help in producing good quality ideas for that
particular problem.
2. Socio-Emotional effects: It can be assumed that direct communication can improve
emotional states of mind in a very general way, not only with respect to the initial
171
acceptance of the tool (see below) but also with respect to e.g. expressing and thus
alleviating temporary emotional indispositions.
3. Consensus or commitment building: While in idea generation processes consensus
or “general acceptability” with respect to ideas may not be necessary or in most cases
even unwanted, direct communication may contribute to a certain consensus or
commitment to some ideas that may give participants incentives to improve, structure
or reposition the idea in question during the creative process without diluting it or
neglecting alternative ideas.
Potential negative factors:
1. Softening and dilution of ideas: An effect that applies to all forms of “free”
communication in creative processes and therefore may also affect chat is that
communication may contribute to softening and dilution of innovative radical ideas.
One possible reason can be the attitude in groups that induces the desire for
agreement on proposed solutions. In order to reach this state the described softening
on ideas in view of general acceptability often takes place which sometimes leads to a
decreased quality or novelty compared to the original ideas.
2. Distraction: While it can be assumed that a chat is a parallel medium and
electronically mediated communication requires fewer adherences to social norms
that enforce listening to others while they speak, participants still have to take their
time to read the chat contributions, which may distract them from their actual task of
idea generation.
3. “Unprofessional” use: In the groups that were able to chat we observed tendencies
for unfocused behavior such as joking, which may exert a social force on other
participants to join this unfocused and thus potentially distracting behavior.
5.2 Impact on acceptance
As our survey shows, the majority of the groups that were not able to communicate
with a chat suggested improving the system by adding a chat feature. This is in
particular interesting given the fact that, after all, these groups were not less satisfied
with the computer support system or the group than the groups that actually had the
chat feature, so we must assume that actually providing them a chat would not have
positively affected their satisfaction level. Nevertheless, there seems to be an inner
need to communicate in group settings in general or the need for a communication
assurance in view of the fear of having to use an unknown tool together with the
pressure to produce good ideas.
As Dennis and Reinicke [17] point out, acceptance of creativity support systems in
practice is still weak, despite of the positive research results in the field. Our
experiment findings suggest that there is a strong a priori demand for having a chat in
a creativity support system, so providing a chat can help lowering acceptance barriers.
172
Conclusion
In this article we presented the results of a study concerning the use of chat for
communicating in computer-supported idea generation processes. First we introduced
some of the typical communication problems in interacting groups and showed how
using creative support systems can decrease the impact of these factors. Then we
reviewed what role recent literature assigns to communication in idea generation
processes. Related work gave valid arguments supporting both theories that direct
communication via a chat during idea generation processes is necessary and beneficial
and that direct communication has negative influence and thus has to be avoided
respectively. For this reason we conducted an experiment that resulted in
approximately equal performance values for teams having chat as a communication
tool and teams without chat function. In the discussion we then interpreted our results
and addressed possible causes for them.
However, further research is needed, first of all to comprehensively identify
possible other effects. Another important question regarding these effects is what
influence they have on the idea generation process and its results. This could be
accomplished by designing new experiments to isolate each of the effects. In addition
there may be hidden context variables that change the weights depending on the
situation, which makes the investigations even more difficult. Such context variables
for example include time, place, setting, problem statement and/or type, creativity
technique used, etc. For example the weight of "unprofessional" use may increase if
the team members know each other well or if the computer support system is buggy.
It is also important to consider that experimental settings have restrictions,
especially regarding relationships between participants, motivation and other
important aspects that can influence the results of experiments significantly.
Therefore we want to emphasize the need to conduct field studies as well with whose
help new effects and aspects of intra-team communication may be discovered. On the
other hand the complexity of field studies is far higher than of experiments. Trying to
isolate effects (as described above) may be very hard or sometimes even impossible in
some field study so that we believe that additional lab experiments still have to be
used (as far as possible) in a complementary way.
<references_biblio/>
References
1. Osborn, A. F., “Applied imagination. Principles and Procedures of Creative Thinking”,
Creative Education Foundation, 3rd edition, 1993
2. Fernald, L. W. and P. Nickolenko, “The creative process: Its use and extent of
formalization by corporations”, Journal of Creative Behaviour, vol. 27, no. 3, 1993, pp.
214–220
3. Isaksen, S. G., “A Review of Brainstorming Research: Six Critical Issues for Inquiry”,
Technical report, Creative Problem Solving Group, Buffalo, New York, 1998
4. Taylor, D., P. Berry, and C. Block, “Does group participation when using brainstorming
facilitate or inhibit creative thinking?”, Administrative Science Quarterly, vol. 6, 1958, pp.
22–47
173
5. Diehl, M. and W. Stroebe, “Productivity loss in idea-generating groups: Tracking down
the blocking effect”, Journal of Personality and Social Psychology, 1991, vol. 61, pp.
392–403.
6. Carte, T. A., L. Chidambaram, M. J. Garfield, L. Hicks, and C. Cole, “Group Creativity
and Collaborative Technologies: Understanding the Role of Visual Anonymity.”, 12th
International Workshop on Groupware CRIWG, 2006, pp. 12–21
7. Shepherd, M. M., R. O. Briggs, B. A. Reinig, J. Yen, and J. F. Nunamaker, Jr., “Invoking
Social Comparison to Improve Electronic Brainstorming: Beyond Anonymity.”, Journal of
Management Information Systems, vol. 12, no. 3, 1995, pp. 155 – 170.
8. Bostrom, R. P. and M. Nagasundaram, “Research in Creativity and GSS”, Proceedings of
the 31. Hawaii International Conference on System Sciences, Washington, DC, USA,
1998, pp. 391–405, IEEE Computer Society
9. Sternberg, R. J. and T. I. Lubart, “The Concept of Creativity: Prospects and Paradigms”,
Cambridge University Press, Chapt. 1, 1999, pp. 3–14
10. Applegate, L. M., B. R. Konsynski, and J. F. Nunamaker, “A group decision support
system for idea generation and issue analysis in organization planning”, CSCW ’86:
Proceedings of the 1986 ACM conference on Computer supported cooperative work,
ACM Press, New York, NY, USA, 1986, pp. 16–34
11. Hilliges, O., L. Terrenghi, S. Boring, D. Kim, H. Richter, and A. Butz, “Designing for
collaborative creative problem solving”, Proceedings of the 6th ACM SIGCHI conference
on Creativity and Cognition, ACM Press, 2007, pp. 137–146
12. Powell, A., G. Piccoli, and B. Ives: “Virtual teams: a review of current literature and
directions for future research”, SIGMIS Database, vol. 35, no. 1, 2004a , pp. 6–36
13. Nunamaker, J., R. O. Briggs, D. D. Mittleman, D. R. Vogel, and P. A. Balthazard,
“Lessons from a Dozen Years of Group Support Systems Research: A Discussion of Lab
and Field Findings”, Journal of Management Information Systems, vol. 13, no. 3, 1997,
pp. 163–207
14. Forster, F., “Distributed Creative Problem Solving over the Web”, ICIW ’08: Proceedings
of the 2008 Third International Conference on Internet and Web Applications and
Services, IEEE Computer Society, Washington, DC, USA, 2008, pp. 283–288
15. Vangundy, A. B., “Techniques of Structured Problem-Solving”, Springer Netherland, 1st
edition, 1988
16. Reinig, B., R. Briggs, and J. Nunamaker, “On the measurement of Ideation Quality”,
Journal of Management Information Systems, vol. 23, no. 4, 2007, pp. 143–161
17. Dennis, A. and B. Reinicke, “Beta vs. VHS and the Acceptance of Electronic
Brainstorming Technology”, MIS Quarterly, vol. 28, 2004, pp. 1–20.
18. Helquist, J., Santanen, E., Kruse, J., “Participant-driven GSS: Quality of Brainstorming
and Allocation of Participant Resources”, Proceedings of the CSCW 2007, ACM Press,
pp. 1530-1605
19. Prante, T., Magerkurth, C., Streitz, N., “Developing CSCW Tools for Idea Finding –
Empirical Results and Implications for Design”, Proceedings of the CSCW 2002,
Luisiana, USA, ACM Press, pp. 106-115
174