Available online at www.sciencedirect.

com
Available online at www.sciencedirect.com

ScienceDirect
ScienceDirect
Available online atonline
Available www.sciencedirect.com
at www.sciencedirect.com
Procedia CIRP 00 (2019) 000–000
ScienceDirect
Procedia CIRP 00 (2019) 000–000
ScienceDirect
www.elsevier.com/locate/procedia
www.elsevier.com/locate/procedia

Procedia CIRP 00 (2017)

Procedia 000–000
CIRP 84 (2019) 566–571
www.elsevier.com/locate/procedia

29th CIRP Design 2019 (CIRP Design 2019)

29th CIRP Design 2019 (CIRP Design 2019)
Efforts
Efforts on
on Capturing
Capturing Prototyping
28th CIRPPrototyping
Design Conference,
and
andMay Design
Design Activity
Activity
2018, Nantes, France
in
in Engineering
Engineering
Design
Design ResearchResearch
A new methodology to analyze the functional and physical architecture of
Jorgen Falck Erichsena* , Andreas Wulvik a
, Martin Steinert a
, Torgeir Welo a
existingJorgen
products
Falck for an
Erichsen assembly
a*
, Andreas Wulvik oriented a
product
, Martin Steinert family
a
, Torgeir identification
Weloa
Norwegian University of Science and Technology, Richard Birkelands Veg 2B, 7491 Trondheim, Norway
a
a
Norwegian University of Science and Technology, Richard Birkelands Veg 2B, 7491 Trondheim, Norway
Paul Stief *, Jean-Yves Dantan, Alain Etienne, Ali Siadat
* Corresponding author. Tel.: +47-73595000; E-mail address: [email protected]
* Corresponding author. Tel.: +47-73595000; E-mail address: [email protected]
École Nationale Supérieure d’Arts et Métiers, Arts et Métiers ParisTech, LCFC EA 4495, 4 Rue Augustin Fresnel, Metz 57078, France

Abstract
*Abstract
Corresponding author. Tel.: +33 3 87 37 54 30; E-mail address: [email protected]
Prototyping is one of the core activities of product development, and understanding prototyping should therefore be of great interest to both
Prototyping
researchers andis one of the coreYet,
professionals. activities
when of product development,
considering and understanding
the many definitions of prototypeprototyping should
in engineering therefore
design be ofprototyping
literature, great interest to fully
is not both
researchers and professionals. Yet, when considering the many definitions of prototype in engineering design
understood. Aimed at engineering design researchers, this article compares various efforts that attempt to understand prototyping by capturing literature, prototyping is not fully
Abstract
understood. Aimed
design activity. Thisat comparison
engineering isdesign used researchers,
as a basis for thisdiscussing
article compares
various various
methods, efforts
toolsthat
andattempt to understand
resources available toprototyping by capturing
the engineering design
design activity.
researcher, This
as well comparison
as the contexts of is used as a basis
the studies for discussing
(i.e. laboratory, various methods,
intermediate and in-situtools and resources available to the engineering design
studies).
Inresearcher,
today’s business environment,
as well as the the
of trend towards(i.e.more productintermediate
variety and and
customization is unbroken. Due to this development, the need of
From this comparison of contexts
studies on the studies
capturing laboratory,
prototyping in engineering design in-situ studies).
research, the authors identify that many of the studies have
agile
From and reconfigurable
this comparison production
of studies on systems
capturing emerged to cope
prototyping in with various design
engineering products and product
research, the families.
authors To design
identify and optimize
that The
many of theargueproduction
studies
relatively low robustness—i.e. the ability to generalize and apply the findings to a wider engineering design context. authors thathave
the
systems as
relatively well
low as to choose
robustness—i.e. the
the optimal
ability product
to matches,
generalize and product
apply theanalysis
findings methods
to a are
wider needed.
engineeringIndeed,
design most of
context. the
Theknown
authorsmethods
argue aimthe
that to
factors that contribute to the relatively low robustness of these studies are a combination of the methods, tools and resources (including
analyze a product or
factors that contribute one product family on the physical level. Different product families, however, may differ largely in terms of the number and
participants) available totothe theresearchers
relativelyfor low robustness
both capturingofand these studiestheare
analyzing a combination
data. Therefore, theofauthors
the methods,
concludetoolsthat and resources
to increase (including
the robustness
nature of components. to
participants) This
theinfact impedes anboth efficient comparison and choice of Therefore,
appropriatetheproduct family combinations for the
theproduction
of research onavailable
prototyping researchers
engineeringfor capturing
design—i.e. ensureandthat
analyzing therealistic
relevant, data. authors
and representative conclude
data that to increase
is captured—more suitable robustness
tools and
system. A new
of research methodology
prototyping inisengineering
proposed todesign—i.e.
analyze existing
ensureproducts in view of theirand
functional and physical architecture. Thesuitable
aim is to cluster
methods areonneeded. that relevant, realistic representative data is captured—more tools and
these products
methods in new assembly oriented product families for the optimization of existing assembly lines and the creation of future reconfigurable
are needed.
assembly systems. Based on Datum Flow Chain, the physical structure of the products is analyzed. Functional subassemblies are identified, and
© 2019 The Authors. Published by Elsevier B.V.
a©©functional
2019 The
2019 Theanalysis
Authors.
Authors. Published
isPublished
performed. by Moreover,
by Elsevier B.V.
Elsevier B.V.
a hybrid functional and physical architecture graph (HyFPAG) is the output which depicts the
Peer-review
Peer-review under responsibility
under product
responsibility of
of the
the scientific
scientific committee
committee of
of the CIRP
the to
CIRP Design Conference
Design Conference 2019.
2019
similarity
Peer-reviewbetween
under families
responsibility of by scientific
the providingcommittee
design supportof the both,
CIRP production
Design system
Conference planners and product designers. An illustrative
2019
example of a nail-clipper is used to explain the proposed methodology. An industrial case study on two product families of steering columns of
Keywords: capturing; prototyping; design activity; engineering design; robustness
thyssenkrupp Presta France
Keywords: capturing; is then
prototyping; carried
design out to
activity; give a first
engineering industrial
design; evaluation of the proposed approach.
robustness
© 2017 The Authors. Published by Elsevier B.V.
Peer-review under responsibility of the scientific committee of the 28th CIRP Design Conference 2018.
1. Introduction and Background for further investigating the use of prototypes and prototyping
1. Introduction
Keywords: Assembly;and Background
Design method; Family identification for PD.
in further
Thereinvestigating
are manythe use ofonprototypes
efforts capturingand prototyping
prototyping in
Prototyping is one of the core activities of Product in PD. There are many efforts on capturing
engineering design research, with the underlying assumption prototyping in
Prototyping is one of the core activities
Development (PD) [1], and has been a relevant topic in industry of Product engineering design research, with the underlying
that there are insights to be gained from observing and assumption
Development
and academia (PD)
for [1], and has[2].
decades beenWalla relevant
et topic
al. [3] instate
industry
that that there are insights
(retrospectively) analyzing to the
be activity.
gained from observing
This article aims and
to
1. Introduction of the product range and characteristics manufactured and/or
and academia for decades [2]. Wall et al.
“prototyping is one of the most critical activities of new product [3] state that (retrospectively)
compare various analyzing
efforts on the activity.
capturing This article
prototyping andaims
designto
assembled in this system. In this context, the main challenge in
“prototyping
development”. is one of the most critical activities of new product compare various efforts on capturing prototyping and design
Due to theConsequently,
fast development understanding in the prototyping
domain is of activity in and
modelling engineering
analysis design
is now research,
not only and to discuss
to cope what
with single
development”.
key interest Consequently,
to the understanding
engineering design prototyping
researcher—yet is of activity
steps can inbeengineering
taken in design
order to research,
increase the and to discuss
robustness of what
studies
communication and an ongoing trend of digitization and products, a limited product range or existing product families,
key interest to the engineering
al. [4] stateenterprises
Camburn et manufacturing design
that “prototyping researcher—yet
may be steps can
capturing be taken in order to increase the robustness of studies
digitalization, are facing important but also to prototyping.
be able to analyze and to compare products to define
Camburn
simultaneouslyet al.one [4]
of the state
most that “prototyping
important and least may
formallybe capturing prototyping.
challenges in today’s market environments: a continuing new product families. It can be observed that classical existing
simultaneously
areas ofone
explored towards of the most important and least formally
design”. 1.2. Defining Prototypes and Prototyping
tendency reduction of product development times and product families are regrouped in function of clients or features.
explored areas of design”. 1.2. Defining Prototypes and Prototyping
shortened product lifecycles. In addition, there is an increasing However, assembly oriented product families are hardly to find.
1.1. Motivation
demand and Aim being at the same time in a global
of customization, Underlining
On the product thefamily
statement from
level, Camburn
products et al.
differ [4], Wall
mainly et
in two
1.1. Motivation and Aim al. Underlining
[3] highlight the
the statement
importance from
of Camburn etwithout
prototyping al. [4],actually
Wall et
competition with competitors all over the world. This trend, main characteristics: (i) the number of components and (ii) the
Though prototyping al. [3] highlight the importanceratherofby prototyping withoutdefines
actually
which is inducing the isdevelopment
a core activity fromin PD,
macro it istonotmicro
fully defining
type the activity,
of components butmechanical,
(e.g. describing
electrical, what
electronical). a
Though
understood prototyping is
by theinengineering a core activity in PD, it is not fully defining the activity, but rather by describing what defines
definea
markets, results diminisheddesign research
lot sizes due community—as
to augmenting prototype. Similarly, Eppinger
Classical methodologies consideringandmainly
Ulrich [5]products
single
understood
shown by by the engineering
Jensen(high-volume
et al. [1]. Hence, design research community—as
there is motivation and need prototype.
prototyping Similarly, Eppinger and Ulrich [5] define
product varieties to low-volume production) [1]. or solitary, simply
alreadyasexisting
the activity of producing
product familiesprototypes.
analyze the
shown by Jensen et al. [1]. Hence, there is motivation and need prototyping simply as the activity of producing prototypes.
To cope with this augmenting variety as well as to be able to product structure on a physical level (components level) which
identify
2212-8271 possible
© 2019 The optimization
Authors. Publishedpotentials in the existing
by Elsevier B.V. causes difficulties regarding an efficient definition and
2212-8271
Peer-review©under
production 2019responsibility
system,The it
Authors. Published
is important
of tobyhave
Elsevier
the scientific B.V.of the
a precise
committee knowledge comparison
CIRP Design Conference 2019 of different product families. Addressing this
Peer-review under responsibility of the scientific committee of the CIRP Design Conference 2019

2212-8271©©2017
2212-8271 2019The
The Authors.
Authors. Published
Published by Elsevier
by Elsevier B.V. B.V.
Peer-review
Peer-review under
under responsibility
responsibility of scientific
of the the scientific committee
committee of the of theCIRP
28th CIRP Design
Design Conference
Conference 2019.
2018.
10.1016/j.procir.2019.04.303
 Jorgen Falck Erichsen et al. / Procedia CIRP 84 (2019) 566–571 567
2 Author name / Procedia CIRP 00 (2019) 000–000

Fig. 1. Number of participants used in literature studying design activity in a professional (left, shown in red) and educational setting (right, shown in blue).

However, the authors argue that prototyping is more than including what the designer is thinking and conceptualizing, as
the activity of producing prototypes—it is a learning activity well as the artefacts that are created during the activity. There
that contributes in generating information, skills and are many contributions in engineering design literature that
knowledge for the designers involved [6]. Therefore, in this reference ‘design activity’ without explicitly using the word
article, the term prototyping is used to describe the activity of prototyping—yet, the authors still consider some of these
exploring various concepts and ideas during the PD process. activities prototyping.
This includes designing, building and testing various aspects of This article presents a brief overview of contexts for
concepts and ideas, which often creates output in the form of capturing prototyping, before discussing the types and number
prototypes. While there are many definitions of prototypes in of participants, as well as the methods, tools and resources
engineering design literature—e.g. the 19 definitions listed by available for capture and analysis. This article identifies that
Jensen et al. [1]—this article uses the term prototype as tangible robustness—the ability to generalize and apply the findings to
output from the activity of prototyping. Following this a wider engineering design context—is relatively low for some
definition, prototypes can be physical artefacts, but can also be of the studies, and argues that this a result of the methods, tools
virtual—e.g. Computer Aided Design (CAD) models or and resources available to the engineering design researchers.
drawings. Based on these findings, the article presents a discussion on
possible steps and approaches for increasing the robustness of
1.3. Scope and Structure future studies.

Ideally, to understand all aspects of prototyping, it would be

very helpful to the engineering design researcher to be able to
fully capture the prototyping activity in all possible formats,
568 Jorgen Falck Erichsen et al. / Procedia CIRP 84 (2019) 566–571
Author name / Procedia CIRP 00 (2019) 000–000 3

2. Contexts for Capturing Prototyping 3. On Robustness of Studies Capturing Prototyping

Cash et al. [7] identify different contexts of empirical There are two trends that are apparent in Fig. 1; many of the
engineering design research, ranging from studying activity in studies have low sample sizes—e.g. when using practitioners
design practice to studying activity in laboratories, with in their ‘natural’ context—and the many of the studies are using
intermediary studies as somewhat of a middle ground between student participants. The use of low sample sizes makes it
the two former—e.g. “Experimental studies using practitioners, difficult to generalize findings because of low statistical power
varying little from normal practice” [7]. These three contexts and potential inflated effect size. While the observations found
vary in realism and controllability. Experiments in the in the studies may be valid for the context they were observed
laboratory are controllable (and constrainable), allowing for in; the use of low samples sizes implies that the observations
detailed examination of a single, less complex phenomenon, may not be reproducible or generalizable to a wider PD context.
while observing practitioners in-situ allows for higher degrees Many of the studies in Fig. 1 arguably capture highly
of realism. Intermediate experiments allow for a compromise relevant data for engineering design research—yet assessing
between controllability and realism, as these experiments often the applicability of the studies is difficult due to the use of small
use practitioners as participants. Cash and Culley [8] emphasize sample sizes and few investigated prototypes. Moreover, it is
the importance of conducting both practice and laboratory also difficult to assess the degree of realism of the studies
studies, aiming to draw from strengths of both the detailed extensively using student participants. The authors have
examinations in a laboratory and the realism of studying identified this difficulty in assessing applicability and realism
practice. They state that “The role of experimentation serves to of studies capturing prototyping as a shortcoming of current PD
support both theory building and theory testing – both of which research. To understand how to remedy this shortcoming, and
must be considered in order to develop meaningful to increase the robustness of research on prototyping in early-
understanding.” stage PD, this article considers the following RQ: “What
While in-situ observations of design activity offer greater factors are causing the relatively low level of robustness of
realism regarding both participants and nature of the task, these research on prototyping in early-stage PD?”
studies often have few—less than 20, sometimes even less than
10—participants [7,9–15]. The number of participants in 4. Investigating the Methods, Tools and Resources
laboratory studies also vary from larger—i.e. more than 20 Required for Capturing Prototyping
participants—controlled and semi-controlled experiments [16–
21] to smaller design sessions considering a handful of students To attempt to answer the RQ, the task and duration of
[22–28]. current studies must be considered—as must the methods, tools
In the laboratory, the availability of and proximity to and resources required for capturing and analyzing the activity.
students make it possible for researchers to capture larger data
sets. The use of students as substitutes for professional 4.1. Capturing Methods of In-Situ and Laboratory
participants leads to questioning if the studies capture realistic Experiments
data. Findings from Salman et al. [29] include that there is no
significant difference in code quality when using software The method chosen in many of the in-situ studies is protocol
engineering students as substitutes for software engineering studies, a method proving high fidelity and detailed transcripts
professionals when doing relatively small programming tasks, of what the participants (often in teams) say and do [9–
and correspond with findings from Höst et al. [30]. However, 12,14,15]. Protocol studies are exhaustive in both data
Smith and Leong [31] capture significant differences between gathering and analysis, and the protocols are often recorded
students and professionals doing simulated design tasks in from short meetings or sessions. There are efforts where the
engineering design, stating that “real differences exist between listed durations are longer, e.g. efforts by Ball and Christensen
the processes used by the student groups and the processes used [11] and Christensen and Schunn [12], where protocols from
by the professional groups”. Consequently, there is not enough nine hours of design meetings are presented. In a more extreme
evidence to state that students are a fully realistic substitute for example of high fidelity capture, Cash et al. [7] present 12
practitioners—especially in the context of PD. weeks of design activity captured on video (using multiple
Fig. 1 is included to show the number of participants used cameras for redundancy) of 7 practitioners doing regular design
in the studies considered in this section, and differentiates the activity at their desks in a company.
studies using professional participants (shown in red) from the In the laboratory experiments, elaborate infrastructure is
studies using student participants (shown in blue). The grey often in place, allowing for systematic capture of video and
columns represent where the studies report ambiguous or audio [11,16,19,20,23,26,27,32]. For instance, to aid
indefinite numbers, e.g. “3 groups of 4-6 students”, which researchers in capturing design activities, the Design
implies that there were minimum 12 and maximum 18 student Observatory was built at Stanford University [32], based on the
participants [28]. work from Tang and Leifer [33,34]. Tang and Leifer [33,34]
focused on fast iterations of “observe—analyze—intervene”,
with the underlying assumption that design activity could be
observed and then forcefully changed (by facilitators) to
 Jorgen Falck Erichsen et al. / Procedia CIRP 84 (2019) 566–571 569
4 Author name / Procedia CIRP 00 (2019) 000–000

improve performance. The Design Observatory was developed DTRS11 dataset [11] for pre-screening larger video recordings
to provide researchers with various tools and technologies for in order to find interesting events. Moreover, Wulvik et al. [41]
conducting design observations, and the observatory addressed have published an article on various tools and technologies for
two fundamental questions; “what are designers doing, capturing body language in engineering design, aiming to
thinking, and experiencing when they do design and how can exemplify other technologies that can be used in addition to
we [Red. the design community] improve their performance?” manual video coding.
[32]. Though built around the idea of “observe—analyze—
intervene”, the facility focused more on observation than 5. Discussion
intervention and although it was built without choosing a
specific capturing technology, video was eventually the From comparing the various studies on capturing
preferred format for capturing the activity [35]. prototyping in engineering design research, the authors argue
that the factors that contribute to the relatively low robustness
4.2. Tools for Capturing Activity of these studies are a combination of the methods, tools and
resources (including participants) available to the researchers
Notably, there are various technologies being explored to for both capturing and analyzing the data. However, it is
aid in capturing design activities. [36] suggest various apparent that this relatively low robustness does not come from
alternatives for capturing activity using other technologies than a lack of effort from the engineering design researchers, as
cameras, e.g. using GPS trackers or wireless signals of many of the methods and tools used in the considered literature
connected devices. Similarly, Sjöman and Steinert [37] present are labor-, cost- and resource-intensive, e.g. Cash et al. [7].
a Radio Frequency Identification (RFID) based tool for sensing The comparatively low robustness is further underlined by
proximity in the design workspace, attempting to capture Lloyd et al. [44], who state that “A major problem with a [sic.]
interactions through other means than cameras. much of what goes under the general rubric of ‘Design
Through advances in both video recording and (digital) Research’ is a poorly defined relationship to empirical
storage technology over the last decade, video capture has evidence”.
become a benchmark for capturing design activities in design However, there are various efforts that attempt to increase
observation [35,38]. In such sessions, multiple cameras and the robustness of engineering design research. One such
microphones record high fidelity images and audio, and this is initiative is the datasets created for DTRS, a biennial effort
often in stored large local storage systems. The sessions are where design researchers can share the same dataset for
often tuned towards particular activities in order to explore comparing and improving their methods [44]. One of these
topics such as the prototyping media used by the design team datasets is presented by Ball and Christensen [11] for the 11th
[16,21,39] or to capture team dynamics and emotion [20]. Design Thinking Research Symposium (often referred to as the
Törlind et al. [35] stress that video and audio quality are ‘DTRS11 dataset’). In this dataset, they “[…] recorded 150+
important factors to consider, yet emphasize that the main hours of video footage of the activities of a professional design
limitation of design observation through video recordings is team (with 7 team members) from a Scandinavian User
resources required to analyze the captured data. Involvement Department”.
Törlind et al. [35] state that a substantial hindrance for
4.3. Tools for Analyzing Captured Activity observation-based design research is the effort required to do
thorough analysis of the data. One solution for overcoming this
While doing video recordings require relatively low effort hindrance is to use computational analysis methods for
from researchers, the material is often manually coded by (automated) audio and visual classification, e.g. TSV as shown
multiple coders that go through and interpret the data by Wulvik et al. [43], to identify points-of-interest in larger
[11,16,19,20,23,26,27,32]. Manual video coding is a laborious datasets, and thus reducing the effort required for analysis.
task [35,40,41], and these sessions are therefore relatively Such analysis tools should be further researched. Beyond
short—often less than 60 minutes per team. However, there are purely focusing on improving the analysis methods, there is
exceptions where the studies are more longitudinal, e.g. studies also the possibility to explore other inputs as supplementary
by Cash et al. [7] and Ball and Christensen [11]—both these data for analysis, e.g. body language [41].
studies include professionals doing design activity captured on Beyond the studies that attempt to capture design activity
video for many hours, which would have required a itself, there are various studies that specifically focus on the
monumental effort in (manual) analysis. These studies are output of the activities—e.g. designers’ logbooks [45] or
notably high in both realism and relevance. sketches [18,25,46-50]. Many of the empirical studies
There are indeed efforts that try to tackle the resource specifically targeting prototypes use them as deliverables,
problem of analysis in design observations and protocol either in university courses or in experiments [51-54]. Here,
studies. Dong [24] and Dong et al. [42] present Latent Semantic prototypes are either photographed or physically collected
Analysis (LSA) as a way of analyzing protocols, Wulvik et al. through the experiments for later analysis—e.g. “[…] pictures
[40,43] present a method for preliminary analysis of longer were taken again to capture the designs during these
video recordings captured from observational studies called demonstrations. These pictures were the ‘after testing’ data.
Temporal Static Visualizations (TSV). This method uses the The pictures were captured from many different angles to
570 Jorgen Falck Erichsen et al. / Procedia CIRP 84 (2019) 566–571
Author name / Procedia CIRP 00 (2019) 000–000 5

obtain sufficient details of the cars, so that if necessary, the cars and Research Methodology, Lyngby/Copenhagen, Denmark, 15.-19.08.
could be reconstructed.” [55]. Notably, while many of these 2011, 2011, pp. 151–162.
[8] P. Cash and S. Culley, “The Role of Experimental Studies in Design
studies have more than 20 participants—e.g. Youmans [53] Research,” in The Routledge Companion to Design Research,
with 120 participants—they are all using student participants, RoutledgeFalmer, 2015.
and not practitioners. [9] S. Ahmed and B. T. Christensen, “An In Situ Study of Analogical
To supplement such efforts, the authors suggest that Reasoning in Novice and Experienced Design Engineers,” J. Mech. Des.,
researchers should also investigate physical prototypes, as vol. 131, no. 11, p. 111004, 2009.
[10] L. J. Ball and B. T. Christensen, “Analogical reasoning and mental
these artefacts provide a tangible and available starting point simulation in design: two strategies linked to uncertainty resolution,”
for further investigation into prototyping, and capturing Des. Stud., vol. 30, no. 2, pp. 169–186, Mar. 2009.
physical artefacts is more available (and is potentially less [11] L. J. Ball and B. T. Christensen, “Designing in the wild,” Des. Stud., vol.
labor-intensive) than capturing the prototyping activity itself. 57, pp. 1–8, Jul. 2018.
[12] B. T. Christensen and C. D. Schunn, “The relationship of analogical
distance to analogical function and preinventive structure: The case of
6. Conclusion engineering design,” Mem. Cognit., vol. 35, no. 1, pp. 29–38, 2007.
[13] C. L. Cramer-Petersen, B. T. Christensen, and S. Ahmed-Kristensen,
This paper has investigated several studies that capture “Empirically analysing design reasoning patterns: Abductive-deductive
prototyping in an engineering design context, and has identified reasoning patterns dominate design idea generation,” Des. Stud., Oct.
that the robustness of many of these studies is relatively low— 2018.
[14] K. Dorst and N. Cross, “Creativity in the design process: co-evolution of
mainly due to the extensive use of small sample sizes and use problem–solution,” Des. Stud., vol. 22, no. 5, pp. 425–437, Sep. 2001.
of student participants. This paper argues that the root cause of [15] J. W. Kan and J. S. Gero, “Acquiring information from linkography in
the comparatively low robustness can be traced back to the protocol studies of designing,” Des. Stud., vol. 29, no. 4, pp. 315–337,
limitations of the tools, methods and resources available to the 2008.
PD researchers. Therefore, the authors conclude that to [16] J. A. Edelman, “Understanding radical breaks: media and behavior in
small teams engaged in redesign scenarios,” Doctoral Thesis, Stanford
increase the robustness of research on prototyping in University, 2011.
engineering design—i.e. ensure that relevant, realistic and [17] G. Goldschmidt and P. A. Rodgers, “The design thinking approaches of
representative data is captured—more suitable tools and three different groups of designers based on self-reports,” Des. Stud.,
methods are needed. This is further emphasized by Cash [56], vol. 34, no. 4, pp. 454–471, Jul. 2013.
who states that “Lack of ability to use these research methods [18] M. Gonçalves, C. Cardoso, and P. Badke-Schaub, “How far is too far?
Using different abstraction levels in textual and visual stimuli,” in DS
effectively prevents researchers from addressing important 70: Proceedings of DESIGN 2012, the 12th International Design
research questions and developing subsequent meaningful Conference, Dubrovnik, Croatia, 2012.
theory or robust scientific knowledge”. This is a bold [19] M. F. Jung, N. Martelaro, and P. J. Hinds, “Using robots to moderate
statement, and one that must be addressed in order to further team conflict: the case of repairing violations,” in Proceedings of the
strengthen and advance engineering design research. Tenth Annual ACM/IEEE International Conference on Human-Robot
Interaction, 2015, pp. 229–236.
[20] M. F. Jung, “Engineering team performance and emotion: Affective
Acknowledgements interaction dynamics as indicators of design team performance,”
Doctoral Thesis, Stanford University, 2011.
This research is supported by the Research Council of [21] N. Sonalkar, K. Jablokow, J. Edelman, A. Mabogunje, and L. Leifer,
Norway through its user-driven research (BIA) funding “Design whodunit: The relationship between individual characteristics
and interaction behaviors in design concept generation,” in ASME 2017
scheme, project number 236739/O30. International Design Engineering Technical Conferences and
Computers and Information in Engineering Conference, 2017, p.
References V007T06A009–V007T06A009.
[22] N. N. A. Ariff, P. Badke-Schaub, Ö. Eris, and S. S. S. Suib, “A
[1] L. S. Jensen, A. G. Özkil, and N. H. Mortensen, “Prototypes in framework for reaching common understanding during sketching in
engineering design: Definitions and strategies,” in 14th International design teams,” 2012.
Design ConferenceInternational Design Conference, 2016, pp. 821–830. [23] P. Cash and A. Maier, “Prototyping with your hands: the many roles of
[2] J. S. Gero and U. Lindemann, Human Behaviour in Design 05. Key gesture in the communication of design concepts,” J. Eng. Des., vol. 27,
Centre of Design Comp & Cogntn, 2005. no. 1–3, pp. 118–145, 2016.
[3] M. B. Wall, K. T. Ulrich, and W. C. Flowers, “Evaluating prototyping [24] A. Dong, “The latent semantic approach to studying design team
technologies for product design,” Res. Eng. Des., vol. 3, no. 3, pp. 163– communication,” Des. Stud., vol. 26, no. 5, pp. 445–461, Sep. 2005.
177, Sep. 1992. [25] Ö. Eris, N. Martelaro, and P. Badke-Schaub, “A comparative analysis of
[4] B. A. Camburn et al., “Methods for Prototyping Strategies in Conceptual multimodal communication during design sketching in co-located and
Phases of Design: Framework and Experimental Assessment,” p. distributed environments,” Des. Stud., vol. 35, no. 6, pp. 559–592, Nov.
V005T06A033, Aug. 2013. 2014.
[5] S. D. Eppinger and K. T. Ulrich, “Product design and development,” [26] A. Larsson, P. Törlind, A. Mabogunje, and A. Milne, “Distributed design
1995, 1995. teams : embedded one-on-one conversations in one-to-many,” presented
[6] D. A. Schön, The Reflective Practitioner: How Professionals Think in at the Common Ground International Conference 2002 : 05/09/2002 -
Action. Basic Books, 1983. 07/09/2002, 2002, pp. 604–614.
[7] P. Cash, B. Hicks, S. Culley, and F. Salustri, “Designer behaviour and [27] A. Mabogunje, O. Eris, N. Sonalkar, M. Jung, and L. J. Leifer, “Spider
activity: An industrial observation method,” in DS 68-2: Proceedings of Webbing: A Paradigm for Engineering Design Conversations During
the 18th International Conference on Engineering Design (ICED 11), Concept Generation,” in About Designing: Analysing Design Meetings,
Impacting Society through Engineering Design, Vol. 2: Design Theory J. McDonnell, and P. Llyod, eds., Taylor & Francis, London, UK, 2009,
pp. 49–65.
 Jorgen Falck Erichsen et al. / Procedia CIRP 84 (2019) 566–571 571
6 Author name / Procedia CIRP 00 (2019) 000–000

[28] J. Stempfle and P. Badke-Schaub, “Thinking in design teams - an [43] A. Wulvik, A. Menning, and M. Steinert, “A computational approach to
analysis of team communication,” Des. Stud., vol. 23, no. 5, pp. 473– expose conversation dynamics in engineering design activities,” in DS
496, Sep. 2002. 87-2 Proceedings of the 21st International Conference on Engineering
[29] I. Salman, A. T. Misirli, and N. Juristo, “Are Students Representatives of Design (ICED 17) Vol 2: Design Processes, Design Organisation and
Professionals in Software Engineering Experiments?,” in 2015 Management, Vancouver, Canada, 21-25.08. 2017, 2017, pp. 101–110.
IEEE/ACM 37th IEEE International Conference on Software [44] P. Lloyd, J. McDonnell, and N. Cross, “ANALYSING DESIGN
Engineering, 2015, vol. 1, pp. 666–676. BEHAVIOUR: THE DESIGN THINKING RESEARCH SYMPOSIA
[30] M. Höst, B. Regnell, and C. Wohlin, “Using Students as Subjects—A SERIES,” Proc Int Assoc. Soc. Des. Res. IASDR07, p. 11, 2007.
Comparative Study of Students and Professionals in Lead-Time Impact [45] H. McAlpine, P. Cash, and B. Hicks, “The role of logbooks as mediators
Assessment,” Empir. Softw. Eng., vol. 5, no. 3, pp. 201–214, Nov. 2000. of engineering design work,” Des. Stud., vol. 48, pp. 1–29, Jan. 2017.
[31] R. P. Smith and A. Leong, “An Observational Study of Design Team [46] A. Acuna and R. Sosa, “The Complementary Role of Representations in
Process: A Comparison of Student and Professional Engineers,” J. Mech. Design Creativity: Sketches and Models,” in Design Creativity 2010,
Des., vol. 120, no. 4, pp. 636–642, Dec. 1998. Springer, 2011, pp. 265–270.
[32] K. Carrizosa, Ö. Eris, A. Milne, and A. Mabogunje, “Building the design [47] O. Atilola, M. Tomko, and J. S. Linsey, “The effects of representation on
observatory: a core instrument for design research,” in DS 30: idea generation and design fixation: A study comparing sketches and
Proceedings of DESIGN 2002, the 7th International Design Conference, function trees,” Des. Stud., vol. 42, pp. 110–136, Jan. 2016.
Dubrovnik, 2002, pp. 37–42. [48] J. J. Shah, N. Vargas-Hernandez, J. D. Summers, and S. Kulkarni,
[33] J. C. Tang and L. J. Leifer, “An observational methodology for studying “Collaborative Sketching (C-Sketch) — An Idea Generation Technique
group design activity,” Res. Eng. Des., vol. 2, no. 4, pp. 209–219, 1991. for Engineering Design,” J. Creat. Behav., vol. 35, no. 3, pp. 168–198,
[34] J. Tang and L. Leifer, “Observations from an Empirical Study of the Sep. 2001.
Workspace Activity of Design Teams,” in Proceedings of the First [49] M. C. Yang, “Observations on concept generation and sketching in
International ASME Conference on Design Theory and Methodology, engineering design,” Res. Eng. Des., vol. 20, no. 1, pp. 1–11, Mar. 2009.
1989. [50] V. Viswanathan and J. Linsey, “A study on the role of expertise in
[35] P. Törlind, N. Sonalkar, M. Bergström, E. Blanco, B. Hicks, and H. design fixation and its mitigation,” in ASME 2012 international design
McAlpine, “Lessons learned and future challenges for design engineering technical conferences and computers and information in
observatory research,” in DS 58-2: Proceedings of ICED 09, the 17th engineering conference, 2012, pp. 901–911.
International Conference on Engineering Design, Vol. 2, Design Theory [51] S. P. Dow, K. Heddleston, and S. R. Klemmer, “The efficacy of
and Research Methodology, Palo Alto, CA, USA, 24.-27.08. 2009, 2009. prototyping under time constraints,” in Proceedings of the seventh ACM
[36] K. Thoring, R. M. Mueller, and P. Badke-Schaub, “Technology- conference on Creativity and cognition, 2009, pp. 165–174.
supported design research,” 80, vol. 11, 2015. [52] C. Kriesi et al., “Distributed Experiments in Design Sciences, a Next
[37] H. Sjöman and M. Steinert, “Applying Sequential Pattern Mining to Step in Design Observation Studies?,” in DS 80-2 Proceedings of the
Portable RFID System Data,” 2016. 20th International Conference on Engineering Design (ICED 15) Vol 2:
[38] P. Törlind, “A Framework for Data Collection of Collaborative Design Design Theory and Research Methodology Design Processes, Milan,
Research,” in Guidelines for a Decision Support Method Adapted to Italy, 27-30.07. 15, 2015.
NPD Processes, Paris, France, 2007, pp. 453–454. [53] R. J. Youmans, “The effects of physical prototyping and group work on
[39] J. Edelman, A. Agarwal, C. Paterson, S. Mark, and L. Leifer, the reduction of design fixation,” Des. Stud., vol. 32, no. 2, pp. 115–138,
“Understanding radical breaks,” in Design Thinking Research, Springer, Mar. 2011.
2012, pp. 31–51. [54] D. Mathias, D. Boa, B. Hicks, C. Snider, P. Bennett, and C. Taylor,
[40] A. Wulvik, M. B. Jensen, and M. Steinert, “Temporal Static “Design variation through richness of rules embedded in LEGO bricks,”
Visualisation of Transcripts for Pre-Analysis of Video Material: in DS 87-8 Proceedings of the 21st International Conference on
Identifying Modes of Information Sharing,” Anal. Des. Think. Stud. Engineering Design (ICED 17) Vol 8: Human Behaviour in Design,
Cross-Cult. Co-Creat. Leiden CRC Press. Francis, 2017. Vancouver, Canada, 21-25.08. 2017, 2017, pp. 099–108.
[41] A. Wulvik, J. Erichsen, and M. Steinert, “Capturing Body Language in [55] V. Viswanathan, O. Atilola, N. Esposito, and J. Linsey, “A study on the
Engineering Design–Tools and Technologies,” in DS 85-1: Proceedings role of physical models in the mitigation of design fixation,” J. Eng.
of NordDesign 2016, Volume 1, Trondheim, Norway, 10th-12th August Des., vol. 25, no. 1–3, pp. 25–43, Mar. 2014.
2016, 2016. [56] P. Cash, “Developing theory-driven design research,” Des. Stud., vol.
[42] A. Dong, A. W. Hill, and A. M. Agogino, “A Document Analysis 56, pp. 84–119, May 2018.
Method for Characterizing Design Team Performance,” J. Mech. Des.,
vol. 126, no. 3, p. 378, 2004.