9789144122304

23 mm

SÄFSTEN GUSTAVSSON | RESEARCH METHODOLOGY

Kristina Säfsten is a professor of production systems at the School of Engineering, Jönköping University, as well as a visiting professor at Mälardalen University.

Maria Gustavsson is a professor of education at the Department of Behavioural Sciences and Learning, Linköping University.

The two authors have many years of experience teaching research methodology.

RESEARCH METHODOLOGY FOR ENGINEERS AND OTHER PROBLEM-SOLVERS This book covers the entire research process: from formulating a research problem to presenting and reviewing the results. It is intended to be used in methodology courses in engineering programs on the basis of the scientific tradition in engineering and how engineers have historically been trained. The book presents a selection of research methods, as well as engineering perspectives on the philosophy of science, research methods and techniques for data collection. In addition to explaining and presenting examples of key concepts in the methodological landscape, it also discusses ethics, quality and communication.

The book is about how to plan, execute and review degree projects and other scientific studies. By presenting several practical examples on how to carry out scientific studies in the field of engineering, the book gives the reader a better understanding of research methods that are particularly relevant to engineers.

RESEARCH METHODOLOGY FOR ENGINEERS AND OTHER PROBLEM-SOLVERS

KRISTINA SÄFSTEN MARIA GUSTAVSSON

Research Methodology – For Engineers and Other Problem-Solvers can also be used as an introductory book on methodology in other disciplines with a focus on problem-solving. The book is suitable for both introductory and postgraduate levels.

Art.nr 39758

studentlitteratur.se

978-91-44-12230-4_cover.indd All Pages

2020-01-21 08:12

Original title: Forskningsmetodik – för ingenjörer och andra problemlösare © The authors and Studentlitteratur, Lund, 2019

Copying prohibited This book is protected by the Swedish Copyright Act. Apart from the restricted rights for teachers and students to copy material for educational purposes, as regulated by the Bonus Copyright Access agreement, any copying is prohibited. For information about this agreement, please contact your course coordinator or Bonus Copyright Access.

Should this book be published as an e-book, the e-book is protected against copying.

Anyone who violates the Copyright Act may be prosecuted by a public prosecutor and sentenced either to a fine or to imprisonment for up to 2 years and may be liable to pay compensation to the author or to the rightsholder.

Studentlitteratur publishes digitally as well as in print formats.

Studentlitteratur’s printed matter is sustainably produced, both as regards paper and the printing process.

Art.No. 39758 ISBN 978-91-44-12230-4 First edition 1:1  The authors and Studentlitteratur 2020 © www.studentlitteratur.se Studentlitteratur AB, Lund Fact-checking: Peter Svensson Book design: Lukas Möllersten/Lyth & Co Cover design: Jens Martin Cover illustration: Shutterstock.com Printed by Dimograf, Poland 2020

CONTENTS

P R E FA C E 1 3

1. Engineers and science The book and its outline An expanded toolbox Contents and outline

15 15 16 17

What is an engineer?

19

Science, research and innovation

24

Needs-based research Utility and innovation Engineering science Technology Technology and science Classifying the field of engineering science If you want to learn more

26 28

29 30 32

33

34

2. The methodological landscape

35

Map over the methodological landscape

36

Quantitative and qualitative research Techniques for collecting and analyzing data, research method and philosophy of science

37 39

Data and information in inquiring systems

40

Science and pseudoscience

42

6 Views of science Ontology Epistemology Scientific ideals Ethical guidelines Philosophy of science approaches Positivism Pragmatism Critical realism Interpretivism If you want to learn more

3. The path from problem to solution The research process Problem-solving Terminology, concepts and definitions How may a scientific study contribute?

Academic contribution Contribution to practical utility and usefulness From problem area to research question The way we ask … … determines the answers we receive Good problem formulations Literature review Structured literature review, step by step The different functions of a literature review Planning the study Study design and method Data collection, analysis and presentation of results Do not get lost along the way

43 44 45 45 52

52 53 53 54 54

56

57 57 59 61

62 62 68

69 71 71 72

74 75 80

80 81 82 83

A logical flow

83

If you want to learn more

84

7 4. Research methods and designs Experiments Independent and dependent variables Experimental setting Good laboratory practices Experimental study designs Factorial designs Systematic parameter variation Modeling and simulation Survey Descriptive survey study Explanatory survey study Execution Case study Case and unit of analysis Design Execution Design research Design research methodology (DRM) DRM in four steps Types of DRM studies Graphical representations as support in DRM Action research Execution The role of the researcher Interactive research Research systems and practitioner systems Division of labor between researchers and practitioners Execution

85 85 87 89 91 91 94

98 99

100 101 101 104

105 107 108 109

112 113 114 115 116

121 122 123

123 124 125 126

Choosing research method

128

If you want to learn more

129

5. Techniques for data collection

131

Choosing technique for data collection

131

Sample

133

8 Measurements To keep in mind during measurements Advantages and drawbacks of measurements Observations To keep in mind during observations Advantages and drawbacks of observations Interviews Interview guide – different types of questions To keep in mind during interviews Advantages and drawbacks of interviews Questionnaires Formulating questions Response alternatives To keep in mind during questionnaires Advantages and drawbacks of using questionnaires Workshops What is a workshop?

Different types of workshops To keep in mind when carrying out workshops How a workshop may serve as a technique for data collection Advantages and drawbacks of carrying out workshops Document studies Different types of documents To keep in mind when engaging in document studies Advantages and drawbacks of engaging in document studies If you want to learn more

6. Analyzing data

137 140 144

144 145 152

152 153 155 158

159 159 163 164 166

167 167

168 169 171 171

172 173 173 175

175

177

From collecting to processing and analyzing data

178

Processing and analyzing quantitative data

178

Type of variable and scale Normal distribution Descriptive statistics Tables and graphical forms of presentation Measure of central tendency Measure of dispersion Measure of correlation

179 180

182 183

188 189 190

9 Statistical inference

192

Hypothesis testing

193

Hypothesis testing – step by step Parametric tests for hypothesis testing Non-parametric tests for hypothesis testing Processing and analyzing qualitative data Thematic analysis Content analysis Qualitative data analysis Presenting the results If you want to learn more

7. Scientific quality criteria Quality in scientific studies Relationship between validity and reliability Systematic and random errors Alternative quality criteria Validity Internal validity Content, construct and criterion validity External validity How to strengthen validity?

Reliability

194 197 203

207 211 213 215 220

222

223 223 224 225 226

227 227 228 229

231

232

Possible actions for ensuring and strengthening validity and reliability

234

If you want to learn more

235

8. Ethical considerations and guidelines

237

Professional ethics

238

Engineering ethics

239

Research ethics Ethics in theory Consequence ethics Duty ethics Engineering ethics in practice

242

243 243 244

246

10 Research ethics in practice The CUDOS norms Scientific misconduct Research involving people Ethical review Protecting personal data Confidentiality or anonymity Informed consent

249 249 250

254 254 255 256 257

Research ethics checklist

259

Engineering ethics vs. research ethics

261

Ethics in degree projects

263

If you want to learn more

265

9. Communicating and reviewing results Scientific communication Historical development The system of scientific communication How is scientific communication carried out?

Purpose and audience Bibliometrics Language and rules for writing academic texts Numbers in the text The importance of a well-written text Writing process

267 268 268 270 270 271 272 274 277 277 278

Structure of an academic text

280

Main text according to IMRaD

284

Introduction Methods Results Discussion Reference management Reference systems Software supporting reference management Illustrating by means of tables and figures Tables Figures

284 286 287 288

289 291 294

295 296 296

11 Reviewing academic texts

298 298

Checklist for reviewers Peer review of scientific articles Verbal presentation and defending academic works

304

306 307

Presentation Public examination and oral defense

308

Which types of texts does a professional engineer write?

309

Advice and strategies in relation to scientific writing

310

Let writing become part of your everyday life Create a structure for the entire text – planning the text Enable a quick start Progression and planning your time Read in order to write

311 312 313 314 315 315

Love your critics If you want to learn more

A P P E N D I X 317 R E FE R E N CE S 321 I N D E X 3 31

316

57

3 The path from problem to solution Just like engineering work, scientific studies concern solving problems.

This chapter describes this process with a focus on activities related to initiating and planning a study. There is a particular emphasis on identifying a problem area and how to formulate a research purpose and its related research questions. The point of departure for a study is existing knowledge, which requires a good overview of the area in question. Since a scientific study is expected to contribute to the development of knowledge, we discuss the meaning of academic and practical contribution. The chapter ends with a description of what the concept of logical flow refers to and why it is important.

This chapter provides answers to the following questions:

→ What does the research process look like, which sub-steps are included and what do we need keep in mind at each step?

→ What does an academic and practical contribution refer to? → What constitutes a good problem formulation? → How is a literature review carried out? → What does a logical flow refer to in a scientific study?

The research process Carrying out a study may be described as a process consisting of a number of activities that combined contribute to achieving the purpose of the study.

The process of getting from start to finish in a scientific study is referred to as a research process. The point of departure for such a process is to be found in practice as well as in theory, see Figure 3.1.

58 Relevant area of knowledge/ theoretical frame of reference Practice Theory

Contributions to academia and practice

Problem area

Purpose and research question

Literature review

Study design

Discussion and conclusion

Analysis and synthesis

Empirical result

Execution of study

Results

Initially, the problem area is identified and an overall purpose and research questions are formulated for the study. A prerequisite for being able to formulate purpose and relevant research questions is that the researcher possesses a reasonable overview of the relevant area of knowledge. A first step is to perform a general literature review in the area. This is followed by an in-depth review of the relevant literature and developing a theoretical framework. Once purpose and research questions have been formulated, the time has come for the researcher to plan how to carry out the study. If the research questions are to be answered on the basis of science, then we need established procedures following certain rules. Collected data are to be processed, analyzed and compiled. The results of the study are to answer the research questions. Through the discussion, the results are placed in a context and we may draw conclusions. Both practical as well as academic contributions from the study should be highlighted.

The research process may appear to be linear in nature. In fact, it is more cyclical as well as, commonly, iterative. The research process is cyclical in the sense that the point of departure is already existing knowledge. A scientific study is expected to add knowledge by either revising or building upon pre-existing knowledge. The research process is iterative in the sense that the activities carried out may need to be repeated on one or several occasions; for example, additional data may need to be collected and an additional litera­ ture review may be necessary. One aspect affecting the need for repetition concerns previous knowledge in the problem area. Is it possible to test or refine already existing knowledge or is entirely new knowledge needed? The more knowledge within an area, the greater the ability to specify research questions that could be answered by means of an approach that may be described as a unidirectional process at a general level, see Figure 3.2a. If, on the other hand, the problem area is largely unresearched, it is frequently

3. T he path from pro b lem to solu tion

© T H E A U T H O R S A N D S T U D E N T L I T T E R A T U R

FIGURE 3.1 The research process is cyclical – beginning and ending in relevant practice and theory.

59 a.

b.

Problem

Problem

Study design

Data collection

Analysis

Study design

Analysis

Data collection

© T H E A U T H O R S A N D S T U D E N T L I T T E R A T U R

FIGURE 3.2 Unidirectional research process (3.2a) and flexible research process (3.2b).

necessary to use a higher degree of flexibility in the research process, see Figure 3.2b. Regardless of whether the overall process is unidirectional or flexible, there are always iterations in relation to specific details.

The research process being described in a linear manner is also apparent in relation to the practical work of writing the report. The written report (product) follows a set structure where the work is presented in chronological order, which gives the impression that both carrying out the study as well as writing (process) is sequential. The contrast between the work performed (carrying out a study and writing a report) and how the results are presented (written report) may be referred to as the process-product paradox (Blomqvist & Hallin, 2015). The activities included in the research process in order to find answers to research questions are not unique in the search for scientific knowledge. Both research and engineering work concern solving problems, and before we go on to a more in-depth description of activities in a research process, we provide a detailed description of what problem-solving is all about.

Problem-solving Regardless of the type of problem we face, we need some form of systematic approach in order to find a solution. A situation where a set objective is to be reached without knowing how to reach it may be referred to as a problem. The approach we choose for reaching the objective is called solution. The thought process preceding the solution is called problem-solving. Simply put, one may describe problem-solving as consisting of three steps: (1) create a picture of the problem (observe), (2) analyze the problem and consider possible solutions (think) and (3) study whether the solution works (test), see Figure 3.3.

Problem-solving exists in everyday life, in the everyday work of engineers

3. T he path from pro b lem to solu tion

60 Think

Test

and in research. What separates different situations from one another are the methods used and the level of thought and accuracy put into the approach.

When solving problems in our everyday life, we might not even reflect upon the fact that we have engaged in problem-solving. It is only when we are faced with more difficult problems that we use a more conscious strategy for solving the problem at hand. An example of structured problem-solving in the work of engineers is product development (Johannesson et al., 2004).

As support in product development, there are various methods and support tools enabling the engineer to systematically and effectively find a technical solution to the problem together with other parties, see Example 3.1. In other words, as support for solving problems, engineers have a toolbox consisting of various methods and support tools as well as knowledge in the fields of, for instance, mathematics, physics and engineering.

In scientific problem-solving, we need to be able to show that the solution addresses the problem in question. Furthermore, it should also be possible for others to arrive at the same solution, given the same conditions. The problem-solving triangle needs to be complemented with clear requirements in relation to thinking, observations and testing, see Figure 3.4. Thinking should be critical, observations reliable and testing rigorous, which requires a toolbox with additional tools. The contents of the scientific toolbox (i.e., research methodology) are what this book is about.

Critical thinking

Reliable observations

Rigorous testing

3. T he path from pro b lem to solu tion

FIGURE 3.4 Scientific problem-solving (Agnew & Pyke, 2007).

© T H E A U T H O R S A N D S T U D E N T L I T T E R A T U R

Observe

FIGURE 3.3 The problem-solving triangle (Agnew & Pyke, 2007).

61

E X A M P L E 3 .1 P R O D U C T D E V E L O P M E N T I S P R O B L E M - S O LV I N G

Many engineers are tasked with designing and developing products, processes and systems. There are a number of theories and methods describing or prescribing how to engage in product development. Here, we may mention integrated product development, axiomatic design and technical systems theory. There are also concrete design support methods, such as FMEA (Failure Mode Effect Analysis), QFD (Quality Function Deployment) and DfA/M (Design for Assembly/Manufacturing).

The work of engineers used to be more free in the past, with an emphasis on the art of engineering. Creative, intuitive

© T H E A U T H O R S A N D S T U D E N T L I T T E R A T U R

solutions were developed by means of a “trial-and-error” methodology. As the market created greater demands with regard to efficiency and as the level of complexity in products increased, there was a need for more systematic approaches in terms of product development. What is now referred to as design research originates from the 1960s when there was more and more interest in the design process, initially in the fields of architecture and industrial design and soon also in construction and product development (engineering design). The initial aim was to create a systematic design process, like the systematic approach used in science, and what is now known as the design methods movement was born. With inspiration from scientific approaches, the process of developing products has become increasingly formalized and the current product development process shares a large number of similarities with the research process.

Sources: Cross, 1993; Hubka & Eder, 1988; Johannesson et al., 2004; Myrup Andreasen & Hein, 2000 and Suh, 2001.

Terminology, concepts and definitions In order to facilitate communication and understanding between participants in an area, a common terminology is frequently used. A word or expression with a well-defined meaning used in a particular field is referred to as a term and a set of terms in a field is called a terminology (Institutet

3. T he path from pro b lem to solu tion

för språk och folkminnen, 2019). Research methodology spans disciplinary and subject-related borders, but a shared terminology is nevertheless used for describing how scientific studies are carried out. Knowledge of the relevant terminology not only facilitates an understanding of scientific texts, it also increases our ability to use a form of language recognized by other readers.

A key factor here are the terms concept and definition. Concepts represent the abstract content of a linguistic term or mental perceptions regarding a phenomenon, such as objects and events in reality (Institutet för språk och folkminnen, 2019). Concepts have to be explained and delimited in relation to other concepts, which is done by means of definitions. A definition (from the Latin word definitio “delimitation”) means pinning down a concept or delimiting the meaning or use of a linguistic expression. A definition may be lexical or stipulative, where the former indicates existing language use (i.e., the definition provided in dictionaries and encyclopedias) and the latter indicates how we ourselves intend to define the concept (Hansson, 2007a).

How may a scientific study contribute?

The overall motive for carrying out scientific studies is to contribute with new knowledge. A contribution is an effort forming a part of a larger whole.

A contribution does not have to represent a revolutionary change meant to solve all problems. A contribution may consist of a smaller effort within an area of knowledge, which, together with the efforts of others, contributes to the development of a field. A single scientific study may be looked upon as a grain of sand on a beach, a small but nonetheless important part – without grains of sand, there is no beach. In other words, carrying out a scientific study is not (just) about us learning something new but also that new knowledge is added to already existing knowledge within a certain area.

In an applied area such as engineering science, researchers are expected to make both academic and practical contributions. Academic contributions are made in relation to the scientific community, whereas practical contributions are made in relation to the part of society representing the intended recipient, which is frequently the industrial sector in the case of engineers.

Academic contribution An academic contribution is frequently related to theories in one way or another (Boer et al., 2015). However, aiming to contribute with new theory

3. T he path from pro b lem to solu tion

© T H E A U T H O R S A N D S T U D E N T L I T T E R A T U R

62

63 is very ambitious and, fortunately, there are also other ways of contributing to the development of knowledge in a field. Models, frameworks and taxonomies represent some examples of academic contributions, which in themselves do not necessarily satisfy the requirements for a theory but which may in the long term lead to the development of new theories. Let us begin by defining theory to then provide examples of other types of academic contributions.

© T H E A U T H O R S A N D S T U D E N T L I T T E R A T U R

What is a theory?

The word theory originates from the Latin word theoria, which means “things looked at,” “contemplation.” A theory is a statement on the relationship between concepts within the framework of formulated assumptions and limitations, a way of using language in order to organize a complex reality and render it possible to communicate (Bacharach, 1989). Criteria characterizing theories include that they contain defined concepts, relationships and conditions for separating concepts from one another. Theories should also be interesting (non-trivial) and enable predictions or provide an increased understanding (Meredith, 1993; Wacker, 1998). A theory should indicate the object or phenomenon being studied, how it should be perceived, its essential features, how various factors are linked together and how they may be explained. A theory indicates relationships between various pheno­ mena in the structure in which they are arranged. Theories also include explanations, which may be different in nature, from causal relationships (cause-effect) to explaining functions (Wallén, 1996). Furthermore, theories may sometimes be expressed as a formula, a mathematical relationship, as in Newton’s equation of motion: F = ma, meaning that force (F) is equal to mass (m) multiplied by acceleration (a).

Creating, testing and refining theories

A theoretical contribution does not need to consist of new theory; rather, the most common approach is to test or refine an existing theory. The contribution may constitute a part of the development toward a potential future theory. Creating theory may be described as an iterative process that includes describing, explaining and testing. The first step is to examine which areas may be suitable and relevant for developing theory and to adopt an exploratory approach, see Table 3.1. This is followed by a variety of different activities in relation to describing the area and initiating the process of developing

3. T he path from pro b lem to solu tion

64 theory. Eventually, theories are developed by identifying relationships that may provide possible explanations, which, in turn, are tested and refined (Handfield & Melnyk, 1998; Karlsson, 2016).

Step

Purpose

Possible questions

Discover an area to study

Identify areas to study and where we may develop theories.

Is there something here that is ­sufficiently interesting to study?

Is this area sufficiently interesting to justify the development of theory?

Describe the area and initiate the process of developing theory, formulate a tentative theory

Identify and describe key variables and their relationships while also explaining causes behind identified relationships.

Which are the key variables?

Which relationships and patterns are found among the key variables?

Which is the suspected cause behind these relationships and patterns?

Test developed theories

Test the tentative theory and determine whether future outcomes may be predicted on the basis of this theory.

Do the tentative theories survive being tested against empirical data?

Was the result of the test to be expected based on the tentative theory?

Refine developed theories

Determine the framework within which the theory is valid and extend its area of application.

To what extent is the theory ­generalizable?

In which contexts and in which situations is the theory valid/not valid?

Theories may be more or less generalizable: from empirical generalizations, via mid-range theories to what is sometimes referred to as grand theories (Swamidass, 1991). Empirical generalizations could serve as a first step in developing a theory, similar to a mid-range theory. A mid-range theory may frequently explain an aspect of a phenomenon. An example of this is the product-process matrix presenting the most advantageous combination of production process and product structure (Hayes & Wheelwright, 1979), see Example 3.2.

A grand theory is useful in a variety of situations and in several areas, but it offers less detail compared to a mid-range theory (Bacharach, 1989). An example of a grand theory is systems theory, which is used in engineering science for describing technical systems (Eder, 2016; Hubka & Eder, 1988).

Technical systems theory is based on a general transformation system, see Figure 3.5, which consists of a transformation process and a number of sub-

3. T he path from pro b lem to solu tion

© T H E A U T H O R S A N D S T U D E N T L I T T E R A T U R

TABLE 3.1 Different steps in the development of theory (based on Handfield & Melnyk, 1998; Karlsson, 2016).

65

E X A M P LE 3. 2 M I D - R A N G E T H E O RY

An example of a mid-range theory in the field of operations management is the product-process matrix, which consists of the two dimensions product structure and production process as well as their relationship, see the figure below. The product structure is described from low-volume and individual units to high-volume and standard. The production process goes from fixed position via functional workshop and production line to continuous process.

Product structure/mix

Fixed position

Production process

Š  T H E A U T H O R S A N D S T U D E N T L I T T E R A T U R

Low volume Low-medium One piece volume Non-standard Many products

Functional workshop

Medium-high High volume volume Standard Few products products

Buildings, ships Wheel loaders Cars

Production line

Continuous process

Chainsaws Goal is increased flexibility

Paper

The diagonal section in the matrix corresponds to the most suitable combination of production process and product structure.

When manufacturing a limited number of units, a fixed position or functional workshop is suitable, while producing high volumes in a continuous process is more cost-effective. Hayes & Wheelwright (1979), who introduced the product-process matrix, also stressed the life cycle aspect, assuming that a product is initially manufactured in low volumes to then increase in volume during its life cycle. The aim is to move in the direction of the arrow, to efficiently produce products only manufactured in a limited number in a production line.

Sources: Hayes & Wheelwright, 1979, Olhager, 2013.

3. T he path from pro b lem to solu tion

23 mm

SÄFSTEN GUSTAVSSON | RESEARCH METHODOLOGY

Kristina Säfsten is a professor of production systems at the School of Engineering, Jönköping University, as well as a visiting professor at Mälardalen University.

Maria Gustavsson is a professor of education at the Department of Behavioural Sciences and Learning, Linköping University.

The two authors have many years of experience teaching research methodology.

RESEARCH METHODOLOGY FOR ENGINEERS AND OTHER PROBLEM-SOLVERS This book covers the entire research process: from formulating a research problem to presenting and reviewing the results. It is intended to be used in methodology courses in engineering programs on the basis of the scientific tradition in engineering and how engineers have historically been trained. The book presents a selection of research methods, as well as engineering perspectives on the philosophy of science, research methods and techniques for data collection. In addition to explaining and presenting examples of key concepts in the methodological landscape, it also discusses ethics, quality and communication.

The book is about how to plan, execute and review degree projects and other scientific studies. By presenting several practical examples on how to carry out scientific studies in the field of engineering, the book gives the reader a better understanding of research methods that are particularly relevant to engineers.

RESEARCH METHODOLOGY FOR ENGINEERS AND OTHER PROBLEM-SOLVERS

KRISTINA SÄFSTEN MARIA GUSTAVSSON

Research Methodology – For Engineers and Other Problem-Solvers can also be used as an introductory book on methodology in other disciplines with a focus on problem-solving. The book is suitable for both introductory and postgraduate levels.

Art.nr 39758

studentlitteratur.se

978-91-44-12230-4_cover.indd All Pages

2020-01-21 08:12