Page 1

Applying a Behavioural Simulation for the Collection of Data Kristina Risom Jespersen The Aarhus School of Business, Department of Marketing and Statistics, Denmark krr@asb.dk http://www.asb.dk/staff/mis/krr.aspx Abstract: To collect real-time data as opposed to retrospective data requires new methodological traits. One possibility is the use of behavioral simulations that synthesize the self-administered questionnaire, experimental designs, role-playing and scenarios. Supported by Web technology this new data collection methodology proves itself valid and with high appeal to respondents. Keywords: Real-time data collection, simulation, Web technology

1. Introduction As the complexity of problems and the speed of changes increase for companies, the accuracy of research relies on the measurement of behavior close to the real world of the respondents. Recent reviews in various research fields of the social sciences have pointed at the need for more interactive data collecting methods (Zaltman 1997; Englis and Solomon 2000). The collection of real-time data as opposed to retrospective data requires new methodological traits. Another aspect stressing the development of new data collection methodologies is the prediction that response rates of e-mail surveys will follow that of paper surveys (Shenan 2001). To obtain data, to make people invest time in research projects, research has to follow the information ‘rules’ of society. Here the increased number of media, the habitually use of the Internet and mobile phones as well as a TV set on multiple channels, makes heavy calls on research to be an interesting experience for researchers to be able to collect the important and necessary data. Not because people do not find science important but because time is a scarce resource in heavy demand. Hence, investing time in research projects has to give the respondents something more than being polite and helpful towards the researcher. The purpose of this paper is to discuss and demonstrate the benefits and challenges of behavioral simulations supported by Web technology as a new methodological trait. Specifically how behavioral simulations as data collecting vehicles also act as learning tools for the respondents thereby giving immediate return in the time invested. The concerns of this paper are the methodological contribution of simulations, the

benefits of Web technology followed by a discussion of the validity of data collected with simulations. Finally, a designed behavioral simulation is presented as an example, before stating implications and future trends.

2. The contribution behavioural simulation

of

a

To understand the methodological contribution of a simulation it is beneficial to look at the research area of agent-based simulations. The first agent-based computational economic models were influenced by experimental economics, which at the time mainly focused on markets and games composed of real human agents (Chen 2004). A criticism of experimental economics on information utility states that the problem lies with the utility functions and whether these resemble the true utilities of information, because human agents are subject to biases, errors, and misconceptions (Einhorn and Hogarth 1981; Feldman and March 1981; Chaturvedi, Mehta et al. 2004). Parallel to this, behavioral economic research proved that the biases creating inference in experimental economics are similar to those in real life. Humans optimize their behavior as assumed in experimental economics, but from a satisfying criterion because humans are bounded rational (Newell and Simon 1972; Bettman, Payne et al. 1993; March and Simon 1994). For the understanding of the effect of bounded rationality in economic models, the research area of agent-based simulations emerged (Chen 2004) as a way to observe actual behavior as opposed to behavior deduced from a set of axioms. It is this objective to observe actual behavior that makes simulations a useful platform for the need to collect real-time data in social sciences. The spin-off from behavioral simulations is that participants are able to learn from them; thereby the models exhibit intelligence

ISSN 1477-7029 141 ©Academic Conferences Ltd Reference this paper as: Risom Jespersen K (2005) “Applying a Behavioural Simulation for the Collection of Data” The Electronic Journal of Business Research Methodology Volume 3 Issue 2, pp 141-148, available online at www.ejbrm.com


Electronic Journal of Business Research Methods Volume 3 Issue 2 2005 (141-148)

(Potgieter and Bishop 2002; Wahle, Bazzan et al. 2002; Boer, Ebben et al. 2003; Chaturvedi, Mehta et al. 2004; Hare and Deadman 2004). Hence, a behavioral simulation makes it possible to collect data on how decisionmakers apply and learn from information in the decision-making processes in companies.

person’s perception of a situation. Participants are asked to engage in the role description and then either to imagine their actions or to act them out - in both cases as they would in fact do, i.e. managers should not play customers or vice-versa. The key is to make the role realistic (Eroglu 1987; Armstrong 2000). Therefore, with role-play the decisionmaker in the simulation can be specified. Table 1 provides some basic design principles. Table 1: Basic design principles for roleplaying* (Jespersen 2004; Armstrong

Adapting the knowledge on agent-based modeling (Chaturvedi, Mehta et al. 2004; Chen 2004), a behavioral simulation synergizes the benefits of self-administered questionnaires and the experimental design, and furthermore introduces role-playing (Dabholkar 1994; Armstrong 2000) and scenario (Frederickson 1984; Frederickson 1985; Eroglu 1987; Sanderson and Sanderson 2000; White, Varadarajan et al. 2002) strategies as very effective methods to ensure high interaction with the respondents.

2000) The role-players should be similar to those being represented. Meaning that a role-player should act as him/herself. Role-players should read instructions for their roles before reading about the situation. The administrator: provides short yet comprehensive descriptions and creates realistic surroundings in order to provide a realistic enactment of the situation.

The aim of the questionnaire is two-fold in ensuring that information about the decision behavior in companies is retained and in validating the observed behavior of respondents in the simulation as to whether it resembles behavior in companies. Hence, with the questionnaire the specification of the decision parameters is ensured.

The scenario is the situation in which the roleplay is acted out by the participants and is therefore the heart of the simulation. The strength of using scenarios as frames for the decision-making process is that it makes the respondents relate more directly to the posed subjects, and to a high degree this results in more accurate responses (Eroglu 1987). The ability of a scenario is threefold in that it (i) increases interest in participation, (ii) makes it possible to create a realistic context, and (iii) provides all respondents with a standard stimulus (Frederickson 1984; Frederickson 1985; White, Varadarajan et al. 2002). Also when respondents are presented with a scenario before decision-making, their attention is guided to the relevant problem area. From this viewpoint, the scenario can successfully complement the experimental design by providing control of the decisionmaking process investigated. Thus, the scenario defines the behavior of the artificial reality in the model.

Where a questionnaire represents the scope, an experiment focuses on depth by offering the opportunity to give a standardized and controlled presentation of the surroundings (the environment) of the decision situation (Perkins and Rao 1990). Hence, contrary to the questionnaire the experiment introduces activity and dynamics though the setting is more simplified than the ‘real world’; for example, the technological development, the competitive environment, or the uncertainty of demand. As such exogenous factors to the decision-making process can be specified and controlled in the simulation through experimental design. Using the questionnaire for measurement and the experiment for variation, these two methodologies provide the environment for the simulation. To make the simulation interactive role-playing and scenarios are essential methods.

The methodological contribution of simulations is a symbiosis of four acknowledged research methods. At the same time, these methods make the simulation very complex. Thus the key to make a simulation suitable for data collection purposes is Web technology, because Web technology can carry out the large database structure upholding the simulation and making the simulation active and dynamic. Therefore, the possibilities with a simulation as a platform for a data collection methodology are closely linked to the use of Web technology when collecting data.

Traditionally, role-playing has been used to forecast decisions in conflict situations among interacting groups, but it can also be used to predict decisions by an individual not interacting directly with others (Armstrong 2000). More importantly the similarity between laboratory research and role-playing is well documented (Dabholkar 1994). The advantage of role-playing is that roles influence a

www.ejbrm.com

142

©Academic Conferences Ltd


Kristina Risom Jespersen

Another important data collection issue is item completion. Here the Web technology is powerful. Item completion can be simplified through the use of pull-down menus, check boxes, radio button scales, and drop-down scales. These options really make the webbased approach user-friendly. Furthermore, the pages can be programmed to check whether all items have been completed before giving access to the next page, thereby eliminating missing values. Additional four advantages of the web technology can be listed: (i) it is easy to modify the research instrument or to create multiple experimental versions, (ii) the automated data collection eliminates coding errors, (iii) reduction of experimenter effects, and (iv) the ability to reach a larger and more diverse subject pool. But also, the following disadvantages have appeared: (i) the potential for systematic bias and measurement error and (ii) the unwillingness of respondents to provide sensitive information over what they view as insecure lines (Birnbaum 2000), as well as iii) the issue of technical variance among respondents and iv) the issue of a selfselected sample (Reips 2000). Thus, as with any data collection methodology care and consideration should be demonstrated in order to maintain data validity.

3. The benefits of Web technology The use of computers for the collection of data through various methods is well known. As early as in the 1970s the first computerized experiment was launched. Especially within the field of decision-making, experiments have been computer-interactive from early on (Connolly and Thorn 1987). In a study comparing Web-versions with laboratory studies, Krantz and Dalal (2000) found a surprising match for surveys, scales and experimental variables thereby stressing the power of the Web as both a research medium and as experimental medium. For research purposes the computational intelligence available with the Web is still by and large an unexplored territory in social sciences (Stanton 1998; Birnbaum 2000; Englis and Solomon 2000; Reips 2000; Klassen and Jacobs 2001; Jespersen 2004). Declining response rates – often termed ‘survey fatigues’ – is of major concern to research. The use of emails has made it easier to target respondents but also easier for respondents to opt-out of research studies (Stanton 1998; Reips 2000; Sheehan 2001). Collecting data with simulations by use of Web technology places the data collection methodology in a known medium for playing games and being active. Hence the appeal of behavioral simulations can be expected to be higher due to Web technology (Stanton 1998; Birnbaum 2000; Englis and Solomon 2000; Reips 2000; Klassen and Jacobs 2001; Jespersen 2004) visualizing the virtual decision-making process. Additionally, as games give players satisfaction from playing, so simulations will provide respondents with satisfaction of a problem-solving experience close to the real worlds of the respondents. Other possible reasons why the response rate may increase with Web technology are (i) the minimization of the response time for participants, (ii) the ease and flexibility of participation since the Internet is available twenty-four hours, seven days a week, and (iii) the lower cost for participants (Birnbaum 2000; Coderre and Mathieu 2004). Still, the experience from email surveys that response rates declines just as for postal surveys may indicate to researchers that they should be very careful about the design of the data collection on the Web. Hence, the design of a Web-based methodology has to be very useroriented. If the design becomes too heavy or too boring, then participants will drop out as they do on regular web pages.

www.ejbrm.com

4. Ensuring the validity of data For Web technology supported simulations to become a new methodological trait, the data collected by means of this new research strategy are to be proved valid. Applying Web technology to the data collection vehicle raises the issue of convergent validity compared to a laboratory based study. Comparing laboratory and Web versions of surveys, scales and experimental variables, Krantz and Dalal (2000) find a remarkably consistent correspondence between the two, thereby proving the validity of Web technology as part of research strategies. A notion supported by Coderre and Mathieu (2004) finding that the predictive power of information gathered by email survey outperformed that observed with postal or telephone collected information. Another issue of internal validity is construct validity by which the results of the data collected with simulations follow theoretically predicted trends. Paying attention to the details when creating the virtual world of the simulation will ensure construct validity. Research using agent-based computational modeling proves that this is not a critical issue for simulations though much lies with the

143

ISSN 1477-7029


Electronic Journal of Business Research Methods Volume 3 Issue 2 2005 (141-148)

thoroughness of the researcher (Wahle, Bazzan et al. 2002; Boer, Ebben et al. 2003; Chaturvedi, Mehta et al. 2004).

5. Info@performance.NPD The behavioral simulation as the data collection instrument example was developed for a study of new product development decision-making (Jespersen 2004). The aim was to gain greater insights concerning the value of information when new products are screened in the various phases of the new product development process. The analysis of the information value information twodimensional including both the value-formoney and the decision-value of information. Hence, the behavioral simulation focused on the information processing cycle of decisionmakers, i.e. whether information was acquired, and whether the acquired information was used for the screening decisions.

The external validity of data collected with simulations is very critical for the generalization of the research results. Despite the many benefits of a simulation it is a constructed reality that the respondents enter. This constrains the analysis results and introduces the possibility of explaining nothing but the behavior within the simulation. To counter this criticism several validation questions should be designed into the simulation to observe whether the simulated managerial decision-making process resembles the actual decision-making processes in the participating companies. Still, it must be expected that some results can be explained by the behavior of the respondents being influenced by the simulation as are decision-makers in specific decision-making situations in companies. Furthermore to validate data, knowledge of various decisionmaking characteristics should be obtained from the respondents using a questionnaire mapping relevant environmental and company moderators of the behavior of the respondents.

5.1 The model: A virtual NPD process A virtual NPD process was created in which the participants were asked to acquire information for the evaluation of a new product idea as the product idea moved through the NPD process from idea to market planning. Hence, the simulation ended with the decision whether to launch or not. Because the game scenario used the Internet as medium, it was named Info@performance.NPD. Figure 1 depicts the structure of the simulation.

Combining simulations and Web technology introduces a new type of validity known in agent-based modeling as solution validity (Chaturvedi, Mehta et al. 2004; Jespersen 2004). Solution validity ensures the compatibility between the empirical data collection methods and the model solution in order to secure reliable and valid data. This means restraining the technological possibilities. Though a model of a real decision-process, some constraints have to be imposed on Web technology. A simple example is the habitually used back and forward buttons on the Internet. If these are not locked, the respondents would be able to go back and alter decisions without the researcher’s knowledge. Such a simple design consideration would jeopardize the data validity.

Your role

Log-on

MacVic

The Budget

Starting questionnaire

Case

Information acquisition

Receiving information products

Ending

Questionnaire

Thank you

Information valuation

Forecasting performance

IDEA Intermediate CONCEPT

Exit Intermediate

PROTOTYPE Exit Intermediate PRODUCT

Exit Intermediate Exit

MARKET PLAN Intermediate

Figure 1: The simulation structure The starting questionnaire contained questions on new product development in the companies related to market orientation, strategy, budget, new product evaluation and information use. Then, the participants were asked to play the role of the new product development manager in the company MacVic1 and were given a job description. Here after followed a brief account of the new product development situation in MacVic followed together with an introduction of the virtual company MacVic. The description of MacVic focused on the external

The data validity issue is always critical when collecting data but the dimensions to consider are increased when Web technology is applied and several methodologies are combined into one. Still, behavioral simulations will not jeopardize the validity of data collected, if designed properly.

1

MacVic is a fully imaginary company invented for the

simulation.

www.ejbrm.com

144

ŠAcademic Conferences Ltd


Kristina Risom Jespersen

primary technical information. Furthermore, the information products were ensured a realistic touch through discussions of content, format and costs with professional market research companies. Additionally, the level of competitive stress facing MacVic and as such the participants were designed experimentally and regulated at three levels – low, medium, and high. The participants would encounter competitive stress in the concept and the prototype phases. To measure whether the acquired information was useful to the decision-making each phase ended with the participants evaluating the potential of the new product idea in the simulation on a scale from zero to 100.

environmental conditions under which MacVic conducts business. As part of the simulation, the participants were given a monetary budget as they would have in their companies. The budget ensured that the participants had to decide among the different information products, since it was only large enough to cover two-thirds of all the information products in the simulation. The size and use of the budget were validated with a cross analysis of questions from the starting questionnaire. The simulated NPD process in the game scenario contained a thorough description of the status of the product idea in the five phases of the development process (idea, concept, prototype, product, and market planning) along with a total of thirty-six information products available at different points in the product development process. The available information products that the participants could acquire were designed in accordance with the ’best practices’ guidelines developed by research on NPD success and failure (Booz, Allen et al. 1968; Cooper and Kleinsmidt 1987; Souder 1989; MontoyaWeiss and Calantone 1994; Brown and Eisenhardt 1995; Cooper and Kleinsmidt 1995; Crawford 1997; Benedetto 1999; Cooper 1999; Henard and Szymanski 2001). Hence the information products were different in each phase of the NPD process, but always represented the four information types secondary market information, primary market information, secondary technical information, Info @p erforma nce .N P D

St art i ng Quest i on nai re Int rodu ct oi n

To illustrate the game scenario Info@performance.NPD, figure 2 shows the concept phase of the NPD process in the simulation. From these web-pages it should be possible to get an idea of how the simulation worked. The first Web-page is a description of the product development in the concept phase (picture 1) - what had happen to the product idea since the idea phase. Then the participant can choose from a list of information products (picture 2). The participant rates each information product on an importance scale. The participant then receives the ordered product (pictures 3-4), and rates the value of the individual information product (picture 5). Finally, the participant is asked to evaluate the product idea (picture 6). In fo@ perfo rman ce. NPD

T h e C on c e p t P ha s e

Hav ing mad e th e ini tial eval uati on of the i dea o f a n in tell igen t m ixer co mb ined wit h an el ectr ical boi ler , yo u ha ve assem bled you r pro duct dev elop men t t eam a s usual wit h peo ple fr om m ar keti ng, eng ineer ing and pr odu ctio n t o a d iscussion of th e a po ssi ble conce pt b ased o n th e pr o duct ide a. This is wh at t he d iscussion gav e:

S t arti ng Qu est oi nna ri e Int rod uct i on

T he New Pro du ct

T h eNew P rod uct

Devel op men t P roc ess

Dev el opm ent Pr ocess

Ph ysical per fo rm ance requ ir emen ts:

Con su mer be nef its:

•The boi ler and the mix er sh oul d sto p au tom atic ally .

•Ease.

•A sensor shou ld be m oni tor in g th e tem per atu re.

•O ne i nstead of two m ach ines.

- Id ea - Co ncep t B uy Informat i on

1)

R ecei ve Informa it on Ev al uat e ht e prod uct - P rot ot yp e - P rodu ct - M arket Pl ann ni g

En di ng

T h e C on c e p t P ha s e

T o a i d your eva lua t ion of t he produc t i dea you ha ve the opti on of a cqui ri ng i nform at i on. B el ow is a l ist of i nformat ion produc ts you ca n buy. You are al so asked t o eval ua te ea ch product whe ther y ou buy it or not .

- Idea

•The same sen sor sh oul d b e pr ogr am mab le b y th e user , i.e. let sau ce bo il f or on e m inu te wh ile st ir rin g. •The mi xer shou ld h ave f lex ibl e rh yth ms f itt ed t o th e use. •The soft war e shou ld ensur e sim pli city but at t he sam e tim e h igh user fr iend lin ess.

Pr oduc t importa nce to you

B u y Inform ati on

Informa tion pr oduc t

R ece vi eIn format i on

Pri ce (in D K R)

(mar k f or No t

Of l ti t l e

Of s ome

im po rt ant

i mpo rt an ce

i mp ort a nce

25 , 00 0

"

"

"

"

"

"

Imp ort an t

Ve ry i mp ort an t

ye s )

E val ua et t he pro duc t

•F lex ible cook ing .

- Pro to ty pe

Is t h ere a n u n us ed po si t i o n i n t he

- Pro duct

•The same r evo lut ion as w ith the mi cro wave oven .

ma rk et …

2)

- Mark et P la nni ng 55 , 00 0

"

"

"

"

"

"

90 , 00 0

"

"

"

"

"

"

T ec h no l og y co ns i de rat i o ns

4 , 00 0

"

"

"

"

"

"

Su p pl y / v en do r c oo p era t oi n

10 , 00 0

"

"

"

"

"

"

Sa l es e x pe ct a t oi n s/ fo rec as it n g

20 , 00 0

"

"

"

"

"

"

C o mp et i t i v e a na l ys is

10 , 00 0

"

"

"

"

"

"

T he op i n i on of c us t om ers T h e n eed fu l fi l l me nt b y t h e p ro du ct

•S op histi cated pro duc t li ke the expr esso mac hin e.

i de a

E nd ni g B u dge t 5 36. 85 0

•The re sho uld be a tou ch scr een f or pr og ram mi ng whi ch ca n stan d stai ns. Pr ice:

Nam es:

•S om ewh ere i n t he r ange of DKK 1, 800 - 4 ,00 0

•S ma rt Cook ;

BePr oCo o k; A ID toCO OK 1

Info @p erforma nce .N P D

Buy

- Con cept

2

In fo@ perfo rman ce. NPD

T h e C on c e p t P ha s e

St art i ng Quest i on nai re

S t arti ng Qu est oi nna ri e

Int rodu ct oi n

T h e C on c e p t P ha s e

You ha ve bought t he fol lowing informa ti on

Int rod uct i on

You have bought : T he New Pro du ct

T h eNew P rod uct

Devel op men t P roc ess

Dev el opm ent Pr ocess

- Id ea

Te c hnology c onsi dera t ions

3)

I n t he d evelo pm ent d ep art men t in MacV ic th e op ini on i s that th e con cept sh oul d bu ild on two new t echno log ical appl ica tio ns:

- Idea

- Co ncep t

- Con cept

B uy Informat i on

B uy In fo rmat oi n

Recei ve In forma it o n

Rec eiv e Infor mati on

Ev al uat e ht e prod uct

E val ua et t he pro duc t

- P rot ot yp e

- Pro to ty pe

- P rodu ct

- Pro duct

- M arket Pl ann ni g

- Mark et P la nni ng

1 ) A n i ntel lig ent G SM - mo dul e that han dles er r ors a nd d efect s bef or e the custo mer r egister t he pr ob lem . The GS M-m od ule can disti ngu ish b etwe en cr iti cal an d no n- cri tical pr obl ems. When the mac hin e r egi st ers t hat a cri tica l p ro blem is em er gin g th en i t cal ls the servi ce an d sup por t te chni cians a nd r epo rt

4)

t he er ro r. Th e er ror - cor r ectio n is t hen e-m ail ed to th e custo mer . Thi s system uses th e lat est m obi le t echno log y.

En di ng

E nd ni g

2 ) A HA CCP system , whi ch ar e a p rev enti ve syste m - a t ool that ensur es th e hy gien e stand ard s in the m achi ne t hro ugh th e u se pr ocesses. P resen tly one of Mac Vic’ s col labo rat ive p art ner s has th e pat ent for th ese tech nol ogi es. Thus b efo re co nti nui ng t he d evelo p men t th is par tn er sh ould be cont acted fo r a d iscussio n of th e po ssi bil iti es. It sho uld be n ote d th at t he use of th ese tech nol ogi es wou ld r equ ir e a li ttl e adju stm ent o f t he p ro duct ion p roc ess. Yet anot her t hing to inv estig ate f ur ther .

3

www.ejbrm.com

4

145

ISSN 1477-7029


Electronic Journal of Business Research Methods Volume 3 Issue 2 2005 (141-148)

Info @p erforma nce .N P D

In fo@ perfo rman ce. NPD

T h e C on c e p t P ha s e

St art i ng Quest i on nai re Int rodu ct oi n

- Id ea - Co ncep t B uy Informat i on

5)

Recei ve In forma it o n Ev al uat e ht e prod uct - P rot ot yp e

Int rod uct i on

I nforma tion va luation

T he New Pro du ct Devel op men t P roc ess

T h e C on c e p t P ha s e

S t arti ng Qu est oi nna ri e

T he info rma tio n w a s r e la te d to the d ec is ion at ha nd T he infor ma tion pr ovide d w a s w ort h the m one y spe nt on it T he info rma tio n a ddre s se d i ss ue s t ha t w e r e une xpe c ted.

F ul l y

P art l y

d i sag ree

d i sag ree

Nei t h er

P art l y a gre e

F ul l y a gre e

"

"

"

"

"

"

"

"

"

"

"

"

"

"

"

"

"

"

At t his point i n t he ne w produc t deve lopme nt proce ss, how do you perc ei ve the ne w product ’s cha nc e s for s ucc ess on t he m arke t in perc ent ?

Wha t is your m os t opt i mi st ic e sti m at e ?

____%

Wha t i s your m ost pes sim is ti c e sti ma t e?

____%

Under norm a l ci rcum sta nce s, wha t is t he li kel ihood of you la unchi ng the product a t t hi s poi nt ?

____%

Do n ’t k no w T h eNew P rod uct Dev el opm ent Pr ocess - Idea - Con cept B uy In fo rmat oi n R ece vi eIn format i on E va lu ate the p rod uct - Pro to ty pe

- P rodu ct

- Pro duct

- M arket Pl ann ni g

- Mark et P la nni ng

En di ng

E nd ni g

____%

5

6)

6

* Note that the screen layouts are suggestive and that the final professional touch was given only on the Danish version.

Figure 2: The concept phase simulation in Info@performance.NPD the information products did not obscure the decision-making process. The particpants evaluated the simulation on four dimensions – coherent and easily comprehended new product development process, realistic content, exiting participation, and time consumption (see figure 3).

5.2 Sampling The sample constitutes larger International companies having their R&D unit placed in Denmark. The targeted companies develop either high or low technology products for the consumer market. Low technology products were represented by the food industry whereas high technology products were represented more diversely by industries such as telecommunication, personal computers, kitchen hardware, speakers, washing machines, tumblers, and headphones. The selection criterion was that the company did product development of consumer products in Denmark. This reduced the population of high technology companies tremendously, and introduced another constraint on the low technology companies: The food companies had to be of a reasonable size because primarily large high technology companies are performing new product development in Denmark. The participants were found either on basis of their job title as R&D manager/director or through organograms from which it was possible to determine whether it would be relevant to contact the marketing, engineering or project manager. A total of forty-two companies used the simulation.

Averag e agreem ent

5 ,0

3 ,0

2 ,0

1 ,0

Ve ry

i ti

l ea

co

do

m

od pr

g

s

es

in

oc pr

e as ph

D

u ns

fo in

e

to

t ic

ti m

ng

is

P

PD

N

N

nt

ti c

e er

l is ea

oh

Ex

R

R

C

s

Figure 3: Appeal of the behavioral simulation In general the simulation was found fun to do and very realistic. The only negative aspect was as expected the time used for participation. Though the measured time for participation was sixty minutes with the possibility to break this time down in minor intervals, the reaction to time from the participants was expected as time is always critical. Despite of this several participants expressed their positive experience of science going new ways. All of which seems promising for behavioral simulations as a new methodological trait.

5.3 Data validity Two important aspects of the simulation as a data collection methodology are data validity and the judgment by the participating companies. Data validity was found through analyses of i) the budget, which showed that the participants used the same amount of money in the simulation as they would have in their companies, ii) the price of the different information products, where the relation between importance and the acquisition of an information product demonstrated a reasonable pricing in the simulation, and iii) the information utility of the individual information products. With high average ratings for all information products on the dimensions – relevance, quality and novelty –

www.ejbrm.com

4 ,0

6. Implications and future trends A behavioral simulation is a suitable methodology for the collection of real-time data in all fields as it provides high interaction with respondents and immediate pay-back

146

©Academic Conferences Ltd


Kristina Risom Jespersen

through learning for the time invested in participation. Furthermore simulations appeal to respondents by applying Web technology. Also, the combination of several methodological traditions is found beneficial for future methodological developments as this will also encourage new collaborations among research fields thereby enhancing the research contribution of future research.

Coderre, F. and A. Mathieu (2004). "Comparison of the quality of qualitative data obtained thorugh telephone, postal and email surveys." International Journal of Market Research 46(3): 347-357. Connolly, T. and B. K. Thorn (1987). "Predecisional Information Acqusition: Effects of task variables on suboptimal search strategies." Organizational Behavior and Human Decision Processes 39: 397-416. Cooper, R. G. (1999). "From Experience: The Invisible Success Factors in Product Development." Journal of Product Innovation Management 16: 115-133. Cooper, R. G. and E. J. Kleinsmidt (1987). "Success factors in product innovation." Industrial marketing management 16: 215-223. Cooper, R. G. and E. J. Kleinsmidt (1995). "New Product Performance: Keys to Success, Profitability & Cycle Time Reduction." Journal of Marketing Management 11: 315-337. Crawford, C. M. (1997). New products management, IRWIN. Dabholkar, P. A. (1994). "Incorporating Choice into an Attitudinal Framework: Analyzing Models of Mental Comparison Processes." Journal of Consumer Research 21(June): 100-118. Einhorn, H. and R. Hogarth (1981). "Behavioral Decision Theory: Processes of Judgement and Choice." Annual review of Psychology 32: 53-88. Englis, B. G. and M. R. Solomon (2000). "LIFE/STYLE ONLINE: A web-based methodology for visually-oriented consumer research." Journal of Interactive Marketing 14(1): 2-14. Eroglu, S. (1987). The Scenario Method: A theoretical, not theatrical, approach. AMA Educators Proceedings. Feldman, M. S. and J. G. March (1981). "Information in Organizations as signal and Symbol." Administrative Science Quarterly 26: 171-186. Frederickson, J. W. (1984). "The comprehensiveness of strateic decision processes: Extension, Observation, future directions." Academy of Management Journal 27(3): 445-466. Frederickson, J. W. (1985). "Effects of decision motive and organizational performance level on strategic decision processes." Academy of Management Journal 28(4): 821-843. Hare, M. and P. Deadman (2004). "Further towards a taxonomy of agent-based simulation models in environmental

Of course behavioral simulations are to be used with extreme care to both design and sampling in order to ensure validity as well as generalizability. Also the speedy technological development imposes a great challenge on researchers and as such the behavioral simulation can only be expected to be a continuous development to preserve its high appeal.

References Armstrong, J. S. (2000). Role playing: A method to forecast decisions. Principles of forecasting: A handbook for Researchers and Practitioners. J. S. Armstrong, Norwell, MA: Kluwer Academic Publishers, 2000. Benedetto, C. A. D. (1999). "Identifying the key success factors in new product launch." Journal of Product Innovation Management 16: 530-544. Bettman, J. R., J. Payne, et al. (1993). The Adaptive decision maker, Cambridge University Press. Birnbaum, M. H. (2000). Psychological Experiments on the Internet, Academic Press. Boer, L. d., M. Ebben, et al. (2003). "Studying purchasing specialization in organizations: a multi-agent simulation approach." Journal of purchasing and supply management 9: 199-206. Booz, Allen, et al. (1968). Management of new products, Booz, Allen and Hamilton. Brown and Eisenhardt (1995). "Product development: past research, resent finding and future directions." The Academy of management review 20(2): 343-78. Chaturvedi, A., S. Mehta, et al. (2004). "Agentbased simulation for computational experimentation: Developing an artificial labor market." European Journal of Operational Research(In press). Chen, S.-H. (2004). "Computational intelligence in economics and finance: Carrying on the legacy of Herbert Simon." Information Sciences(In press).

www.ejbrm.com

147

ISSN 1477-7029


Electronic Journal of Business Research Methods Volume 3 Issue 2 2005 (141-148)

Reips, U.-D. (2000). The Web Experiment Method: Advantages, Disadvantages, and Solutions. Psychological Experiments on the Internet. M. H. Birnbaum, Academic Press: Chapter 4. Sanderson, A. and S. Sanderson (2000). Design and Manufacturing Learning Environment: Innovation and product family management. Sheehan, K. (2001). "E-mail Survey Response Rates: A review." Journal of Computer Mediated Communication 6(2). Souder, W. E. (1989). Managing new product innovations. Lexigton, Mass., Lexigton Books. Stanton, J. M. (1998). "An Empirical Asssessment of Data Collection using the Internet." Personnel Psychology 51(3): 709-725. Wahle, J., A. L. C. Bazzan, et al. (2002). "The impact of real-time inforamtion in a tworoute scenario using agent-based simulation." Transportation Research Part C 10: 399-417. White, J. C., P. R. Varadarajan, et al. (2002). "The effect of Cognitive Style and Perceived Organizational Culture on Managers Interpretation of and Response to Marketing information." Journal of Marketing. Zaltman (1997). "Rethinking market research: Putting People Back in." Journal of Marketing Research 34(November): 424437.

management." Mathematics and Computers in Simulation 64: 25-40. Henard, D. H. and D. M. Szymanski (2001). "Why some new products are more successful than others." Journal of marketing research 38(August): 362-375. Jespersen, K. R. (2004). Information and New Product Development Decision-making. Department of Marketing, Informatics and Statistics. Denmark, The Aarhus School of Business. Klassen, R. D. and J. Jacobs (2001). "Experimental comparison of Web, electronic and mail surevey technologies in operations management." Journal of Operations Management 19: 713-728. March, J. and H. Simon (1994). Organizations, Blackwell Publishers, USA. Montoya-Weiss and Calantone (1994). "Determinants of New Product Performance: A review and metaanalysis." Journal of Product Innovation Management 11(11): 397-417. Newell, A. and H. A. Simon (1972). Human Problem Solving, Prentice-Hall, INC. Perkins and Rao (1990). "The role of experience in information use and decision making by marketing managers." Journal of marketing research 27(februar): 1-10. Potgieter, A. and J. Bishop (2002). Bayesian agensies in control. Computational Intelligence in COntrol. M. Mohammadian. Hersey, PA, USA, IDea Group Inc.: 168.

www.ejbrm.com

148

ŠAcademic Conferences Ltd

Applying a Behavioural Simulation for the Collection of Data  

To collect real-time data as opposed to retrospective data requires new methodological traits. One possibility is the use of behavioral simu...

Applying a Behavioural Simulation for the Collection of Data  

To collect real-time data as opposed to retrospective data requires new methodological traits. One possibility is the use of behavioral simu...