Simulation-Based Engineering of Complex Systems Dr. John R. Clymer, INCOSE Fellow

Module 3: Logic and Statistics Concepts for Simulations

University of Waterloo October 6-7, 2010

1

Module 3: Logic and Statistics Concepts for Simulations • Statistical use of OpEMCSS Blocks for Systems Engineering and Simulation • Case Study & OpEMCSS Blocks: Producer-Consumer • Summary 2

Convergence and the Law of Large Numbers • The Law of Large Numbers states that given a large enough population, a group will conform to statistical probability – Probability estimators (Sn/N) converge to true probabilities where SN number of successes and N is number of trials – Expected value estimators Σ(Xi/N) converge to true mean where Xi is a sample and N is the number of trials.

• Convergence means as we increase the pool of simulation runs, the closer to the expected statistical outcome we will be to the true value. 3

Communications Process: Example of the Convergence and the Law of Large Numbers P1*(1-P2)

P1*P2

(1-P1) P1*(1-P2) (1-P1)

KPPs are the probabilities and reaction times 4

Example of Probabilistic Decision: Alternate Action Block Dialog How to simulate a discrete probability distribution Q1 Q2 Q3 0

1

1. Draw a random number R 2. If R<Q1 Do action 1 else we eliminate Q1 region 3. If R< (Q1+Q2) Do action 2 else we eliminate Q1 and Q2 region and Do action 3 • First decision is IF ( Rnd < (P1*(1-P2)) ) DECISION = 1; • Second decision is IF ( Rnd < (P1*(1-P2)) + P1*P2) ) DECISION = 2; which is same as IF ( Rnd < P1) DECISION = 2; • Third decision is the default. 5

Law of Large Numbers: Convergence of Communication System MOEs/MOPs 160 140 120 100

Percent Block Time

80

Total Transmission Time

60 40 20

Number of Trials

0 0

5000

10000

15000

As the number of trials increase closure on the correct answer occurs 6

Sensitivity Analysis Plots May be Generated Utilizing Converged MOE/MOPs versus KPP • Measures of Effectiveness (MOE) and Measures of Performance (MOP) are the quantities by which we decide a system meets its requirements – Percent block time – Total transmission time • Key Performance Parameters (KPP) are the quantifiable factors we identify as best able to influence achieving the MOEs & MOPs – Probability or reaction time We must use quantifiable KPPs to ensure the Simulation will properly model the System 7

More Simulation Statistics For The Curious â€˘ For more details about Simulation statistics, read textbook chapters: 4 for Markov Modeling, 5 for Reliability, and 6 for Queuing Theory

8

Module 3: Logic and Statistics Concepts for Simulations • Statistical use of OpEMCSS Blocks for Systems Engineering and Simulation • Case Study & OpEMCSS Blocks: Producer-Consumer • Summary

9

Producer-Consumer Process Shown in a Factory Scenario PRODUCER

Work Station No. 1

DOOR 1

CONVEYOR BELT

DOOR 2

CONSUMER Work Station No. 2

Producer-Consumer process shown in a factory scenario provides an example of a parallel process simulation where the process interact. 10

Producer-Consumer: Prioritizing a Single Manufacturing Line

Produce the calculator

Package the calculator

11

Event Occurrence and Alternate Action Blocks

Event Occurrence

Alternate Action

â€˘ Event Occurrence lets us merge multiple inputs together, also modifying simulation attributes â€˘ Alternate Action lets us select one of three alternate transition paths per a logic equation 12

Wait Until Event Blocks Control Assembler’s Access to Conveyer Belt

• Assembler makes a calculator and wants to place it on the conveyer belt • In order to do this: – first a space on the belt must be available (attribute NE > 0) – second the conveyer belt must be available (attribute B > 0)

13

Wait Until Event Blocks Control Packer’s Access to Conveyer Belt

• Packer wants to obtain calculator from belt and pack it • In order to do this: – first a calculator must be available on belt (attribute NQ > 0) – second the conveyer belt must be available (attribute B > 0) 14

Experiment • Decrease Assembler time from 60 to 40 seconds and run model – Observe scoreboard

• Decrease Packer time from 90 to 60 seconds and run model – Observe scoreboard

• Decrease Belt time from 10 to 5 seconds and run model – Observe scoreboard 15

Sensitivity Curve: Throughput vs Production Reaction Time 0.7

Throughput

0.6 0.5

Greatest Improvement Gain

0.4 0.3 0.2 0.1 0 0

10

20

30

40

50

60

70

80

90

100

Production Time

â€˘

Sensitivity curve is a plot of a converged Measure of Effectiveness (MOE) values versus a Key Performance Parameter (KPP) values 16

Module 3: Logic and Statistics Concepts for Simulations • Statistical use of OpEMCSS Blocks for Systems Engineering and Simulation • Case Study & OpEMCSS Blocks: Producer-Consumer • Summary

17

Summary of Module 3 • Learned Statistical use of OpEMCSS Blocks for Systems Engineering and Simulation • Studied case of Producer-Consumer Manufacturing • Each person introduced to the OpEMCSS modeling methodology A simulation must have a firm logical and statistical basis to be valid AND useful for Systems Engineering. 18