Project Title: Pseudo-Random Number Generator Name: Sheetal Murkute MS, Industrial Engineering

Email:sxm4946@rit.edu RIT ID: 68400-5611

Abstract: A method of generating random numbers is necessary for the simulation of any system with components that are random. Hence random number generators are one of the most important parts of any stochastic simulation. Some common techniques for doing this are the use of dice, coin flipping, shuffling of cards, etc. However computer simulation of a real system requires very long range random numbers and the simple techniques mentioned above are inadequate for this task. Natural random phenomena such as quantum physics can produce truly random number generators. For computer simulations, random number generators which are as close to these true random numbers are desired. A pseudo random number generator for high speed simulation should have several properties: (1) Its period should be massive, (2) It should exhibit uniform distribution between 0 and 1. (3) It should exhibit a good structure (usually a lattice structure) in high dimensions, and (4) It should be eďŹƒciently computable (preferably with a base b which is a power of 2). (5) It should produce the previously generated random number stream exactly

Typically the RNGs belong to a family of comparable generators with diďŹ€erent parameters. It is desired that parameters and seeds can be easily chosen so as to guarantee properties (1), (2), (3), (4) and (5). Very few RNGs satisfy all of these properties. Specifically parameter selection plays an important role in generating random numbers. This paper analyzes and summarizes the study of the role of these parameters in developing a random number generator using simple algebraic techniques.

Introduction: Random number generators lie at the heart of stochastic simulation. Generating random numbers should consider abstract algebra and number theory on one hand and systems programming and computer hardware on the other hand. Linear congruential Generators are the most widely used and best understood generators. Other types are Quadratic congruential generators, composite generators, generators using mid-square methods etc. The random character of the numbers generated by these methods can be tested using various empirical and theoretical tests. Typically all generators exhibit some strengths and shortcomings.

Techniques for random number generation, performance aspects of various techniques that they aim to optimize and relative comparison to evaluate the merits of approaches: For testing and debugging of various systems, the pseudorandom generators should produce the previously generated random number stream exactly. Also the methods like reading in table like Rand Corporation table involve large memory requirements. Hence, numerical arithmetic ways which use fixed mathematical formula are being used. This section briefly explains some of the common techniques used for generating random numbers.

Mid-square method: The first generator used generates random numbers using mid-square method. It starts with a four digit number. Append zeros to left of its square to make 8 digit number. From this number, middle 4 digits are considered and decimal point is placed to left of it to get a number between 0 and 1. Example: Initial Seed Z0 = 7182 ; (Z0)^2 = 51581124 ; U1 = 0.5811 Z1 = 5811 ; (Z1)^2 = 36767721 ; U2 = 0.7677 Z2 = 7677 ; (Z2)^2 = 58936329 ; U3 = 0.9363 So on………….. Advantages: This generator intuitively provides good scrambling of 1 number to obtain the next. Disadvantages: In statistical analysis, it does not work well at all. This shows a strong tendency to degenerate fairly rapidly to zero, where it stays for long. It is dangerous to assume that good pseudo-random generator is obtained by doing something strange to obtain 1 number from the next. In addition to the basic properties stated in abstract section, there are some more properties that a good random number generator should possess. Properties of a good random number generator: 1. The numbers generated should be distributed uniformly, and should not exhibit correlation with each other; otherwise simulation results will be invalid. 2. It should generate the numbers fast and also avoid need of lot of storage. 3. It should reproduce stream of random numbers again, because, a) This makes debugging or verification of computer program easier b) One might want to use identical random numbers in simulating different systems for more precise comparison.

4. It should produce separate streams of random numbers. User can dedicate particular stream to particular stream of randomness. For example inter-arrival times, service times etc. Modern simulation packages satisfy this condition as well. Many generators fail to produce uniformly distributed and independent numbers. Hence they become statistically unacceptable.

Linear Congruential Generators: In this type of generators, random numbers are generated using recursive formula: Zi = (a*Z i-1 + c) (mod m) In addition to non-negativity, it should also satisfy conditions 0 <m, a < m, c < m, Z0 < m. Advantages: 1. This generator has simple and efficient practical implementation. 2. Good parameter selection can lead to have a good LCG. 3. Reproducibility is simple in LCG. We must only remember initial seed. And one can initiate the generator with this value to get same stream. 4. One can use previously obtained number as a seed, to obtain non-overlapping, independent sequences. Disadvantages: 1. The number generates are not truly random and predicted very easily by mathematical formula. 2. As a, Z, c are integers, (a*Z i-1 + c) is an integer value. Hence, Zi can take only the values 0 , 1/m, 2/m, 3/m â€Ś. (m-1)/m. Values that lie in between are not obtained and so the nature is not uniform. 3. Values depend on parameters and the floating point method used. 4. There is no possibility of getting values between 0.1/m â€Ś. 0.9/m, hence it is not uniform. Values of m chosen large so that points between 0, 1 where Ui can fall is very dense, and thus appear uniform. 5. Restriction of Ui to the discrete values could result in inaccuracy in simulation models. Importance of full period: Large simulation projects can use many random numbers, hence it is very essential to have LCGs with long period. Having full period also contributes uniformity. The period can be maximum (m-1). Hence values from 0 to m-1 appear exactly once in cycle. These contribute uniformity of the random numbers. But the numbers generated may be non-uniform within the cycle. For example, if the period is m/2, then these m/2 consecutive Zi , leave large gap. Hence parameter selection is very important to have full period. Conditions for parameter selection of LCG:[1] 1. Parameters c and m should be relatively prime. 2. If q is a prime number that divides m, then q should divide a-1. 3. If 4 divides m, then 4 divides a-1.

Such parameter selection improves the properties of LCG to a great extent. Parameters also determine other properties as well such as good statistical properties like apparent independence, computational and storage efficiency, reproducibility and facility for separate streams. To get higher efficiency for random number generation, other generators are developed with some modifications in the basic structure of LCG.

Mixed generators: These are LCGs when c>0. LCG are desirable to have full period. Large period will also lead to high density but this needs large values of m. Divide by m operation in LCGs to obtain remainder is arithmetically slow operation. Hence for implementation of random number generators in computers, m is selected as 2^b. For 32bit comp, m = 2^31. Selecting values of m as 2^b, computers take advantage of integer overflow to avoid explicit division. Caution that should be considered is that the sign bit might be turned on. This illustrates the choice of parameter m. But choice of ‘c’ and ‘a’ is also critical. ‘c’ is selected an odd number. ‘a-1’ should be divisible by 4. a = z^l + 1 is preferred. This gives, z = 2^l zi-1+ zi-1 Thus random numbers can be obtained by shifting operation instead on explicit division. Shift and add operation can be used. Disadvantages: These generators give poor stat prop. Hence use of mixed generators should be avoided.

Multiplicative generators: When LCGs are formed with c=0, the generators obtained are called as multiplicative generators. Advantages: These generators are well studied in literature and easier to implement. Carry value ‘C’ is not used in these generators which makes the structure simpler and implementation easier. Disadvantages: 1. One does not usually obtain a full period for multiplicative random number generators. 2. But a period of m-1 can be obtained by selecting the value of ‘a’ very carefully. Most LCG are multiplicative as improvement expected in mixed generators is not demonstrated. Parameter Selection: Various research work on these type of generators has proved the efficient selection of parameters. As stated earlier, m is selected as 2^b for the efficient use in computers. But the period obtained for this selection is 2^b-2 which 1/4th of the full period. This may lead to the other issues such as non-uniformity as it cannot be predicted where these m/4 integers will fall which might leave unacceptably large gap. It has been studied that even a period of 2^b-2 is obtained only if z odd and a is selected as (8k+3) or (8k+5). Hence this is not efficient.

Hence other ways of specifying the values of ‘m’ are needed to be studied.

PMMLCG: It has been studied that instead of selecting m as 2^b, if m is selected as largest prime number less than 2^b , which can be [(2^31) – 1] for computers, it gives a period of m-1 if a is primitive element modulo m [ That is the smallest integer ‘l’ for which al – 1 is divisible by m is m-1. These generators are called as Prime Modulus Multiplicative LCGs (PMMLCGs).[1] Similarly many techniques are being studied to improve the performance of LCGs. These techniques include efficient parameters selection, use of shift registers, Use of seed which is also generated randomly by random number generator etc. The generator which has been proved most efficient and which is being widely used is Combined Multiple Recursive Generator (CMRG).

Combined Multiple Recursive Generators (CMRG): This generator generates two sequences of generators. A1, A2….An and B1, B2, …… defined by An = (1403580 * An-2 - 810728 * An-3) mod 4294967087 Bn = (527612 * Bn-1- 1370589 * Bn-3) mod 4294944443 Zn = (An – Bn) mod 4294967088 Un = Zn / 4294967088 4294967088/4294967088

if Zn ≠ 0 if Zn = 0

Advantages: 1. CMRG has very good statistical properties. 2. The period of CMRG is 3.1* 10^57 when parameters are selected correctly. Disadvantages: 1. The generation of two sequences increases the processing time. 2. For larger application, this results into slower simulation. Though, it’s widely used because of its good statistical properties.

Multiply with Carry (MWC) technique for generating random numbers:[4] Multiply with carry method for random number generation which was proposed by G. Marsaglia. The method uses simple computer integer arithmetic and enables speedy generation of sequences of random numbers with huge periods. This technique is similar to linear congruential generators. ax ୬ିଵ + C୬ − 1 x୬ = (ax୬ିଵ + C୬ିଵ ) mod b, C୬ = Floor ൨ b Multiply with carry is similar to LCG in technique with the difference that instead of using C as a constant carry, it is updated for every calculation. Usually b = 232, because arithmetic modulo of power of 2 is

automatic in most computers, but sometimes a base such as b = 232 − 1 is used, for MWC sequences based on modulus 232 has some poor properties that use of b = 232 − 1 avoids. For improving statistical properties, lag lag-r MWC can be used instead of lag-1 1 MWC as described above. lag-r MWC generator requires a base b, a multiplier a, and a set of r+1 random seed values, consisting of r residues of b, x0, x1, x2 ,..., xr−1, and an initial carry cr−1 < a. The lag-r MWC sequence is then a sequence of pairs xn, cn determined by (ax C mod b, C Floor

ax C 1 ,n b

r

and the MWC generator output is the sequence of x's, xr , xr+1 , xr+2, ... Because b = 232 cannot be a primitive root of p = abr − 1, there are no MWC generators for base 232 that have the maximum possible period, one of the difficulties that use of b = 232 − 1 overcomes.

Complementary-multiply multiply-with-carry generators (CMWC): There is a slight change in the generation of a new pair

With b=232, the period of the lag-1024 lag CMWC

With b = 232 and a = 3636507990, p = ab1359 − 1 is a safe prime, so the MWC sequence based on that a has period 3636507990 243487

1013101 [4].

Comparisons with linear congruential generators and Combined Multiple Recursive Generators (CMRG): (CMRG) Linear congruential generators are implemented as x ax c mod 2 Computers make use of shift and add operations instead of explicit multiplications and an divisions for carrying out these calculations.

Computational Advantages: Processors rocessors are able to put the multiplier a and the current x in 32-bit bit registers, form the 64-bit 64 product in n adjoining registers, and take the lower 32 bits as the product, that is, form a ∗ x mod 2 Adding the 32-bit c to that lower half then provides (ax+c) mod 232. Example: If a mod 8 is 1 or 5 and c is odd, the resulting base 232 congruential sequence will have period 232. Computational advantage of multiply ultiply with carry carry: A lag-1 multiply with carry generator allows us to make the period nearly 263 by using those same computer operations, except that this time the top half of the 64 64-bit bit product is used rather than ignored after the 64 bits are formed. It's used as a new carry value c rather than the fixed carry value of the standard congruential sequence: Get ax+c in 64-bits, then form a new c as the top half of those 64 bits, and the new x as the bottom half. With multiplier a specified, each pair of input values x, c is converted nverted to a new pair,

Full Period of MWC in comparison with LCG and CMRG: For b = 232, with appropriate parameter selection, full period of LCG = 232 – 1. Maximum period for For b = 232, and a = 3636507990, p = ab1359 − 1 is a safe prime, so the complementary MWC sequence based on that a has period 3636507990 243487

1013101 [5].

For CMRG, full period is in the range of 1050

Advantages of MWC: 1. The main advantages of the MWC method are that it invokes simple computer integer arithmetic and leads to very fast generation of sequences of random numbers with immense periods, ranging from around 260 to 22000000.[4],[5] [4],[5]

Disadvantages of MWC: 1. Consecutive d-tuples tuples (xk, xk+1, . . . , xk+d−1) xk+d−1) of numbers generated by the MWC generator do not form a d-dimensional dimensional lattice.

2. A theoretical problem with MWC generators, pointed out by Couture and l'Ecuyer (1997) is that the most significant bits are slightly biased; complementary-multiply-with-carry generators do not share this problem. 3. Complementary-multiply-with-carry generators also require slightly more computation time per iteration, so there is a tradeoff to evaluate depending on implementation requirements.

IMPLEMENTATION Software code for Multiply With Carry random number generation and implementing it in Arena simulation: Linear Congruential Generators (MATLAB Implementation): While (value of x1 reoccurs) { Xn = (A*Xn-1 + C) mod M Un= Xn/M } Display U Display period = n at the end of program End Example with Full Period: A = 3xn-1 mod 31 Here , m= 25. Period = 30 = (m-1)

Multiply With Carry Generators (MATLAB Implementation): While (value of x1 reoccurs) { Xn = (A*Xn-1 + Cn-1) mod M Un= Xn/M Cn = Floor [ (A*Xn-1 + Cn-1) / M] } Display U Display period = n at the end of program End

Combined Multiple Recursive Generators (CMRG): function cmrg(A,B) n = 4; done = 0; while done == 0 A(n) = mod(1403580*A(n-2) - 810728*A(n-3),4294967087); B(n) = mod(527612*B(n-1) - 1370589*B(n-3),4294944443); Z(n) = mod(A(n) - B(n),4294967088); if Z(n) == 0 U(n) = 4294967088/4294967088; else U(n) = Z(n) / 4294967088; end; if n > 4 if n > 8000 || U(n) == U(4) done = 1; end; end; n = n + 1; end; disp(['Period: ',num2str(n)]); xlswrite('cmrgsequence.xls',U');

Testing of random number generators: Statistical Tests: Statistical tests are carried out with 2^16 modulo using MATLAB implementation. The scatter plots of the results are plotted to study the statistical properties. Approximately 8000 numbers are used for scatter plots.

Linear Congruential Generators:

1.2

1

0.8

0.6 Series1 0.4

0.2

0 0

2000

4000

6000

8000

10000

The output of LCG is not completely random. This shows linear patterns. Thus these generators do not show good statistical properties.

Multiply With Carry: 1.2 1 0.8 0.6 Series1 0.4 0.2 0 0

2000

4000

6000

8000

10000

Statistical properties for Multiply with carry generators show no trends. The numbers are uniformly distributed as compared to LCG. Combined Multiple Recursive Generators (CMRG):

1.2 1 0.8 0.6 Series1 0.4 0.2 0 0

2000

4000

6000

8000

10000

CMRG also show good statistical properties as the data does not show any trend.

Conclusions: The study of various techniques for random number generation indicates various advantages and disadvantages of the methods studied. MWC shows long periods, but there are some disadvantages of this method. Statistical tests clearly indicate the differences between properties of various methods. MWC shows long periods, but there are some disadvantages of this method. A theoretical problem with MWC generators is that the most significant bits are slightly biased; complementary-multiply-withcarry generators do not share this problem. Complementary-multiply-with-carry generators also require slightly more computation time per iteration, so there is a tradeoff to evaluate depending on implementation requirements. Arena Implementation indicates implementation methodology to use the random numbers generators to create random arrival times. For this, random numbers are generated using the mathematical methods described above using MATLAB implementation. All the techniques are designed for achieving properties of random numbers, for example, good statistical tests, long periods, independent sequences and ease of implementation in computer systems. Natural sources of random numbers like optical sources show true randomness. For computer implementation, mathematical random generator should possess properties that resemble properties of a true random number generator. Study has been continuously being done to achieve a better randomness.

REFERENCES 1. 2. 3. 4. 5. 6. 7. 8.

Law A. M. , Kelton W. D., ‘Simulation Modeling And Analysis’, Third Edition. Kelton W. D., Sadowski R. P, Swets N. B, ‘Simulation with Arena’, Fifth Edition George Marsaglia, On the Randomness of Pi and Other Decimal Expansions GORESKY M., KLAPPER A., ‘Eﬃcient multiply-with-carry random number generators with maximal period’, University of Kentucky Multiply-With-Carry (Random Number Generator) (2008), From Wikipedia, http://en.wikipedia.org/wiki/Multiply_with_Carry Jengnan Tzeng ; I-Te Chen ; Jer-Min Tsai (2009) Random Number Generator designed by the divergence of scaling functions ; Nat. Chengchi Univ., Taipei, Taiwan; IEEE Xplore Shen Hai-Wei ; Li Jin-Ping ;(31 December 2009) A High-Speed and Long-Period Combined Pseudo-random Number Generator; Inf. Inst., Beijing Union Univ., Beijing, China; IEEE Xplore Wayne, M.A.; Jeffrey, E.R.; Akelsrod, G.M.; Kwiat, P.G.; (2008) High-speed quantum random number generation, IEEE Xplore