Probability and Bayesian Modeling 1st edition Jim Albert, Jingchen Hu (Instructor Solution Manual

SOLUTION MANUAL

Solutions to Probability and Bayesian Modeling Jim Albert and Monika Hu February 2020

Contents Chapter 1 - Probability: A Measure of Uncertainty

1

Chapter 2 - Counting Methods

12

Chapter 3 - Conditional Probability

20

Chapter 4 - Discrete Distributions

35

Chapter 5 - Continuous Distributions

60

Chapter 6 - Joint Probability Distributions

80

Chapter 7: Learning About a Binomial Probability

96

Chapter 8: Modeling Measurement and Count Data

136

Chapter 9: Simulation by Markov Chain Monte Carlo

158

Chapter 10: Bayesian Hierarchical Modeling

184

Chapter 11: Simple Linear Regression

235

Chapter 12 Bayesian Multiple Regression and Logistic Models

267

Chapter 13 Case Studies

298

Chapter 1 - Probability: A Measure of Uncertainty Exercise 1. Probability Viewpoints In the following problems, indicate if the given probability is found using the classical viewpoint, the frequency viewpoint, or the subjective viewpoint. a. Joe is doing well in school this semester { he is 90 percent sure that he will receive an A in all of his classes. subjective b. Two hundred raffle tickets are sold and one ticket is a winner. Someone purchased one ticket and the probability that her ticket is the winner is 1/200. classical

1

c. Suppose that 30% of all college women are playing an intercollegiate sport. If we contact one college woman at random, the chance that she plays a sport is 0.3. frequency d. Two Polish statisticians in 2002 were questioning if the newBelgium Euro coin was indeed fair. They had their students flip the Belgium Euro 250 times, and 140 came up heads. frequency e. Many people are afraid of flying. But over the decade 1987-96, the death risk per flight on a US domestic jet has been 1 in 7 million. frequency f. In a roulette wheel, there are 38 slots numbered 0, 00, 1, . . . , 36. There are 18 ways of spinning an odd number, so the probability of spinning an odd is 18/38. classical

Exercise 2. Probability Viewpoints In the following problems, indicate if the given probability is found using the classical viewpoint, the frequency viewpoint, or the subjective viewpoint. a. The probability that the spinner lands in the region A is 1/4. classical b. The meteorologist states that the probability of rain tomorrow is 0.5. You think it is more likely to rain and you think the chance of rain is 3/4. subjective c. A football fan is 100% certain that his high school football team will win their game on Friday. subjective d. Jennifer attends a party, where a prize is given to the person holding a raffle ticket with a specific number. If there are eight people at the party, the chance that Jennifer wins the prize is 1/8. classical e. What is the chance that you will pass an English class? You learn that the professor passes 70% of the students and you think you are typical in ability among those attending the class. frequency f. If you toss a plastic cup in the air, what is the probability that it lands with the open side up? You toss the cup 50 times and it lands open side up 32 times, so you approximate the probability by 32/50 frequency

Exercise 3. Equally Likely Outcomes For the following experiments, a list of possible outcomes is given. Decide if one can assume that the outcomes are equally likely. If the equally likely assumption is not appropriate, explain which outcomes are more likely than others. a. A bowl contains six marbles of which two are red, three are white, and one is black. One marble is selected at random from the bowl and the color is observed.

2

Outcomes: {red, white, black} not equally likely – white are more likely to be chosen b. You observe the gender of a baby born today at your local hospital.

Outcomes: {male, female} equally likely c. Your school’s football team is playing the top rated school in the country.

Outcomes: {your team wins, your team loses} not equally likely – your team is more likely to lose d. A bag contains 50 slips of paper, 10 that are labeled “1”, 10 labeled “2”, 10 labeled “3”, 10 labeled “4”, and 10 labeled “5”. You choose a slip at random from the bag and notice the number on the slip.

Outcomes: {1, 2, 3, 4, 5} equally likely

Exercise 4. Equally Likely Outcomes For the following experiments, a list of possible outcomes is given. Decide if one can assume that the outcomes are equally likely. If the equally likely assumption is not appropriate, explain which outcomes are more likely than others. a. You wait at a bus stop for a bus. From experience, you know that you wait, on average, 8 minutes for this bus to arrive.

Outcomes: {wait less than 10 minutes, wait more than 10 minutes} not equally likely – more likely to wait less than 10 minutes b. You roll two dice and observe the sum of the numbers.

Outcomes: {2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12} not equally likely – some sums (7) are more likely to come up c. You get a grade for an English course in college.

Outcomes: {A, B, C, D, F} not equally likely — relatively unlikely for D or F d. You interview a person at random at your college and ask for his or her age.

Outcomes: {17 to 20 years, 21 to 25 years, over 25 years} not equally likely – students are relatively unlikely to be over 25 years

Exercise 5. Flipping a Coin Suppose you flip a fair coin until you observe heads. You repeat this experiment many times, keeping track of the number of flips it takes to observe heads. Here are the numbers of flips for 30 experiments.

131211261211113211215217333123 a. Approximate the probability that it takes you exactly two flips to observe heads.

P(2 flips) = 7/30

3

b. Approximate the probability that it takes more than two flips to observe heads.

P(more than 2 flips) = 9/30 c. What is the most likely number of flips? most likely number of flips is 1

Exercise 6. Driving to Work You drive to work 20 days, keeping track of the commuting time (in minutes) for each trip. Here are the twenty measurements.

25.4, 27.8, 26.8, 24.1, 24.5, 23.0, 27.5, 24.3, 28.4, 29.0 29.4, 24.9, 26.3, 23.5, 28.3, 27.8, 29.4, 25.7, 24.3, 24.2 a. Approximate the probability that it takes you under 25 minutes to drive to work.

8 days you took less than 25 minutes, so probability would be approximately 8/20 b. Approximate the probability it takes between 25 and 28 minutes to drive to work.

7/20 c. Suppose one day it takes you 23 minutes to get to work. Would you consider this unusual? Why? I would consider 23 minutes unusual since probability of 23 minutes or smaller is 1/20.

Exercise 9. Frequency of Vowels in Huckleberry Finn Suppose you choose a page at random from the book Huckleberry Finn by Mark Twain and find the first vowel on the page. a. If you believe it is equally likely to find any one of the five possible vowels, fill in the probabilities of the vowels below.

Vowel

a

e

i

o

u

Probability

1/5

1/5

1/5

1/5

1/5

b. Based on your knowledge about the relative use of the different vowels, assign probabilities to the vowels.

Vowel

a

e

i

o

u

Probability

(You should give certain vowels like e high probabilities.) c. Do you think it is appropriate to apply the classical viewpoint to probability in this example? (Compare your answers to parts a and b.

No, it would not be appropriate to apply theclassical viewpoint of probability here. d. On each of the first fifty pages of Huckleberry Finn, your author found the first five vowels. Here is a table of frequencies of the five vowels: Vowel

a

e

i

o

u

Frequency

61

63

34

70

22

4

Use this data to find approximate probabilities for the vowels.

Vowel

a

e

i

o

u

Frequency

61/250

63/250

34/250

70/250

22/250

Exercise 10. Purchasing Boxes of Cereal Suppose a cereal box contains one of four different posters denoted A, B, C, and D. You purchase four boxes of cereal and you count the number of posters (among A, B, C, D) that you do not have. The possible number of “missing posters” is 0, 1, 2, and 3. a. Assign probabilities if you believe the outcomes are equally likely.

Number of missing posters

0

1

2

3

Probability

1/4

1/4

1/4

1/4

b. Assign probabilities if you believe that the outcomes 0 and 1 are most likely to happen.

Number of missing posters

0

1

2

3

Probability

(Assign the values 0 and 1 high probabilities.) c. Suppose you purchase many groups of four cereals, and for each purchase, you record the number of missing posters. The number of missing posters for 20 purchases is displayed below. For example, in the first purchase, you had 1 missing poster, in the second purchase, you also had 1 missing poster, and so on.

1, 1, 1, 2, 1, 1, 0, 0, 2, 1, 2, 1, 3, 1, 2, 1, 0, 1, 1, 1 Using these data, assign probabilities.

Number of missing posters

0

1

2

3

Probability

3/20

12/20

4/20

1/20

d. Based on your work in part c, is it reasonable to assume that the four outcomes are equally likely?

Why?

Exercise 11. Writing Sample Spaces For the following random experiments, give an appropriate sample space for the random experiment. You can use any method (a list, a tree diagram, a two-way table) to represent the possible outcomes. a. You simultaneously toss a coin and roll a die. one sample space would be H1, H2, H3, H4, H5, H6, T1, T2, T3, T4, T5, T6 b. Construct a word from the five letters a, a, e, e, s. words are arrangements of letters like aaees, aeaes, aeeas, aeesa, . . .

5

c. Suppose a person lives at point 0 and each second she randomly takes a step to the right or a step to the left. You observe the person’s location after four steps. location after 4 steps could be {-4, -2, 0, 2, 4} d. In the first round of next year’s baseball playoff, the two teams say the Phillies and the Diamondbacks play in a best-of-five series where the first team to win three games wins the playoff.

If P denotes event that Phillies win a game and D denotes event that Diamondbacks win, then possible outcomes are PPP, DDD, DPPP, DDPPP, etc. e. A couple decides to have children until a boy is born.

You could record the total number of children – outcomes are 1, 2, 3, . . . f. A roulette game is played with a wheel with 38 slots numbered 0, 00, 1, . . . , 36. Suppose you place a $10 bet that an even number (not 0) will come up in the wheel. The wheel is spun.

Outcome could be the result of your bet: you win or you lose. g. Suppose three batters, Marlon, Jimmy, and Bobby, come to bat during one inning of a baseball game.

Each batter can either get a hit, walk, or get out.

One outcome would be HWH (Marlon gets a hit, Jimmy walks, Bobby gets a hit) or WHH (Marlon gets a walk, Jimmy walks, Bobby gets a hit), etc. Other outcomes are HHH, WWW, OWH, etc.

Exercise 12. Writing Sample Spaces For the following random experiments, give an appropriate sample space for the random experiment. You can use any method (a list, a tree diagram, a two-way table) to represent the possible outcomes. a. You toss three coins. one possible sample space is HHH, HHT, HTH, etc where you are recording the result of each flip b. You spin the spinner (shown below) three times. if you record location of each spin, outcomes are AAA, AAB, ABC, . . . c. When you are buying a car, you have a choice of three colors, two different engine sizes, and whether or not to have a CD player. You make each choice completely at random and go to the dealership to pick up your new car. one outcome would be first color, engine size 1, and no cd player. There are 2 x 2 x 2 = 8 possible outcomes d. Five horses, Lucky, Best Girl, Stripes, Solid, and Jokester compete in a race. You record the horses that win, place, and show (finish first, second, and third) in the race. one outcome would be the horses finishing in order Lucky, Best Girl, Stripes (Lucky is first). There are 5 x 4 x 3 = 60 possible outcomes. e. You and a friend each think of a whole number between 0 and 9. one outcome would be person 1 chooses 2 and person 2 chooses 3 or (23). There are 100 possible outcomes. f. On your computer, you have a playlist of 4 songs denoted by a, b, c, d. You play them in a random order. outcome would be arrangement of songs like abcd or acbd, etc. There are 24 possible outcomes. g. Suppose a basketball player takes a “one-and-one” foul shot. (This means that he attempts one shot and if the first shot is successful, he gets to attempt a second shot.) could record the number of shots made: outcomes would be 0, 1, 2

6

Exercise 13. Writing Sample Spaces For the following random experiments, give an appropriate sample space for the random experiment. You can use any method (a list, a tree diagram, a two-way table) to represent the possible outcomes. a. Your school plays four football games in a month. your team can win (W) or lose (L) each game. Outcome could be number of games won (0, 1, 2, 3, 4). Or outcome could be result of each game, like WLWW – then there are 16 possible outcomes. b. You call a “random” household in your city and record the number of hours that the TV was on that day. number of hours of TV watched could be 0, 1, 2, 3, Ö, 24 (assuming answer is recorded to nearest hour) c. You talk to an Ohio resident who has recently received her college degree. How many years did she go to college? number of years attending college = 3, 4, 5, 6, . . . , 10 d. The political party of our next elected U.S. President. political party: republican, democratic, independent e. The age of our next President when he/she is inaugurated. age of next President: 35, 36, 37, . . . , 75 (or some other reasonable upper limit) f. The year a human will next land on the moon. year next human lands on moon: 2020, 2021, . . .

Exercise 14. Writing Sample Spaces For the following random experiments, give an appropriate sample space for the random experiment. You can use any method (a list, a tree diagram, a two-way table) to represent the possible outcomes. a. The time you arrive at your first class on Monday that begins at 8:30 AM. time arriving to class: 8:20, 8:21, . . . , 8:45 b. You throw a ball in the air and record how high it is thrown (in feet). height of ball in feet: continuous value between 2 and 12 feet c. Your cost of textbooks next semester. cost of textbooks: 0, $1, . . . , $300 d. The number of children you will have. number of children: 0, 1, 2, 3,. . . , 20 e. You take a five question true/false test. number of questions correct: 0, 1, 2, 3, 4, 5 f. You drive on the major street in your town and pass through four traffic lights. could record the number of lights where you hit a red light; outcomes 0, 1, 2, 3, 4

7

Exercise 15. Probability Assignments Give reasonable assignments of probabilities based on the given information. a. In the United States, there were 4058 thousand babies born in the year 2000 and 1980 thousand were girls. Assign probabilities to the possible genders of your next child.

Gender

Boy

Girl

Probability

2078/4056

1980/4056

b. Next year, your school will be playing your neighboring school in football. Your neighboring school is a strong favorite to win the game.

Winner of Game

Your school

Your neighboring school

Probability

small

large

c. You have an unusual die that shows 1 on two sides, 2 on two sides, and 3 and 4 on the remaining two sides.

Roll

1

2

3

4

5

6

Probability

2/6

2/6

1/6

1/6

0

0

Exercise 16. Probability Assignments Based on the given information, decide if the stated probabilities are reasonable. If they are not, explain how they should be changed. a. Suppose you play two games of chess with a chess master. You can either win 0 games, 1 game, or 2 games, so the probability of each outcome is equal to 1/3.

This is not reasonable – not likely that you’d win 2 games against a chess master, so “2 games” should get a small probability. b. Suppose 10% of cars in a car show are Corvettes and you know that red is the most popular Corvette color. So the chance that a randomly chosen car is a red Corvette must be larger than 10 %.

This is not reasonable. P(red Corvette) can’t be larger than P(Corvette) since “red Corvette” is a subset of “Corvette”. c. In a Florida community, you are told that 30% of the residents play golf, 20% play tennis, and 40% of the residents play golf and tennis.

These are reasonable probabilities. d. Suppose you are told that 10% of the students in a particular class get A, 20% get B, 20% get C, and 20% get D. That means that 30% of the class must fail the class.

These are reasonable probability statements.

Exercise 17. Finding the Right Key Suppose your key chain has five keys, one of which will open up your front door of your apartment. One night, you randomly try keys until the right one is found.

8

Here are the possible numbers of keys you will try until you get the right one: 1 key, 2 keys, 3 keys, 4 keys, 5 keys (a) Circle the outcome that you think is most likely to occur.

1 key, 2 keys, 3 keys, 4 keys, 5 keys (b) Circle the outcome that you think is least likely to occur.

1 key, 2 keys, 3 keys, 4 keys, 5 keys (c) Based on your answers to parts a and b, assign probabilities to the six possible outcomes.

The answers here will depend on the choices for (a) and (b).

Exercise 18. Playing Roulette One night in Reno, you play roulette five times. Each game you bet $5 – if you win, you win $10; otherwise, you lose your $5. You start the evening with $25. Here are the possible amounts of money you will have after playing the five games. $0, $10, $20, $30, $40, $50 . (a) Circle the outcome that you think is most likely to occur. $0, $10, $20, $30, $40, $50 . (b) Circle the outcome that you think is least likely to occur. $0, $10, $20, $30, $40, $50 . (c) Based on your answers to parts a and b, assign probabilities to the six possible outcomes.

The answers here will depend on the choices for (a) and (b).

Exercise 19. Cost of Your Next Car Consider the cost of the next new car you will purchase in the future. There are five possibilities: • cheapest: the car will cost less than $5000 • cheaper: the car will cost between $5000 and $10,000.

• moderate: the car will cost between $10,000 and $20,000 • expensive: the car will cost between $20,000 and $30,000 • really expensive: the car will cost over $30,0000 (a) Circle the outcome that you think is most likely to occur. cheapest, cheaper, moderate, expensive, really expensive (b) Circle the outcome that you think is least likely to occur. cheapest, cheaper, moderate, expensive, really expensive (c) Based on your answers to parts a and b, assign probabilities to the five possible outcomes.

The answers here will depend on the choices for (a) and (b).

9

Exercise 20. Flipping a Coin Suppose you flip a coin twice. There are four possible outcomes (H stands for heads and T stands for tails).

HH, HT, T H, T T (a) Circle the outcome that you think is most likely to occur.

HH, HT, T H, T T (b) Circle the outcome that you think is least likely to occur.

HH, HT, T H, T T (c) Based on your answers to parts a and b, assign probabilities to the four possible outcomes.

The answers here will depend on the choices for (a) and (b).

Exercise 21. Playing Songs in Your iPod Suppose you play three songs by Jewell (J), Madonna (M), and Plumb (P) in a random order. (a) Write down all possible ordering of the three songs.

JM P, JP M, M JP, M P J, P JM, P M J (b) Let M = event that the Madonna song is played first and B = event that the Madonna song is played before the Jewell song. Find P (M ) and P (B).

P (M ) = 2 / 6, P (B)= 3 / 6 (c) Write down the outcomes in the event M ∩ B and find the probability P (M ∩ B).

M ∩ B = {M JP, M P J}, P (M ∩ B) = 2/6 (d) By use of the complement property, find P (B̄).

P (B̄) = 1 − P (B) = 1 − 3/6 (e) By use of the addition property, find P (M ∪ B).

P (M ∪ B) = P (M ) + P (B) − P (M ∩ B) = 2/6 + 3/6 − 2/6 = 3/6

Exercise 22. Student of the Day Suppose that students at a local high school are distributed by grade level and gender.

Freshmen

Sophomores

Juniors

Seniors

TOTAL

25 20 45

30 32 62

24 28 52

19 15 34

98 95 193

Male Female TOTAL

Table of grade level and gender.

Suppose that a student is chosen at random from the school to be the “student of the day”. Let F = event that student is a freshmen, J = event that student is a junior, and M = event that student is a male. (a) Find the probability P (F̄ ).

88 / 193

10

(b) Are events F and J mutually exclusive. Why? Yes, a student can’t be both a freshman and a junior. (c) Find P (F ∪ J) .

45 / 193 + 52 / 193 = 97 / 193 (d) Find P (F ∩ M ).

25 / 193 (e) Find P (F ∪ M ).

45 / 193 + 98 / 193 - 25 / 193 = 118 / 193

Exercise 23. Proving Properties of Probabilities Given the three probability axioms and the properties already proved, prove the complement property P (Ā) = 1 − P (A). An outline of the proof is written below. (a) Write the sample space S as the union of the sets A and Ā.

By properties of sets, S = A ∪ Ā (b) Apply Axiom 3.

By Axiom 3, since A and Ā are mutually exclusive, P (S) = P (A) + P (Ā) (c) Apply Axiom 2.

By Axiom 2, P (S) = 1, so 1 = P (A) + P (Ā) which gives result.

Exercise 24. Proving Properties of Probabilities Given the three probability axioms and the properties already proved, prove the addition property P (A ∪ B) = P (A) + P (B) − P (A ∩ B). A Venn diagram and an outline of the proof are written below. (a) Write the set A ∪ B as the union of three sets that are mutually exclusive.

A ∪ B = (A ∩ B) ∪ (Ā ∩ B) ∪ (A ∩ B̄) (b) Apply Axiom 2 to write P (A ∪ B) as the sum of three terms.

Since these three sets are mutually exculsive, we can apply Axiom 3: P (A ∪ B) = P (A ∩ B) + P (Ā ∩ B) + P (A ∩ B̄) (c) Write the set A as the union of two mutually exclusive sets.

A = (A ∩ B) ∪ (A ∩ B̄) (d) Apply Axiom 2 to write P (A) as the sum of two terms.

Since these two sets are mutually exculsive, we can apply Axiom 3: P (A) = P (A ∩ B) + P (A ∩ B̄) (e) By writing the set B as the union of two mutually exclusive sets and applying Axiom 2, write P (B) as the sum of two terms.

By similar work, we can show P (B) = P (A ∩ B) + P (Ā ∩ B) (f) By making appropriate substitutions to the expression in part b, one obtains the desired result.

11

P (A ∪ B) = P (A ∩ B) + P (Ā ∩ B) + P (A ∩ B̄) P (A ∪ B) = P (A ∩ B) + [P (B) − P (A ∩ B)] + [P (A) − P (A ∩ B)] and we obtain P (A ∪ B) = P (A) + P (B) − P (A ∩ B)

Chapter 2 - Counting Methods Exercise 1: Constructing a Word Suppose you select three letters at random from {a, b, c, d, e, f} to form a word. a. How many possible words are there?

120 b. What is the probability the word you choose is fad?

1/120 c. What is the probability the word you choose contains the letter “a”?

60/120 d. What is the chance that the first letter in the word is “a”?

20/120 e. What is the probability that the word contains the letters “d”, “e”, and “f”?

1/20

Exercise 2. Running a Race There are seven runners in a race { three runners are from Team A and four runners are from Team B. a. Suppose you record which runners finish first, second, and third. Count the number of possible outcomes of this race.

7 P 3 = 7 x 6 x 5 = 210 b. If the runners all have the same ability, then each of the outcomes in a. are equally likely. Find the probability that Team A runners finish first, second, and third.

3! / 120 = 6/120 c. Find the probability that the first runner across the finish line is from Team A. 7 possible winners, of which 3 are from Team A – prob = 3/7

Exercise 3. Rolling Dice Suppose you roll three fair dice. a. How many possible outcomes are there?

6ˆ3 = 216 b. Find the probability you roll three sixes.

12

1/216 c. Find the probability that all three dice show the same number.

6/216 d. Find the probability that the sum of the dice is equal to 10. different ways 1 + 3 + 6, 1 + 4 + 5, 2 + 2 + 6, 2 + 3 + 5, 2 + 4 + 4, 3 + 3 + 4 prob = (6 + 6 + 3 + 6 + 3 + 3)/216 = 27/2163.

Exercise 4. Ordering Hash Browns When you order Waffle House’s world famous hash browns, you can order them scattered (on the grill), smothered (with onions), chunked (with ham), topped (with chili), diced (with tomatoes), and peppered (with peppers). How many ways can you order hash browns at Waffle House?

2ˆ6 = 64

Exercise 5. Selecting Balls from a Box A box contains 5 balls – 2 are white, 2 are black, and one is green. You choose two balls out of the box at random without replacement. a. How many outcomes do you if the order that you choose balls is not important?

10 b. Find the probability that you choose two white balls.

1/10 c. Find the probability you choose two balls of the same color.

2/10

Exercise 6. Dividing into Teams Suppose that ten boys are randomly divided into two teams of equal size. Find the probability that the three tallest boys are on the same team.

Number of ways of choosing teams = 10 C 5 = 252 Prob = (3 C 3 x 7 C 2 + 3 C 0 x 7 C 5)/10 C 5 = 42 / 252

Exercise 7. Choosing Numbers Suppose you choose three numbers from the set {1, 2, 3, 4, 5, 6, 7, 8} (without replacement). a. How many possible choices can you make?

8 C 3 = 56 b. What is the probability you choose exactly two even numbers?

(4 C 2 x 4 C 1)/(8 C 3) = 24 / 56 c. What is the probability the three numbers add up to 10?

4 / 8 C 3 = 4 / 56 13

Exercise 8. Choosing People Suppose you choose two people from three married couples. a. How many selections can you make?

6 C 2 = 15 b. What is the probability the two people you choose are married to each other?

3/15 c. What is the probability that the two people are of the same gender?

(3 + 3) / 15 = 6 / 15

Exercise 9. Football Plays Suppose a football team has five basic plays, and they will randomly choose a play on each down. a. On three downs, find the probability that the team runs the same play on each down.

5 / 125 b. Find the probability the team runs three different plays on the three downs.

5 x 4 x 3 / 125 = 60 / 125

Exercise 10. Playing the Lottery In a lottery game, you make a random guess at the winning threedigit number (each digit can be 0, 1, 2, 3, 4, 5, 6, 7, 8, 9). You win $200 if your guess matches the winning number, $20 if your guess matches in exactly two positions and $2 if your guess matches in exactly one position. Find the probabilities of winning $200, winning $20, and winning $2.

Win $200 with probability 1/1000 Win $20 with probability 27/1000 Win $2 with probability 243/1000

Exercise 11. Dining at a Restaurant Suppose you are dining at a Chinese restaurant with the menu given on the previous page. You decide to order a combination meal where you get to order one soup or appetizer, one entree (seafood, beef, or poultry), and a side dish (either fried rice or noodles). a. How many possible combination meals can you order?

8 x 14 x 10 = 1120 b. If you are able to go to this restaurant every day, approximately how many years could you order different combination meals? about 3 years c. Suppose that you are allergic to seafood (this includes crab, shrimp, and scallops). How many different combination meals can you order?

6 x 9 x 9 = 486

14

d. Suppose your friend orders two different entrees completely at random. How many possible dinners can she order? What is the probability the two entrees chosen contain the same meat?

14 C 2 = 91, [5 C 2 + 4 C 2 + 5 C 2] / 91 = 26 / 91

Exercise 12. Ordering Pizza If you buy a pizza from Papa John’s, you can you order the following toppings: ham, bacon, pepperoni, Italian sausage, sausage, beef, anchovies, extra cheese, baby portabella mushrooms, onions, black olives, Roma tomatoes, green peppers, jalapeno peppers, banana peppers, pineapple, grilled chicken. a. If you have the option of choosing two toppings, how many different two topping pizzas can you order?

17 C 2 = 136 b. Suppose you want your two toppings to be some meat and some peppers. How many two-topping pizzas are of this type?

6 x 3 = 18 c. If you order a “random” two-topping pizza, what is the chance that it will have peppers?

(3 C 2 14 C 0 + 3 C 1 14 C 1) / 136 = 45 / 136 d. If you are able to order at most four toppings, how many different pizzas can you order?

14 C 0 + 14 C 1 + 14 C 2 + 14 C 3 + 14 C 4 = 1471

Exercise 13. Mixed Letters You randomly mix up the letters “s”, “t”, “a”, “t”, “s”. a. Find the probability the arrangement spells the word “stats”.

Number of ways is 5! / (2! 2! 1) = 30 P(spell ‘stats’) = 1 / 30 b. Find the probability the arrangement starts and ends with “s”.

P(s***s) = 3 / 30

Exercise 14. Arranging CDs Suppose you have three Madonna cds and three Jewel cds sitting on a shelf as follows. The cds are knocked off of the shelf and you place them back on the shelf completely at random. a. What is the probability that the mixed-up cds remain in the same order?

1 / (6 P 3) = 20 b. What is the probability that the first and last cds on the shelf are both Jewel music?

0.2 c. What is the probability that the Jewel cds stay together on the shelf?

0.2

15

Exercise 15. Playing a Lottery Game The Minnesota State Lottery has a game called Daily 3. A three digit number is chosen randomly from the set 000, 001, . . . , 999 and you win by guessing correctly certain characteristics of this three digit number.

The lottery website lists the following possible plays such as First Digit, Front Pair, etc. Find the probability of winning for each play.

• First Digit: Pick one number. To win, match the first number drawn.

• Front Pair: Pick 2 numbers. To win, match the first 2 numbers drawn in exact order • Straight: Pick 3 numbers. To win, match all 3 numbers drawn in exact order.

• 3-Way Box: Pick 3 numbers, 2 that are the same. To win, match all three numbers drawn in any order.

• 6-Way Box: Pick 3 different numbers. To win, match all 3 numbers drawn in any order.

First digit 1/10 Front pair 1/100 Straight 1/1000 3-Way Box 3/1000 6-Way Box 6/1000

Exercise 16. Booking a Flight Suppose you are booking a flight to San Francisco on Orbitz. To save money, you agree to either leave Monday, Tuesday, or Wednesday, and return on either Friday, Saturday, or Sunday. Assume that Orbitz randomly assigns you a day to leave and randomly assigns you a day to return. a. What is the probability you leave on Tuesday and return on Saturday?

1/9 b. What is the chance that your trip will be exactly three days long?

1/9 c. What is the most likely trip length in days?

5 days d. Do you think that the assumptions about Orbitz are reasonable? Explain.

Assumptions are probably not reasonable.

Exercise 17. Assigning Grades A math class of ten students takes an exam. a. If the instructor decides to give exam grades of A to two randomly selected students, how many ways can this be done?

10 C 2 = 45 b. Of the remaining eight students, three will receive B’s and the remaining will receive C’s. How many ways can this be done?

8 C 3 = 56 c. If the instructor assigns at random, two A’s, three B’s and five C’s to the ten students, how many ways can this be done?

16

(10 C 2) (8 C 3) = 2520 d. Under this grading method, what is the probability that Jim (the best student in the class) gets an A? (9 C 1) / (10 C 2) = 9/ 56

Exercise 18. Choosing Officers A club consisting of 8 members has to choose three officers. a. How many ways can this be done?

8 C 3 = 56 b. Suppose that the club needs to choose a president, a vice president, and a treasurer. How many ways can this be done?

8x7x6 c. If the club consists of 4 men and 4 women and the officers are chosen at random, Find the probability the three officers are all of the same gender.

(4 C 3 + 4 C 3) / 56 d. Find the probability the president and the vice-president are different genders.

(4 C 1 x 4 C 1) / (8 C 2) = 16/28

Exercise 19. Playing Yahtzee Find the number of ways and the corresponding probabilities of getting all of the following patterns in Yahtzee.

Here are some hints for the different patterns.

• Four of a kind: The pattern here is x, x, x, x, y, where x is the number that appears four times and y is the number that appears once.

(1) choose (x, y): 6 x 5 (2) mix up orderings: 5 (3) Total number = 6 x 5 x 5 = 150 • Large straight: (1) can either contain 1, 2, 3, 4, 5 or 2, 3, 4, 5, 6 (2) mix up five numbers (3) Total number = 2 * 5! = 2 x 120 = 240 • Small straight: This roll will either include the numbers 1, 2, 3, 4, the numbers 2, 3, 4, 5, or the numbers 3, 4, 5, 6. If the numbers 1, 2, 3, 4 are the small straight, then the remaining number can not be 5 (otherwise it would be a large straight).

(1) 1234? 120 + 4 x 60 (2) 2345? 4 x 60 (3) 3456? 120 + 4 * 60 (4) Total = 360 + 360 + 240 = 960 (720 have one-pair) • Full house: The pattern here is x, x, x, y, y, where x is the number that appears three times and y is the number that appears twice.

(1) Choose x and y: 6 x 5 (2) Mix up numbers: (5 choose 2) = 10 (3) Total number = 30 * 10 = 300

17

• Three of a kind: The pattern here is x, x, x, y, z, where x is the number that appears three times, and y and z are the numbers that appear only once.

(1) choose x, y, z: 6 x (5 choose 2) = 60 (2) mix up numbers: 5! / 3! = 20 (3) Total number = 60 * 20 = 1200 • Two pair: The pattern here is x, x, y, y, z.

I obtained that there are 1800 rolls of this type.

• One pair: The pattern here is x, x, w, y, z, where x is the number that appears two times, and w, y and z are the numbers that appear only once. NOTE: I have to subtract the number of small straights in this group.

(1) 6 * (5 choose 3) = 60 (2) 5! / 2! = 60 (3) Total = 60 x 60 - 720 = 2880 • Nothing: This is the most difficult number to count directly. Once the number of each of the remaining patterns is found, then the number of “nothings” can be found by subtracting the total number of other patterns from the total number of rolls (7776).

I obtained there are 320 ‘nothings’.

Exercise 20. Sampling Letters The built-in vector letters contains the 26 lower-case letters of the alphabet. a. Using the sample() function, take a sample of 10 letters without replacement from letters. sample(letters, size = 10, replace = FALSE) ##

[1] "m" "a" "f" "z" "g" "e" "s" "x" "h" "r" b. Using the sample() function, take a sample of 10 letters with replacement from letters.

sample(letters, size = 10, replace = TRUE) ##

[1] "h" "i" "i" "q" "c" "l" "y" "m" "r" "i"

Exercise 21. Sampling Letters (continued) a. Write a function to take a sample of 10 letters without replacement from letters. letters_sample <- function(){ my_sample <- sample(letters, size = 10, replace = FALSE) my_sample } letters_sample() ##

[1] "t" "h" "p" "l" "q" "d" "f" "m" "s" "b" b. Add a line in the function so that the function returns the number of vowels in the sample.

letters_sample <- function(){ my_sample <- sample(letters, size = 10, replace = FALSE) sum(my_sample %in% c("a", "e", "i", "o", "u")) } letters_sample()

18

## [1] 1 c. Using the function replicate(), take 50 samples, storing the number of vowels from the samples in the vector n_vowels. (n_vowels <- replicate(50, letters_sample())) ## [1] 3 1 3 3 1 1 2 4 2 2 3 2 1 2 3 2 3 2 1 1 5 2 2 1 1 0 2 2 0 1 2 2 0 1 2 2 3 1 ## [39] 1 1 4 4 3 1 2 2 0 3 1 3 d. Approximate the probability that there are two vowels in your sample. n_vowels <- replicate(50, letters_sample()) sum(n_vowels == 2) / 50 ## [1] 0.4

Exercise 22. Simulating Permutations Suppose a license plate in a particular state consists of two letters followed by a number (for example, “CD9” and “EE0” are two possible license plates). a. Write a function to simulate a random license plate. random_license_plate <- function(){ paste(sample(LETTERS, size = 1), sample(LETTERS, size = 1), sample(0:9, 1), sep = "") } random_license_plate() ## [1] "FJ3" b. Using the replicate() function to simulate 50 random license plates. (many_plates <- replicate(50, random_license_plate())) ## [1] "SI7" "KV4" "RH8" "GY5" "JQ6" "AR3" "MA1" "YP9" "HJ5" "CG5" "HI4" "XO1" ## [13] "KQ2" "WZ5" "CF6" "DZ0" "KU3" "RI5" "AP0" "EJ2" "MJ1" "SM1" "NO8" "RX0" ## [25] "UL3" "CU2" "ZX7" "FY9" "YB0" "IT2" "BS8" "NY9" "BO3" "ZV7" "MT9" "RA9" ## [37] "FM0" "OB8" "FZ1" "QE8" "TN9" "ZQ5" "QH2" "WF7" "KF2" "WO5" "QN7" "IL0" ## [49] "EY9" "RG0" c. From the simulated plates, approximate the probability there is at least one vowel in the license plate. random_license_plate2 <- function(){ letters <- sample(LETTERS, size = 2) number <- sample(0:9, size = 1) sum(letters %in% c("A", "E", "I", "O", "U")) } many <- replicate(50, random_license_plate2()) sum(many >= 1) / 50 ## [1] 0.36

Exercise 23. Simulating Yahtzee a. Write a function to roll five dice and record by a 1 or 0 if one observes a large straight.

19

yahtzee1 <- function(){ rolls <- sample(6, size = 5, replace = TRUE) y1 <- sum(sort(rolls) == c(1, 2, 3, 4, 5)) == 5 y2 <- sum(sort(rolls) == c(2, 3, 4, 5, 6)) == 5 y1 + y2 > 0 } b. Use the replicate() function and the function found in part c. to approximate the probability of rolling a large straight. many_sims <- replicate(10000, yahtzee1()) mean(many_sims) ## [1] 0.0287 This is close to the exact probability of 240 / 7776 = 0.0309 c. By changing the function in part a. and using the replicate() function, approximate the probability of rolling a small straight. d. In a similar fashion, approximate the probability of rolling a full house.

Chapter 3 - Conditional Probability Exercise 1. Flipping Coins Suppose you flip a fair coin four times. The 16 possible outcomes of this experiment are shown below.

Outcome

Outcome

Outcome

Outcome

HHHH HT HH T HHH T T HH

HHHT HT HT T HHT T T HT

HHT T HT T T T HT T TTTT

HHT H HT T H T HT H TTTH

(a) Let A denote the event that you flip exactly three heads. Find the probability of A. 4 / 16 = 0.25 (b) Suppose you are given the information N that at least two heads are flipped. Circle the possible outcomes in the reduced sample space based on knowing that event N is true.

Should circle 11 outcomes. (c) Using the reduced sample space, find the conditional probability P (A | N ).

4 / 11 = 0.363 (d) Compare P (A) computed in part a with P (A | N ) computed in part (c). Based on this comparison, are events A and N independent? Why? Since the answers are not equal, events A and N are not independent.

Exercise 2. Choosing a Committee Suppose you randomly choose three people from the group {Sue, Ellen, Jill, Bob, Joe, John} to be on a committee. Below we have listed all possible committees of size three: 20

Outcome

Outcome

Outcome

{Sue, Ellen, Jill} {Sue, Jill, Bob} {Sue, Bob, John} {Ellen, Jill, John} {Jill, Bob, Joe} {Sue, Ellen, John} {Ellen, Joe, John}

{Sue, Ellen, Bob} {Sue, Jill, Joe} {Sue, Joe, John} {Ellen, Bob, Joe} {Jill, Bob, John} {Sue, Bob, Joe} {Bob, Joe, John}

{Sue, Ellen, Joe} {Sue, Jill, John} {Ellen, Jill, Bob} {Ellen, Bob, John} {Jill, Joe, John} {Ellen, Jill, Joe}

(a) Find the probability of the event A that exactly two women are in the committee (Sue, Ellen, and Jill are women; Bob, Joe, and John are men).

9 / 20 (b) Suppose you are told that Jill is on the committee– call this event J. Circle the possible outcomes in the reduced sample space if we know that J is true.

You should circle 10 outcomes. (c) Compute the conditional probability P (A | J).

6 / 10 (d) Based on your computations in parts (a) and (c), are events A and J independent?

No, since (a) and (c) are different values. (e) Let F denote the event that more women are on the committee than men. Find P (F ).

10 / 20 (f) Suppose you are given the information S that all three people on the committee are of the same gender.

Find P (F | S).

1/2 (g) Based on your computations in parts (e) and (f), are events F and S independent?

Yes, F and S are independent.

Exercise 3. Arranging Letters Suppose you randomly arrange the letters a, s, s, t, t. You used a computer to do this arranging 200 times and below lists all of the possible “words” that came up. There were 30 distinct arrangements.

Word

Word

Word

Word

Word

Word

asstt sastt stast tasst tssta

astst satst stats tasts tstas

astts satts stsat tatss tstsa

atsst ssatt ststa tsast ttass

atsts sstat sttas tsats ttsas

attss sstta sttsa tssat ttssa

(a) Assuming each possible arrangement is equally likely, what is the probability that the word formed is “stats”?

1 / 30 (b) What is the probability that the word formed begins and ends with an “s”?

21

3 / 30 (c) Suppose you are told that the word formed starts with “s” – write down all of the possible words in the reduced sample space. stats, sttas, satts (d) Given that the word begins with “s”, what is the probability the word is “stats”?

1/3

Exercise 4. Rolling Two Dice Suppose two dice are rolled. (a) Suppose you are told that the sum of the dice is equal to 7. Write down the six possible outcomes.

Six possible outcomes are (1, 6), (2, 5), (3, 4), (4, 3), (5, 2), (6, 1) (b) Given the sum of the dice is equal to 7, find the probability the largest die roll is 6.

P(largest roll is 6 | sum is 7) = 2 / 6 = 0.333. (c) Suppose you are told that the two dice have different numbers. Write down the possible outcomes.

There are 30 outcomes where dice have different numbers. (d) If the two dice have different numbers, what is the probability the largest die roll is 6?

P(largest roll is 6 | different numbers) = 10 / 30 = 0.333.

Exercise 5. Choosing Sport Balls Suppose you have a bin in your garage with three sports balls – four are footballs, three are basketballs, and two are tennis balls. Suppose you take three balls from the bin – you count the number of footballs and the number of basketballs. The first time this is done, the following balls were selected: basketball, basketball, football, so the number of footballs selected was 1 and the number of basketballs selected was 2. We repeat this sampling experiment 1000 times, each time recording the number of footballs and basketballs we select. Table 3.12 summarizes the results of the 1000 experiments.

Table 3.12. Results of 1000 experiments of choosing two sports balls.

Total

Numbers of Footballs

0 1 2 3 Total

0 13 49 118 109 289

1 66 198 180 0 444

2 64 169 0 0 233

3 34 0 0 0 34

177 416 298 109 1000

Let F1 denote that event that you have chosen exactly one football and B1 the event that you chose exactly one basketball from the bin. (a) Find P (F1 ) and P (B1 ).

P(F1) = 0.416, P(B1) = 0.444

22

(b) Find P (F1 | B1 ).

198 / 444 (c) Find P (B1 | F1 ).

198 / 416 (d) From your calculations above, explain why F1 and B1 are not independent events.

Show that P(F1) is not equal to P(F1 | B1)

Exercise 6. Rating Movies On the Internet Movie Database www.imdb.com, people are given the opportunity to rate movies on a scale from 1 to 10. Table 3.13 shows the ratings of the movie “Sleepless in Seattle” for men and women who visited the website.

Table 3.13. Ratings of “Sleepless of Seattle” by males and females.

Rating Males Females TOTAL

8,9,10

5,6,7

1,2,3,4

(High) 2217 1059 3276

(Medium) 3649 835 4484

(Low) 754 178 932

TOTAL 6620 2072 8692

(a) Suppose you choose at random a person who is interested in rating this movie on the website. Find the probability that the person gives this movie a high rating between 8 and 10 – that is, P (H).

3276 / 8692 = .377 (b) Find the conditional probabilities P (H | M ) and P (H | F ), where M and F are the events that a man and a woman rated the movie, respectively.

2217 / 6620 = .335, 1059 / 2072 = .511 (c) Interpret the conditional probabilities in part (b) – does this particular movie appeal to one gender?

This movie appears to be appeal more to females. (d) Table 3.14 below shows the ratings of the movie "Die Hard” for men and women who visited the website.

Answer questions (a), (b), and (c) for this movie.

Table 3.14. Ratings of “Die Hard” by males and females.

Rating Males Females TOTAL

8,9,10

5,6,7

1,2,3,4

(High) 16197 1720 17917

(Medium) 6737 1243 8180

(Low) 882 258 1140

P(High) = 17917 / 27237 = .658 P(H | M) = 16197 / 24016 = .674 P(H | F) = 1720 / 3221 = .534 This movie appeals more to men

23

TOTAL 24016 3221 27237

Exercise 7. Rating Movies (continued) The Internet Movie Database also breaks down the movie ratings by the age of the reviewer. For the movie “Sleepless in Seattle”, Table 3.15 classifies the reviewers by age and their rating.

Table 3.15. Ratings of “Sleepless in Seattle” by reviewers of different ages.

Rating under 18 18-29 30-44 45+ TOTAL

8,9,10

5,6,7

1,2,3,4

(High) 74 1793 886 438 3191

(Medium) 76 2623 1280 300 4279

(Low) 16 555 272 60 903

TOTAL 166 4971 2438 798 8373

(a) Find the probability that a reviewer gives this movie a high rating – that is, find P (H).

P(H) = 3191 / 8373 = 0.3811059 (b) Define a “young adult” (Y A) as a person between the ages of 18 and 29, and a “senior” (S) as a person 45 or older. Compute P (H | Y A) and P (H | S).

P(H | YA) = 1793 / 4971 = 0.360692 P(H | S) = 438 / 798 = 0.5488722 (c) Based on your computations in parts (a) and (b), is “giving a high rating” and age independent events?

If not, explain how the probability of giving a high rating depends on age.

These are not independent events – chance of a high rating is higher for seniors than for young adults

Exercise 8. Family Planning Suppose a family plans to have children until they have two boys. Suppose there are two events of interest, A = event that they have at least five children and B = event that the first child born is male. Assuming that each child is equally likely to be a boy or girl, and genders of different children born are independent, then this process of building a family was simulated 1000 times. The results of the simulation are displayed in Table 3.16.

Table 3.16. Number of children and gender of first child born in simulation of 1000 families.

Number of Children

Female

Male

TOTAL

0 125 126 250 501

247 138 58 56 499

247 263 184 306 1000

2 3 4 5 or more TOTAL

(a) Use the table to find P (A).

P(A) = 306 / 1000 = 0.306 (b) Find P (A | B) and decide if events A and B are independent.

P(A | B) = 56 / 499 = 0.1122244 Events A and B are not independent 24

(c) Suppose another family plans to continue to have children until they have at least one of each gender.

Table 3.17 shows simulated results of 1000 families of this type . Again find P (A), P (A | B) and decide if events A and B are independent.

Table 3.17. Number of children and gender of first child born in simulation of 1000 families under different scenario.

Number of Children

Female

Male

TOTAL

235 106 71 50 462

261 152 63 62 538

496 258 134 112 1000

2 3 4 5 or more TOTAL

P(A) = 112 / 1000 = 0.112 P(A | B) = 62 / 538 = 0.1152416 Here events A and B appear to be independent.

Exercise 9. Conditional Nature of Probability For each of the following problems • Make a guess at the probabilities of the three events based on your current knowledge.

• Ask a friend about this problem. Based on his or her opinion about the event, make new probability assignments.

• Do some research on the Internet to learn about the right answer to the question. Make new probability assignments based on your new information.

The answers here will vary for each student. (a) What is the area of Pennsylvania?

Initial probabilities: Event

under 30,000 sq miles

between 30,000 and

over 50,000 sq miles

between 30,000 and

over 50,000 sq miles

between 30,000 and

over 50,000 sq miles

Probability

Probabilities after talking with a friend: Event

under 30,000 sq miles

Probability

Probabilities after doing research on the Internet.

Event

under 30,000 sq miles

Probability (b) Robin Williams has appeared in how many movies?

Probabilities after talking with a friend: 25

Event

Under 15

Between 16 and 30

Over 30

Between 16 and 30

Over 30

Between 16 and 30

Over 30

Probability

Probabilities after doing research on the Internet.

Event

Under 15

Probability

Probabilities after doing research on the Internet.

Event

Under 15

Probability

Exercise 10. Conditional Nature of Probability For each of the following problems • Make a guess at the probabilities of the three events based on your current knowledge.

• Ask a friend about this problem. Based on his or her opinion about the event, make new probability assignments.

• Do some research on the Internet to learn about the right answer to the question. Make new probability assignments based on your new information.

The answers here will vary for each student. (a) How many plays did Shakespeare write?

Initial probabilities: Event

Under 30

Between 31 and 50

Over 50

Between 31 and 50

Over 50

Between 31 and 50

Over 50

Probability

Probabilities after talking with a friend: Event

Under 30

Probability

Probabilities after doing research on the Internet.

Event

Under 30

Probability (b) What is the average temperature in Melbourne, Australia in June?

Initial probabilities:

26

Under 40◦

Event

Between 40◦ and 60◦

Over 60◦

Between 40◦ and 60◦

Over 60◦

Between 40◦ and 60◦

Over 60◦

Probability

Probabilities after talking with a friend: Under 40◦

Event Probability

Probabilities after doing research on the Internet.

Under 40◦

Event Probability

Exercise 11. Picnic Misery Twenty boys went on a picnic. 5 got sunburned, 8 got bitten by mosquitoes, and 10 got home without mishap.

What is the probability that the mosquitoes ignored a sunburned boy? What is the probability that a bitten boy was also burned?

Let S = sunburned and M = bitten From the information provided: Sunburn Status

M

not M

S not S

3 5

2 10

P(not M | S) = 2 / 5 = 0.40 P(S | M) = 3 / 8 = .375

Exercise 12. A Mall Survey Suppose 30 people are surveyed at a local mall. Half of the 10 men surveyed approve the upcoming school levy and a total of 17 people do not approve of the levy. Based on the survey data, (a) What is the probability a woman is in favor of the levy?

Based on the info, we can create the following table: Opinion about Levy

Male

Female

Favor Don’t Favor

5 5

8 12

P(Favor | Woman) = 8 / 20 (b) If the person is in favor of the levy, what is the probability the person is a woman?

P(Woman | Favor) = 8 / 13 27

Exercise 13. Drawing Tickets Have 12 tickets numbered from 1 to 12. Two tickets are drawn, one after the other, without replacement. (a) Find the probability that both numbers are even.

P(both are even) = (6 / 12) (5 / 11) = .227 (b) Find the probability both numbers are odd.

P(both are odd) = (6 / 12) (5 / 11) = .227 (c) Find the probability one number is even and one is odd.

P(one is odd, one is even) = 2 (6 / 12) (6 / 11) = .545

Exercise 14. Testing for Steroids Suppose that 20% of all baseball players are currently on steroids. You plan on giving a random player a test, but the test is not perfectly reliable. If the player is truly on steroids, he will test negative (for steroids) with probability 0.1. Likewise, if the player is not on steroids, he will get a positive test result with probability 0.1. (a) What is the probability the player is on steroids and will test negative?

Given: P(Neg Test | S) = 0.1, P(Pos Test | not S) = 0.1, P(S) = 0.20 So P(S and Neg Test) = 0.20 (0.10) (b) If you give a player a test, what is the probability he will test positive?

P(Pos Test) = P(S and Pos Test) + P(not S and Pos Test) = 0.20 (0.9) + 0.80 (0.1) (c) If the test result is positive, what is the probability the player is on steroids?

P(S | Pos Test) = 0.20 (0.9) / (0.20 (0.9) + 0.80 (0.1))

Exercise 15. Preparing for the SAT Suppose a student has a choice of enrolling (or not) in an expensive program to prepare for taking the SAT exam. The chance that she enrolls in this class is 0.3. If she takes the program, the chance that she will do well on the SAT exam is 0.8. On the other hand, if she does not take the prep program, the chance that she will do well on the SAT is only 0.4. Let E denote the event “enrolls in the class” and W denote the event “does well on the SAT exam”. (a) Find P (W | E) P(W | E) = 0.8 (b) Find P (E ∩ W ) P(E) P(W | E) = 0.3 (0.8) = .24 (c) Find P (E | W ), that is, the probability that she took the class given that she did well on the test.

0.24 / (0.24 + 0.7 (0.4)) = 24 / 52 = .462

Exercise 16. Working Off-Campus At a college campus, 33% of the students are freshman and 25% are seniors. 13% of the freshman works over 10 hours off-campus, and 37% of the seniors work over 10 hours off-campus.

28

(a) Suppose you sample a student who is either a freshman and senior. Find the probability she works over 10 hours off-campus.

Here the sample space is “either a freshmen or a senior”. So the initial probabilities of freshman and senior are P(F) = .33 / (.33 + .25) = .57 P(S) = .25 / (.33 + .25) = .43 P(OC) = P(F and OC) + P(S and OC) = (0.57) (0.13) + (0.43) (0.37) (b) If this person does work over 10 hours off campus, find the probability she is a senior.

P(S | OC) = (0.43)(0.37) / ((0.57) (0.13) + (0.43) (0.37))

Exercise 17. Flipping Coins You flip a coin three times. Let A be the event that a head occurs on the first flip and B is the event that (exactly) one head occurs. Are A and B independent?

P(B) = 3 / 8 P(B | A) = 1 / 4 So A and B are not independent.

Exercise 18. A Two-headed Coin?

One coin in a collection of 65 has two heads. Suppose you choose a coin at random from the collection – you toss it 6 times and observe all heads. What is the probability it was the two-headed coin?

P(2H) = 1 / 65, P(regular) = 64 / 65 D = “observe 6 heads” P(D | 2H) = 1, P(D | regular) = 1 / 64 Want P(2H | D) = 1 / 65 (1) / ((1/65)(1) + (64/65)/64) = 1/2

Exercise 19. Smoking and Gender Suppose the proportion of female students at your school is 60%. Also you know that 26% of the male students smoke and only 16% of the female students smoke. Suppose you randomly select a student. (a) Find the probability the student is a male smoker.

P(male and smokes) = (0.4) (0.26) = .104 (b) Find the probability the student smokes.

P(smokes) = (0.4) (0.26) + (0.6) (0.16)= .2 (c) If the student smokes, what is the probability the student is female?

P(female | smokes) = (0.6) (0.16) / .2 = .48

29

Exercise 20. Choosing Until One Selects a Red Suppose you have a box with 4 green and 2 red balls. You select balls from the box one at a time until you get a red, or until you select three balls. If you do not select a red on the first draw, find the probability that you will select three balls.

T = takes three balls NR = don’t select red on first draw want P(T | NR) = P(T and NR) / P(NR) We know you don’t pick Red on first draw, and you will select three balls only if Red is not chosen in 2nd draw. Assuming selections are done without replacement, get P(T | NR) = 3 / (3 + 2)

Exercise 21. Mutually Exclusive and Independence Suppose that two events A and B are mutually exclusive. Are they independent events?

To be independent, must have P(A and B) = P(A) P(B) But LHS is 0 since A, B are mutually exclusive.

A, B will not be independent unless one event (either A or B or both) has 0 probability

Exercise 22. Blood Type of Couples Consider the example where white Americans have the blood types O, A, B, AB with proportions .45, .40, .11, 04. If two people are married (a) Find the probability both people have blood type A. P(A and A) = P(A) P(A) = (.40) (.40) = .16 (b) Find the probability the couple have A and B blood types.

P(A and B) + P(B and A) = 2 (.40) (.11) = .088 (c) Find the probability neither person has an A type.

(1 - .40) (1 - .40) = .36

Exercise 23. Five-Game Playoffs Consider the “best of five” playoff series between the Yankees and the Indians. We assume the probability the Yankees win a single game is 0.6. (a) Find the probability the Yankees win in three games.

0.6 x 0.6 x 0.6 (b) Find the probability the series lasts exactly three games.

0.6 ˆ 3 + 0.4 ˆ 3 (c) Find the probability the series lasts five games and no team wins more than one game in a row.

30

Want to find probability of these outcomes: YIYIY or IYIYI so answer = 0.6 ˆ 2 x 0.4 ˆ 3 + 0.6 ˆ 3 x 0.4 ˆ 2

Exercise 24. Computer and Video Games The Entertainment Software Association reports that of all computer and video games sold, 53% are rated E (Everyone), 30% are rated T (Teen), and 16% are rated M (Mature). Suppose three customers each purchase a game at a local store. Assume that the software choices for the customers can be regarded as independent events. (a) Find the probability that all three customers buy games that are rated E. 0.53 ˆ 3 (b) Find the probability that exactly one customer purchases an M rated game.

Can have M?? or ?M? or ??M where ? means T or E. 3 x .16 (0.84) ˆ 2 (c) Find the probability that the customers purchase games with the same rating.

Outcomes are MMM or EEE or TTT 0.53 ˆ 3 + 0.3 ˆ 3 + 0.16 ˆ 3

Exercise 25. Washer and Dryer Repair Suppose you purchase a washer and dryer from a particular manufacturer. From reading a consumer magazine, you know that 20% of the washers and 10% of the dryers will need some repair during the warranty period. (a) Find the probability that both the dryer and washer will need repair during the warranty period.

P(both need repair) = 0.20 (0.10) (b) Find the probability that exactly one of the machines will need repair.

P(one needs repair) = (0.20) (1 - 0.10) + (1 - 0.20) (0.10) (c) Find the probability the neither machine will need repair.

P(neither needs repair) = (1 - 0.20) (1 - 0.10)

Exercise 26. Basketball Shooting In a basketball game, a player has a “one and one” opportunity at the free-throw line. If she misses the first shot, she is done. If instead she makes the first shot, she will have an opportunity to make a second shot.

From past data, you know that the probability this player will make a single free-throw shot is 0.7. (a) Find the probability the player only takes a single free-throw.

0.3 (b) Find the probability the player makes two shots.

0.7 (0.7) (c) Find the probability the player makes the first shot and misses the second.

31

0.7 (0.3)

Exercise 27. Playing Roulette You play the game of roulette in Reno. Each game you always bet on “red” and the chance that you win is 18/38. Suppose you play the game four times. (a) Find the probability you win in all games.

(18 / 38) ˆ 4 (b) Find the probability you win in the first and third games, and lose in the second and fourth games.

(18 / 38) ˆ 2 (20 / 38) ˆ 2 (c) Find the probability you win in exactly two of the four games.

6 (18 / 38) ˆ 2 (20 / 38) ˆ 2

Exercise 28. Is a Die Fair?

Suppose a friend is about to roll a die. The die either is the usual “fair” type or it is a “special” type that has two sides showing 1, two sides showing 2, and two sides showing 3. You believe that the die is the fair type with probability 0.9. Your friend rolls the die and you observe a 1. (a) Find the probability that a 1 is rolled.

P(roll 1) = (0.9) (1 / 6) + (0.1) (2 / 6) = 0.183 (b) If you observe a 1, what is the probability your friend was rolling the fair die?

(0.9) (1 / 6) / P(roll 1) = 0.820

Exercise 29. How Many Fish?

You are interested in learning about the number of fish in the pond in your back yard. It is a small pond, so you do not expect many fish to live in it. In fact, you believe that the number of fish in the pond is equally likely to be 1, 2, 3, or 4. To learn about the number of fish, you will perform a capture-recapture experiment.

You first catch one of the fish, tag it, and return it to the pond. After a period of time, you catch another fish and observe that it is tagged and this fish is also tossed back into the pond. (a) There are two stages of this experiment. At the first stage you have 1, 2, 3, or 4 fish in the pond, and at the second stage, you observe either a tagged or not-tagged fish. Draw a tree diagram to represent this experiment, and label the branches of the tree with the given probabilities. (b) Find the probability of getting a tagged fish.

P(tagged fish) = 1/4 (1) + 1/4 (1/2) + 1/4 (1/3) + 1/4 (1/4) = 0.5208333 (c) If you find a tagged fish, find the probability there was exactly 1 fish in the pond. Also find the probabilities of exactly 2 fish, 3 fish, and 4 fish in the pond.

Number of fish

Probability

1 2 3 4

.25 / .521 = 0.476 .125 / .521 = 0.238 .083 / .521 = 0.158 .067 / .521 = 0.128

32

Exercise 30. Shopping at the Mall Suppose that you are shopping in a large mall in a metropolitan area. The people who shop at this mall either live downtown or in the suburbs. Recently a market research firm surveyed mall shoppers — from this survey, they believe that 70% of the shoppers live in the suburbs and 30% live downtown. You know that there is a relationship between one’s political affiliation and where the person lives. You know that 40% of the adults who live in the suburbs are registered Democrats and 80% of the downtown residents are Democrats. (a) If you let T = event that shopper lives downtown, S = event that shopper lives in the suburbs and D = event that shopper is a Democrat, write down the probabilities given in the above paragraph.

P(T) = 0.30, P(D | T) = 0.80, P(D | S) = 0.40 (b) Suppose you interview a random shopper. Find the probability that the shopper is a Democrat.

P(D) = 0.30 * 0.80 + 0.70 * 0.40 = 0.52 (c) If your shopper is a Democrat, find the probability he/she lives in the suburbs.

P(S | D) = 0.70 * 0.40 / (0.30 * 0.80 + 0.70 * 0.40) = 0.538

Exercise 31. What Bag? Suppose that you have two bags in your closet. The “white bag” contains four white balls and the “mixed bag” contains two white and two black balls. The closet is dark and you just grab one bag out at random and select a ball. The ball you choose can either be white or black. (a) Suppose there are 1000 hypothetical bags in your closet. By use the Bayes’ box shown below, classify the 1000 bags by the type “white” and ‘mixed" and the ball color observed.

Bag type TOTAL

White Ball

Black Ball

TOTAL

500 250 750

0 250 250

500 500 1000

White Mixed

(b) Using the Bayes’ box, find the probability that you observe a white ball.

From table, P(White Ball) = 750 / 1000 = 0.750 (c) If you observe a white ball, find the probability that you were selecting from the white bag.

P(white bag | white ball selected) = 500 / 750 = 0.667

Exercise 32. Conditional Probability Suppose you have two spinners – one spinner is equally likely to land on the numbers 1, 2, 3, and the second spinner is equally likely to land on 1, 2, 3, 4, 5.

We can show these two spinners using the ProbBayes package. library(ProbBayes) many_spinner_plots(list(c(1, 1, 1), c(1, 1, 1, 1, 1)))

33

Spinner A

Spinner B Region Region

1

1

2

2

3

3

4 5

(a) Using two applications of the sample() function, create a data frame containing 1000 random spins from the first spinner and 1000 random spins from the second spinner. df <- data.frame(Spin_1 = sample(1:3, size = 1000, replace = TRUE), Spin_2 = sample(1:5, size = 1000, replace = TRUE)) (b) Use the filter() function to take a subset of the data frame created in part (a), keeping only the rows where the sum of spins is less than 5. library(dplyr) df1 <- filter(df, Spin_1 + Spin_2 < 5) (c) Using the output from part (b), approximate the probability the first spin is equal to 1 given that the sum of spins is less than 5. mean(df1$Spin_1 == 1) ## [1] 0.5227273 (d) By use of the filter() function, approximate the probability the sum of spins is less than 5 given that the first spin is equal to 1. df %>% filter(Spin_1 == 1) %>% summarize(P = mean(Spin_1 + Spin_2 < 5)) ## P ## 1 0.6408669

Exercise 33. Rolling a Random Die Suppose you spin a spinner that is equally likely to land on the values 1, 2, 3, 4. If the spinner lands on 1, then you roll a fair die; otherwise (if the spinner lands on 2, 3, 4), you roll a weighted die where an even roll is twice as likely as an odd roll. (a) Create a data frame where Spin contains 1000 spins from the random spinner, Die1 contains 1000 rolls from the fair die and Die2 contains 1000 rolls from the biased die. df <- data.frame(Spin = sample(1:4, size = 1000, replace = TRUE), Die1 = sample(1:6, size = 1000,

34

replace = TRUE), Die2 = sample(1:6, size = 1000, prob = c(1, 2, 1, 2, 1, 2), replace = TRUE)) (b) By use of the ifelse() function, define a new variable Die representing the roll of the “random” die, where the outcome depends on the value of the spinner. df$Die <- ifelse(df$Spin == 1, df$Die1, df$Die2) (c) Find the probability the random die roll is equal to 3. mean(df$Die == 3) ## [1] 0.126

Chapter 4 - Discrete Distributions Exercise 1. Coin-tossing Game In the Peter-Paul coin-tossing game described in the text, let the random variable X be the number of times Paul is in the lead. For example, if the coin tosses are HT HHT , Paul’s running winnings are $-2, 0, $2, $4, $2, and the number of times he is in the lead is X = 4.

Outcome

Outcome

Outcome

Outcome

HHHHH HHHHT HHHT H HHHT T HHT HH HHT HT HHT T H HHT T T

HT HHH HT HHT HT HT H HT HT T HT T HH HT T HT HT T T H HT T T T

T HHHH T HHHT T HHT H T HHT T T HT HH T HT HT T HT T H T HT T T

T T HHH T T HHT T T HT H T T HT T T T T HH T T T HT TTTTH TTTTT

(a) Find the probability distribution for X. Here are the values of X for each outcome 5, 4, 3, 1 5, 4, 3, 0 5, 3, 2, 0 5, 2, 1, 0 5, 2, 1, 0 5, 1, 0, 0 4, 1, 0, 0 3, 1, 0, 0 X

P(X)

0

10/32

35

X

P(X)

1 2 3 4 5

6/32 3/32 4/32 3/32 6/32

(b) Construct a graph of the pmf for X. library(ggplot2) df <- data.frame(x = 0:5, P_x = c(10, 6, 3, 4, 3, 6) / 32) ggplot(df, aes(x, P_x)) + geom_col()

0.3

P_x

0.2

0.1

0.0 0

2

4

x (c) What is the most likely value of X? Most likely value of X is 0. (d) Find the probability that X > 2. d. P(X>2) = 13/32 = 0.406

Exercise 2. Sampling Without Replacement Suppose you choose two coins from a box with two nickels and three quarters. Let X denote the number of nickels you draw. (a) Write out all possible 10 outcomes of this experiment.

36

N1 N2, N1 Q1, N1 Q2, N1 Q3, N2 Q1, N2 Q2, N2 Q3, Q1 Q2, Q1 Q3, Q2 Q3 (b) Find the probability distribution for X. X

P(X)

0 1 2

3/10 6/10 1/10

(c) What is the most likely value of X? Most likely value of X is 1 (d) Find the probability that X > 1.

P (X > 1) = 1/10

Exercise 3. Shooting Free Throws Suppose you watch your favorite basketball player attempt five free throw shots during a game. You know that the chance that he is successful on a single shot is 0.5, so that the possible sequences of successes (S) and misses (M ) shown below are equally likely. Suppose you measure the number of runs X where a run is defined to be a streak of S’s or M ’s. For example, in the sequence M M SSM , there are three runs (one run of two misses, one run of two successes, and one run of one miss).

Outcome

Outcome

Outcome

Outcome

SSSSS 1 SSSSM 2 SSSM S 3 SSSM M 2 SSM SS 3 SSM SM 4 SSM M S 3 SSM M M 2

SM SSS 3 SM SSM 4 SM SM S 5 SM SM M 4 SM M SS 3 SM M SM 4 SM M M S 3 SM M M M 2

M SSSS 2 M SSSM 3 M SSM S 4 M SSM M 3 M SM SS 4 M SM SM 5 M SM M S 4 M SM M M 3

M M SSS 2 M M SSM 3 M M SM S 4 M M SM M 3 M M M SS 2 M M M SM 3 MMMMS 2 MMMMM 1

(a) Find the probability distribution for X. X

P(X)

1 2 3 4 5

2/32 8/32 12/32 8/32 2/32

(b) Construct a graph of the pmf for X. library(ggplot2) df <- data.frame(x = 1:5, P_x = c(2, 8, 12, 8, 2) / 32) ggplot(df, aes(x, P_x)) + geom_col()

37

P_x

0.3

0.2

0.1

0.0 1

2

3

4

5

x (c) What is the most likely number of runs in the sequence?

Most likely value of X is 3. (d) Find the probability that you have at most 2 runs in the sequence.

P (X ≤ 2) = 10/32 = 0.3125

Exercise 4. Rolling Two Dice Suppose you roll two dice and you keep track of the larger of the two rolls which we denote by X. For example, if you roll a 4 and a 5, then X = 5. (a) Find the probability distribution for X. X

P(X)

1 2 3 4 5 6

1/36 3/36 5/36 7/36 9/36 11/36

(b) Construct a graph of the pmf for X. library(ggplot2) df <- data.frame(x = 1:6, 38

P_x = c(1, 3, 5, 7, 9, 11) /36) ggplot(df, aes(x, P_x)) + geom_col() 0.3

P_x

0.2

0.1

0.0 2

4

x (c) What is the most likely value of X? Most likely value of X is 6. (d) Find the probability that X is either 5 or 6.

P (X = 5, 6) = 20/36 = 0.556

Exercise 5. Spinning a Spinner Let X denote the number you get when you spin the spinner shown below. library(ProbBayes) spinner_plot(c(2, 2, 2, 3, 3))

39

6

Region 1 2 3 4 5

(a) Find the probability distribution for X. X

P(X)

1 2 3 4 5

1/6 1/6 1/6 1/4 1/4

(b) Find the probability that X ≥ 2 .

P(X >= 2) = 5/6 (c) Find the mean and standard deviation of X. Mean = 3.25, Sigma = 1.42

Exercise 6. Rolling Four Dice Suppose you are asked to roll four dice and record the sum X. A lazy student thinks this is too much work.

As a shortcut, he decides to roll only two dice, record the sum of the dice, and then double the result – call this random variable Y .

The probability distributions of X and Y are shown in Tables 4.12 and 4.13. The distribution of X was obtained by simulating the rolls of four dice for one million trials.

Table 4.12. Probability distribution for X.

40

x

P (X = x)

x

P (X = x)

4 5 6 7 8 9 10 11 12 13 14

0.001 0.003 0.008 0.016 0.027 0.044 0.062 0.080 0.097 0.108 0.113

15 16 17 18 19 20 21 22 23 24

0.108 0.096 0.080 0.062 0.043 0.027 0.015 0.008 0.003 0.001

y

P (Y = y)

y

P (Y = y)

4 6 8 10 12 14

0.028 0.056 0.083 0.111 0.139 0.167

16 18 20 22 24

0.139 0.111 0.083 0.056 0.028

Table 4.13. Probability distribution for Y .

(a) Compute the mean and standard deviation of the probability distributions of X and Y .

Variable

Mean

Standard Deviation

X Y

14.035 14.014

3.45 4.84

(b) Plot the probability distributions of X and Y on the same graph. library(ggplot2) df1 <- data.frame(Variable = 'X', x = 4:24, Prob = c(1, 3, 8, 19, 27, 44, 62, 80, 97, 108, 113, 108, 96, 80, 62, 43, 27, 15, 8, 3, 1) / 1000) df2 <- data.frame(Variable = "Y", x = seq(4, 24, by = 2), Prob = c(28, 56, 83, 111, 139, 167, 139, 111, 83, 56, 28) / 1000) ggplot(rbind(df1, df2), aes(x, Prob)) + geom_col() + facet_wrap(~ Variable, ncol = 1)

41

X 0.15

0.10

0.05

Prob

0.00 Y 0.15

0.10

0.05

0.00 5

10

15

20

25

x (c) Compare and contrast the two probability distributions. How are the distributions similar? How are they different? How would you respond to the lazy student who thinks that doubling a two-dice result is equivalent to finding the sum of four fair dice?

Both X and Y have similar averages, but Y clearly has wider spread than X.

Exercise 7. Running a Marathon Race Suppose three runners from college A and four runners from college B are participating in a marathon race.

Suppose that all seven runners have equal abilities and so all possible orders of finish of the seven runners are equally likely. For example, one possible order of finish is AAABBBB where the three A runners finish first, second, and third. Let X denote the finish position of the best runner from college A. (a) Find the probability distribution of X. There are 7 choose 3 = 35 possible finish positions of the three runners from college A. Positions

Positions

Positions

Positions

1, 2, 3 1, 2, 7 1, 3, 7 1, 5, 6 2, 3, 5 2, 4, 6 2, 6, 7 3, 5, 6 4, 5, 7

1, 2, 4 1, 3, 4 1, 4, 5 1, 5, 7 2, 3, 6 2, 4, 7 3, 4, 5 3, 5, 7 4, 6, 7

1, 2, 5 1, 3, 5 1, 4, 6 1, 6, 7 2, 3, 7 2, 5, 6 3, 4, 6 3, 6, 7 5, 6, 7

1, 2, 6 1, 3, 6 1, 4, 7 2, 3, 4 2, 4, 5 2, 5, 7 3, 4, 7 4, 5, 6

42

X

P (X)

1 2 3 4 5

15 / 35 10 / 35 6 / 35 3 / 35 1 / 35

(b) Find the probability that X is at most 2.

P (X ≤ 2) = 25/35 (c) Find the average finish of the best runner from college A. x <- 1:5 p <- c(15, 10, 6, 3, 1) / 35 (m <- sum(x * p)) ## [1] 2

Exercise 8. Choosing a Slip from a Random Box Suppose you roll a die. If the die roll is 1 or 2, you choose a slip from box 1; otherwise you choose a slip from box 2. Let Y denote the number on the slip.

This line was missing: It should read: "Box 1 contains four slips numbered 1 through 4 and Box 2 contains five slips numbered 2 through 6. (a) Find the probability distribution for Y .

Y

P(Y)

1 2 3 4 5 6

(1/3) (1/4) = 0.083 (1/3) (1/4) + (2/3) (1/5) = 0.217 (1/3) (1/4) + (2/3) (1/5) = 0.217 (1/3) (1/4) + (2/3) (1/5) = 0.217 (2/3) (1/5) = 0.133 (2/3) (1/5) = 0.133

(b) Find the probability that Y is between 2 to 4.

P (2 ≤ Y ≤ 4) = 0.217 + 0.217 + 0.217

Exercise 9. A Random Walk Suppose that a person starts at location 0 on the number line and each minute he is equally likely to take a step to the left and to the right. Let Y denote the person’s location after four steps. (ASSUME EACH STEP IS ONE UNIT.) (a) Find the probability distribution for Y .

43

Y

P(Y)

-4 -2.

0.

2.

4.

1/16 4/16 6/16 4/16 1/16

(b) Find the probability that he is at least two steps away from his start after four steps.

P (|Y | ≤ 2) = 1 − 6/16 = 10/16 (c) Suppose there is some gravitational pull towards the 0 (home) location. Then if he is currently at a negative location, the probability he will take a positive step is 0.7, and likewise if he is at a positive location, the probability he takes a negative step is 0.7. If he is at point 0, he is equally likely to take a negative or positive step. Find the probability distribution of Y .

Here are the probs for each possible sequence of steps (the values of Y are also given): Sequence

Probability

Y

LLLL LLRL LLLR LLRR LRLL LRRL LRLR LRRR RLLL RLRL RLLR RLRR RRLL RRRL RRLR RRRR

.5 x .3 x .3 x .3 .5 x .3 x .7 x .3 .5 x .3 x .3 x .7 .5 x .3 x .7 x .7 .5 x .7 x .5 x .3 .5 x .7 x .5 x .7 .5 x .7 x .5 x .7 .5 x .7 x .5 x .3 .5 x .7 x .5 x .3 .5 x .7 x .5 x .7 .5 x .7 x .5 x .7 .5 x .7 x .5 x .3 .5 x .3 x .7 x .7 .5 x .3 x .3 x .7 .5 x .3 x .7 x .3 .5 x .3 x .3 x .3

-4 -2 -2 0 -2 0 0 2 -2 0 0 2 0 2 2 4

I get the following distribution: Y

P(Y)

-4.

-2 0.

2.

4.

0.0135 0.168 0.637 0.168 0.0135

(d) Compare the two probability distributions in parts (a) and (c) using the mean and standard deviation.

Can show that both distributions have mean 0 First distribution: sigma = 2 Second distribution: sigma = 1.33

44

Exercise 10. Selecting a Prize from a Bag Suppose you select a prize (with replacement) from a bag that contains three prizes – one worth $1, one worth $5, and one worth $10. You have three opportunities to select a prize and you get to keep the largest prize of the three you select. Let X denote the value of the prize you keep. (a) Find the probability distribution of X. There are 3 x 3 x 3 = 27 possible outcomes.

X

Probability

1 5 10

1/27 = 0.037 7/27 = 0.222 19/27 = 0.704

(b) Find the probability you win more than $1.

26 / 27 = 0.963 (c) Find your expected winning. x <- c(1, 5, 10) p <- c(1, 7, 19) / 27 sum(x * p) ## [1] 8.37037

Exercise 11. Playing Roulette Suppose you place a single $5 bet on three numbers (the Trio Bet) in roulette that has a payoff odds of 11 to 1. Let X denote your payoff. Recall that if you win you receive 11 times your betting amount plus your $5 bet; if you lose, your payoff is nothing. (a) Find the probability distribution for X X

P(X)

0 60

35/38 3/38

(b) Find the mean of X. On average, how much money do you lose in a single $5 bet?

Mean of X is $4.73, 27 cents (c) Consider placing $5 instead on a Five Number Bet that pays at 6 to 1. Find the probability distribution for the payoff Y for this bet. Compute the mean of Y . How does this average payoff compare with the average payoff for the Trio Bet? X

P(X)

0 35

33/38 5/38

Mean of Y is $4.61 (d) Find the standard deviation of the payoffs for X and Y . Which bet has the larger standard deviation?

45

Interpret what it means to have a large standard deviation. sd(X) = 16.2, sd(Y) = 11.83

Exercise 12. Sum of Independent Random Variables Suppose you have k random variables X1 , ..., Xk . Each random variable has a mean µ and a standard deviation σ. Suppose the random variables are independent – this means that the probability that one variable, say takes a value will not be affected by the values of the other random variables. In this case, it can be shown that the mean and standard deviation of the sum S = X1 + ... + Xk will have mean E(S) = kµ √ and standard deviation SD(S) = kσ. (a) It has been shown that if X denotes the roll of a single die, then the mean and standard deviation of X are given by µ = 3.5 and σ = 1.71. Suppose you roll 10 dice and the outcomes of these dice are represented by X1 , ..., X10 . Using the above result, find the mean and standard deviation of the sum of these 10 rolls. √ Here n = 10, so the mean of S is 10 x 3.5 = 35, and the standard deviation of S is 10 × 1.71 = 5.41 (b) Suppose you spin the spinner pictured here five times and record the sum of the five spins S. Find the mean and standard deviation of S. [Hint: First you need to find the mean and standard deviation of X, a single spin of the spinner. Then you can apply the above result.]

Probability distribution for X: X

P(X)

1 2 3

1/2 1/3 1/6

mean and sd of X are µ = 1.67 and σ = 0.74 So mean and sd for S are µ = 5 × 1.67 = 8.35 and σ ×

√

5 × 0.74 = 1.65

Exercise 13. Selecting a Coin from a Box Suppose you select a coin from a box containing 3 nickels, 2 dimes and one quarter. Let X represent the value of the coin. (a) Find the probability distribution of X. X

p(X)

5 10 25

3/6 2/6 1/6

(b) Find the mean and standard deviation of X. µ = 10, σ = 7.07 (c) Suppose that your instructor will give you twice the value of the coin that you select, so your profit is Y = 2X. Make intelligent guesses at the mean and standard deviation of Y . (d) Check your guesses by actually computing the mean and standard deviation of Y .

46

mean of Y : µ = 2 × 10 = 20 sd of Y : 2 × σ = 7.07 = 14.14 (e) This is an illustration of a general result. If X has mean µ and standard deviation σ and Y = cX where c is a positive constant, then the mean of Y is equal to cµ and the standard deviation of Y is equal to cσ.

Exercise 14. How Many Tries to Open the Door?

You have a ring with four keys, one of which will open your door. Suppose you try the keys in a random order until you open the door. Let X denote the number of wrong keys you try before you find the right one.

It can be shown that X has the following distribution. x

P (X = x)

0 1 2 3

1/4 1/4 1/4 1/4

(a) Find the mean and standard deviation of X. Mean of X is 1.5, sd is 1.12 (b) Suppose you record instead Y , the total number of keys you try. Note that Y = X + 1. Find the probability distribution for Y and the mean and standard deviation. x

P (Y = x)

1 2 3 4

1/4 1/4 1/4 1/4

Mean of Y is 2.5, sd is 1.12 (c) This is an illustration of a general result. If X has mean µ and standard deviation σ and Y = X + c for some constant c, then the mean of Y is equal to and the standard deviation of Y is equal to .

Exercise 15. The Hat Check Problem Consider the hat check problem described in Section 4.1. Consider the special case where n = 4 men are checking their hats. If the names of the four men are represented by the initials A, B, C, D, then you can represent the hats given to these four men by the arrangements ABCD, ABDC, and so on. (a) Write down the 24 possible arrangements and find the probability distribution for X, the number of matches.

Arrangement

X

Arrangement

X

Arrangement

X

Arrangement

X

ABCD ABDC ACBD

4 2 2

BACD BADC BCAD

2 0 1

CABD CADB CBAD

1 0 2

DABC DACB DBAC

0 1 1

47

Arrangement

X

Arrangement

X

Arrangement

X

Arrangement

X

ACDB ADBC ADCB

1 1 2

BCDA BDAC BDCA

0 0 1

CBDA CDAB CDBA

1 0 0

DBCA DCAB DCBA

2 0 0

X

P(X)

0 1 2 4

9/24 8/24 6/24 1/24

(b) Find the probability of no matches.

9/24 = 0.375 (c) Find the expected number of matches.

P µ= xP (x) = 1

Exercise 16. Binomial Experiments Is each random process described below a Binomial experiment? If it is, give values of n and p. Otherwise, explain why it is not Binomial. (a) Roll a die 20 times and count the number of sixes you roll. binomial, n = 20, p = 1/6 (b) There is a room of 10 women and 10 men – you choose five people from the room without replacement and count the number of women you choose. not binomial (c) Same process as part (b) but you sample with replacement instead of without replacement. binomial, n = 20, p = 1/2 (d) You flip a coin repeatedly until you observe 3 heads. negative binomial, r = 3, p = 1/2 (e) The spinner below is spun 50 times – you count the number of spins in the black region. binomial, n = 50, p = 1/4

Exercise 17. Binomial and Negative Binomial Experiments Each of the random processes below is a Binomial experiment, a Negative Binomial experiment, or neither. If the process is Binomial, give values of n and p, and if the process is Negative Binomial, give values of r and p. (a) Suppose that 30% of students at a college regularly commute to school. You sample 15 students and record the number of commuters.

Binomial - n = 15, p = .3 (b) Same scenario as part (a). You continue to sample students until you find two commuters and record the number of students sampled.

48

Negative binomial - r = 2, p = .3 (c) Suppose that a restaurant offers apple and orange juice. From past experience, the restaurant knows that 30% of the breakfast customers order apple juice, 50% order orange juice, and 20% order no juice.

One morning, the restaurant has 30 customers and the numbers ordering apple juice, orange juice, and no juice are recorded. no — multinomial (d) Same scenario as part (c). The restaurant only records the number ordering orange juice out of the first 30 customers.

Binomial, n = 30, p = .5 (e) Same scenario as part (c). The restaurant counts the number of customers that order breakfast until exactly three order apple juice.

Negative binomial, r = 3, p = .5 (f) Same scenario as part (c). Suppose that from past experience, the restaurant knows that 40% of the breakfast bills will exceed $10. Of the first 30 breakfast bills, the number of bills exceeding $10 is observed.

Binomial, n = 30, p = .4

Exercise 18. Shooting Free Throws Suppose that Michael Jordan makes 80% of his free throws. Assume he takes 10 free shots during one game. (a) What is the most likely number of shots he will make?

If X = number of successful shots, here is the n = 10, p = 0.8 binomial distribution. cbind(x = 0:10, dbinom(0:10, size = 10, prob = 0.8)) ## x ## [1,] 0 0.0000001024 ## [2,] 1 0.0000040960 ## [3,] 2 0.0000737280 ## [4,] 3 0.0007864320 ## [5,] 4 0.0055050240 ## [6,] 5 0.0264241152 ## [7,] 6 0.0880803840 ## [8,] 7 0.2013265920 ## [9,] 8 0.3019898880 ## [10,] 9 0.2684354560 ## [11,] 10 0.1073741824 The most likely value of X is 8. (b) Find the probability that he makes at least 8 shots.

P (X ≥ 8) = 1 - pbinom(7, size = 10, prob = .8) ## [1] 0.6777995 (c) Find the probability he makes more than 5 shots.

P (X > 5) =

49

1 - pbinom(5, size = 10, prob = .8) ## [1] 0.9672065

Exercise 19. Purchasing Audio CDs Suppose you know that 20% of the audio cd’s sold in China are defective. You travel to China and you purchase 20 cd’s on your trip.

Here X = number of defectives which is binomial with n = 20, p = 0.2. (a) What is the probability that at least one cd in your purchase is defective?

P(X >= 1) = 1 - pbinom(0, size = 20, prob = 0.2) ## [1] 0.9884708 (b) What is the probability that between 4 and 7 cd’s are defective?

P (4 ≤ X ≤ 7) = pbinom(7, size = 20, prob = 0.2) - pbinom(3, size = 20, prob = 0.2) ## [1] 0.5564085 (c) Compute the “average” number of defectives in your purchase. np = 20 × 0.2 = 4

Exercise 20. Rolling Five Dice Suppose you roll five dice and count the number of 1’s you get. (a) Find the probability you roll exactly two 1’s. Perform an exact calculation.

P(2) = (5 choose 2) (1/6)ˆ2 (5/6)ˆ3 = .161 (b) Find the probability all the dice are 1’s. Perform an exact calculation.

P(5) = (1/6)ˆ5 = .00013 (c) Find the probability you roll at least two 1’s. Perform an exact calculation.

1 - P(0) - P(1) = .1968

Exercise 21. Choosing Socks from a Drawer Suppose a drawer contains 10 socks, of which 4 are brown. We select 5 socks from the drawer with replacement. (a) Find the probability two of the five selected are brown.

P(2) = (5 choose 2) .4 ˆ 2 .6 ˆ 3 = .346 (b) Find the probability we choose more brown than non-brown.

P(X > 2) = .317 (c) How many brown socks do we expect to select?

5 (.4) = 2

50

(d) Does the answer to part a change if we select socks from the drawer without replacement? Explain.

YES – no longer a binomial experiment

Exercise 22. Choosing Socks from a Drawer Suppose that we select socks from the drawer with replacement until we see two that are brown. (a) Find the probability that it takes us four selections.

P(Y = 4) = NB(4, .4, 2) = .173 (b) Find the probability it takes more than 2 selections.

P(Y > 2) = .840 (c) How many selections do we expect to make?

E(Y) = 2 / .4 = 5

Exercise 23. Sampling Voters In your local town, suppose that 60% of the residents are supportive of a school levy that will be on the ballot in the next election. You take a random sample of 15 residents. (a) Find the probability that a majority of the sample support the levy.

Binomial, X is number in support of levy, n = 15, p = 0.6.

P (X ≤ 8) = 0.7868968 (b) How many residents in the sample do you expect will support the levy?

Mean µ = n ∗ p = 15(.8) =12 (c) If you sample the residents one at a time, find the probability that it will take you five residents to find three that support the levy.

Negative binomial with r = 3, p = 0.6.

Y is the number of failures until 3rd success.

Want P (Y = 2) = 0.20736

Exercise 24. Taking a True/False Test Suppose you take a true/false test with twenty questions and you guess at the answers. (a) Find the probability you pass the test assuming that passing is 60% or higher correct.

X is number of correct answers, binomial with parameters n = 20 and p = 0.5.

P (X ≥ 12) = 1 - pbinom(11, size = 20, prob = 0.5) ## [1] 0.2517223 (b) Find the probability you get a B or higher where B is 80% correct.

51

P (X ≥ 16) = 1 - pbinom(15, size = 20, prob = 0.5) ## [1] 0.005908966 (c) If you get an 80% on this test, is it reasonable to assume that you were guessing? Explain.

Since the probability of getting 80% or higher is so small if you were guessing, it is not reasonable to make this assumption.

Exercise 25. Bernoulli Experiment Let X1 denote the result of one Binomial trial, where X1 = 1 if you observe a success and $X_1 = 0 $ if you observe a failure. Find the mean and variance of X1 .

X1 has this distribution: X1

P (X1 )

0 1

1−p p

µ = p, σ 2 = p(1 − p)

Exercise 26. Rolling a Die Suppose we roll a die until we observe a 6. This is a special case of a Negative Binomial experiment where r = 1 and p = 1/6. When we are interested in the number of trials until the first success, this is a Geometric experiment and Y is a Geometric random variable. (a) Find the probability that it takes you 4 rolls to get a 6.

This is equivalent to P (Y = 3) = 0.0964506 (b) Find the probability that it takes you more than 2 rolls to get a 6.

P (Y >= 1) = 0.8333333 (c) How many rolls do you need, on average, to get a 6?

E(Y ) = 1/(1/6) = 6

Exercise 27. Heights of Male Freshmen Suppose that one third of male freshmen entering a college are over 6 feet tall. Four men are randomly assigned to a dorm room. Let X denote the number of men in this room that are under 6 feet tall. You can ignore the fact that the actual sampling of men is done without replacement. (a) Assuming X has a Binomial Distribution, what is a “success” and give values of n and p. “success” is under 6 feet ball X is binomial with n = 4 and p = 2/3 (b) What is the most likely value of X? What is the probability of this value?

52

Here is the prob distribution for X: cbind(0:4, dbinom(0:4, size = 4, prob = 2/3)) ## [,1] [,2] ## [1,] 0 0.01234568 ## [2,] 1 0.09876543 ## [3,] 2 0.29629630 ## [4,] 3 0.39506173 ## [5,] 4 0.19753086 X = 3 is the most likely value. (c) Find the probability that at least three men in this room will be under 6 feet tall.

P (X ≥ 3) = 1 - dbinom(2, size = 4, prob = 2/3) ## [1] 0.7037037

Exercise 28. Basketball Shooting Suppose a basketball player is practicing shots from the free-throw line. She hasn’t been playing for a while and she becomes more skillful in making shots as she is practicing. Let X represent the number of shots she makes in 50 attempts. Explain why the Binomial distribution should not be used in finding probabilities about X. I would think that the probability of success would change (increase) as she continues shooting, so the binomial distribution wouldn’t apply.

Exercise 29. Collecting Posters from Cereal Boxes Suppose that a cereal box contains one of four posters and you are interested in collecting a complete set.

You first purchase one box of cereal and find poster #1. (a) Let X2 denote the number of boxes you need to purchase to find a different poster than #1. Find the expected value of X2 .

This is a negative binomial experiment with r = 1 and p = 3/4 – the expected value of boxes is 1/(3/4) = 4/3. (b) Once you have found your second poster, say #2, let X3 denote the number of boxes you need to find a different poster than #1 or #2. Find the expected value of X3 .

This is a negative binomial experiment with r = 1 and p = 1/2 – the expected value of boxes is 1/(1/2) = 2. (c) Once you have collected posters #1, #2, #3, let X4 denote the number of boxes you need to purchase to get poster #4. Find the expected value of X4 .

This is a negative binomial experiment with r = 1 and p = 1/4 – the expected value of boxes is 1/(1/4) = 4. (d) How many posters do you need, on average, to get a complete set of four?

Adding up answers, we get 1 + 4/3 + 2 + 4 = 8.3333333

53

Exercise 30. Baseball Hitting In baseball, it is important for a batter to get “on-base” and batters are rated in terms of their on-base percentage. In the 2004 baseball season, Bobby Abreu of the Philadelphia Phillies had 705 “plate appearances” or opportunities to bat. Suppose we divide his plate appearances into groups of five – we record the number of times Abreu was on-base for plate appearances 1 through 5, for 6 through 10, for 11 through 15, and so on.

If we let X denote the number of times on-base for five plate appearances, then we observe the following counts for X: x

0

1

2

3

4

5

Total

Count

10

29

44

40

15

3

141

To help understand this table, note that the count for X = 1 is 29 – this means there were 29 periods where Abreu was on-base exactly one time. The count for X = 2 is 44 – this means that for 44 periods Abreu was on-base two times.

Since each outcome is either a success or failure, where success is getting on-base, one wonders if the variation in these data can be explained by a Binomial distribution. (a) Find the probabilities for a Binomial distribution with n = 5 and p = 0.443. This value of p is Abreu’s on-base rate for the entire 2004 baseball season. Place these probabilities in the P (X = x) row of the table. x <- 0:5 px <- round(dbinom(x, size = 5, p = 0.443), 4) rbind(x, px) ## [,1] [,2] [,3] [,4] [,5] [,6] ## x 0.0000 1.0000 2.0000 3.0000 4.0000 5.0000 ## px 0.0536 0.2132 0.3391 0.2697 0.1073 0.0171 (b) Multiply the probabilities you found in part (a) by 141, the number of periods in the 2004 season. Place these numbers in the Expected Count row of the table. These represent the expected number of times Abreu would have 0, 1, 2, .., 5 times on-base if the probabilities followed a Binomial distribution. expected <- round(141 * px, 1) rbind(x, px, expected) ## [,1] [,2] [,3] [,4] [,5] [,6] ## x 0.0000 1.0000 2.0000 3.0000 4.0000 5.0000 ## px 0.0536 0.2132 0.3391 0.2697 0.1073 0.0171 ## expected 7.6000 30.1000 47.8000 38.0000 15.1000 2.4000 (c) Compare the expected counts with the actual observed counts in the first table. Does a Binomial distribution provide a good description of these data? observed <- c(10, 29, 44, 40, 15, 3) rbind(x, expected, observed) ## [,1] [,2] [,3] [,4] [,5] [,6] ## x 0.0 1.0 2.0 3 4.0 5.0 ## expected 7.6 30.1 47.8 38 15.1 2.4 ## observed 10.0 29.0 44.0 40 15.0 3.0 We observed a few more 0’s and a few less 2’s than one would expect. Actually the binomial distribution looks like a good model in this case.

54

Exercise 31. Graphs of Binomial Distributions Figure 4.6 shows the Binomial distributions with n = 20 and p = 0.5 and n = 20 and p = 0.2 .

Recall in Section 4.4 that if a probability distribution is approximately bell-shaped, then approximately 68% of the probability falls within one standard deviation of the mean. (a) For the Binomial distribution with n = 20 and p = 0.5, find the mean µ and standard deviation σ and compute the interval (µ − σ, µ + σ). p Mean is 20 × 0.5 = 10, stand. deviation is 20(.5)(.5) = 2.24 (b) Find the exact probability that X falls in the interval (µ − σ, µ + σ).

P (10 − 2.24 ≤ X ≤ 10 + 2.24) = 0.736824 (c) Repeat parts a and b for the Binomial distribution n = 20 and p = 0.2. p Mean is 20 × 0.2 = 4, stand. deviation is 20(.2)(.8) = 1.79 P (10 − 1.79 ≤ X ≤ 10 + 1.79) = 0.5981231 (d) For which distribution was the 68% rule more accurate? Does that make sense based on the shapes of the two distributions?

Approximation is more accurate for the case where p = 0.5. This makes sense since histogram is more normal shaped in this case.

Exercise 32. Guessing on a Test Students in a statistics class were given a five-question baseball trivia quiz. On each question, the students had to choose one of two possible answers. The number correct X was recorded for each student – a count table of the values of X are shown below. (a) Suppose the students know little about baseball and so they are guessing on each question. If this is true, find the probability distribution of the number correct X. X has a binomial distribution with n = 5 and p = 0.5. (b) Using this distribution, find the probability of each value of X and place these probabilities in the above table. x <- 0:5 px <- round(dbinom(x, size = 5, prob = 0.5), 3) rbind(x, px) ## [,1] [,2] [,3] [,4] [,5] [,6] ## x 0.000 1.000 2.000 3.000 4.000 5.000 ## px 0.031 0.156 0.312 0.312 0.156 0.031 (c) By multiplying these probabilities by the number of students (21), find the expected number of students for each value of X. expected <- round(21 * px, 1) rbind(x, px, expected) ## [,1] [,2] [,3] [,4] [,5] [,6] ## x 0.000 1.000 2.000 3.000 4.000 5.000 ## px 0.031 0.156 0.312 0.312 0.156 0.031 ## expected 0.700 3.300 6.600 6.600 3.300 0.700

55

(d) Compare your expected counts with the actual counts – does a Binomial distribution seem like a reasonable assumption in this example? observed <- c(0, 3, 4, 7, 6, 1) rbind(x, expected, observed) ## [,1] [,2] [,3] [,4] [,5] [,6] ## x 0.0 1.0 2.0 3.0 4.0 5.0 ## expected 0.7 3.3 6.6 6.6 3.3 0.7 ## observed 0.0 3.0 4.0 7.0 6.0 1.0 I see that the students tended to get more correct than expected. (Maybe these students knew some baseball.)

Exercise 33. Playing Roulette Suppose you play the game roulette 20 times. Each game, you place a Trio Bet on three numbers and you win with probability 3/38. (a) Find the probability you win the game exactly two times.

Number of wins X is binomial with parameters n = 20 and p = 3/38.

P (X = 2) = 0.2694939 (b) Find the probability that you are winless in the 20 games.

P (X = 0) = 0.1930585 (c) Find the probability you win at least once.

P (X ≤ 1) = 0.8069415 (d) How many games do you expect to win?

20 × (3/38) = 1.5789474

Exercise 34. The Galton Board Consider the Galton board described in Section 4.5. A ball is placed above the first peg and dropped. When it strikes a peg, it is equally likely to fall left or right. The location at the bottom X is equal to the number of times that the ball falls right. (a) Explain why X has a Binomial distribution and give the values of n and p.

X is binomial with n = 5 and p = 0.5. (b) Find P (X = 2).

P (X = 2) = 0.3125 (c) Find the probability the ball falls to the right of the location “1”.

P (X > 1) = 0.8125 (d) Suppose that we change the experiment so that the probability of falling right is equal to 1/4. Explain how this changes the Binomial experiment and find P (X = 2).

Parameters are now n = 5 and p = 1/4.

P (X = 2) = 0.2636719

56

Exercise 35. Drug Testing In a New York Times article “Facing Questions, Rodriguez Raises More?” (February 21, 2008), Major League Baseball is said to have a drug-testing policy where 600 tests are randomly given to a group of 1200 professional ballplayers. Alex Rodriguez claimed one season that he received five random tests. (a) If every player is equally likely to receive a single random blood test, what is the probability that Rodriguez gets tested? p = 1/1200 (b) If X represents the number of tests administered to Rodriguez among the 600 test, then explain why X has a Binomial distribution and give the values of n and p. n = 600, p = 1/1200 (c) Compute the probability that Rodriguez receives exactly one test.

P (X = 1) = 0.303455 (d) Recall Rodriguez’s claim that he received five random tests. Compute the probability of this event.

P (X = 5) = \ensuremath{1.5594983\times 10ˆ{-4}} (e) You should find the probability computed in part (d) to be very small. If Rodriguez is indeed telling the truth, what do you think about the randomness of the drug-testing policy?

It does not appear to be a random drug policy.

Exercise 36. Peter-Paul Game (a) Implement the Peter-Paul game simulation as described in the text, storing 1000 values of the gain variable in the R variable G. one_play <- function(){ flips <- sample(c("H", "T"), size = 5, replace = TRUE) 2 * sum(flips == "H") 2 * sum(flips == "T") } G <- replicate(1000, one_play()) (b) Use the simulated values to estimate the probability P (G > 2). sum(G > 2) / 1000 ## [1] 0.206 (c) Estimate the standard deviation of G from the simulated values. sd(G) ## [1] 4.303974

Exercise 37. The Hat Check Problem (continued) Suppose that n = 10 men are checking their hats. It would be too tedious to write down all 10! = 3, 628, 800 possible arrangements of hats, but it is straightforward to design a simulation experiment for this problem. (a) Write a function to mix up the integers 1 through 10 and returning the number of matches.

57

one_hat_mixup <- function(n){ random_hats <- sample(n) sum(1:n == random_hats) } one_hat_mixup(10) ## [1] 4 (b) Using the function written in part (a), simulate this experiment 1000 times. Approximate the probability of no matches and the expected number of matches. Compare your answers with the “large sample” answers given in the introduction to this chapter.

Matches <- replicate(1000, one_hat_mixup(10)) sum(Matches == 0) / 1000 ## [1] 0.379 mean(Matches) ## [1] 0.979

Exercise 38. A Random Walk (continued) Suppose that a person starts at location 0 on the number line and each minute he is equally likely to take a step to the left and to the right. Let Y denote the person’s location after four steps. (a) Write a function to implement one random walk, returning the person’s location after four steps. one_walk <- function(){ sum(sample(c(-1, 1), size = 4, replace = TRUE)) } one_walk() ## [1] -4 (b) By use of the replicate() function, simulate this random walk for 1000 iterations. Summarize the simulated locations by a mean and standard deviation. many_walks <- replicate(1000, one_walk()) mean(many_walks) ## [1] -0.026 sd(many_walks) ## [1] 1.979708 (c) Make an adjustment to your function so that if the person is currently at a negative location, the probability he will take a positive step is 0.7, and likewise if he is at a positive location, the probability he takes a negative step is 0.7. (If he is at point 0, he is equally likely to take a negative or positive step.) Simulate this adjusted random walk 1000 iterations. Compute the mean and standard deviation of this new random walk and compare to the values computed in part (b). new_walk <- function(){ location <- 0 for (j in 1:4){ if(location == 0){ step <- sample(c(-1, 1), size = 1) } else

58

if(location < 0){ step <- sample(c(-1, 1), prob=c(.3, .7), size = 1) } else { step <- sample(c(-1, 1), prob=c(.7, .3), size = 1) } location <- location + step

} location

} new_walks <- replicate(1000, new_walk()) mean(new_walks) ## [1] -0.008 sd(new_walks) ## [1] 1.336309

Exercise 39. Dice Rolls (a) Construct a data frame with variables roll1, roll2, . . . , roll5, each containing 1000 simulated rolls of a fair die. roll1 <- sample(1:6, size = 1000, replace = TRUE) roll2 <- sample(1:6, size = 1000, replace = TRUE) roll3 <- sample(1:6, size = 1000, replace = TRUE) roll4 <- sample(1:6, size = 1000, replace = TRUE) roll5 <- sample(1:6, size = 1000, replace = TRUE) (b) Using the function pmax() as shown below, define a new variable Max that is equal to the maximum among the five rolls for each of the 1000 iterations.

Max <- pmax(roll1, roll2, roll3, roll4, roll5) (c) Estimate the probability that the maximum roll is equal to 6. sum(Max == 6) / 1000 ## [1] 0.581 (d) Estimate the mean and standard deviation of the maximum roll. mean(Max) ## [1] 5.409 sd(Max) ## [1] 0.8212146

Exercise 40. Binomial Experiments (a) Suppose 25 percent of the students are commuters. You take a survey of 12 students and count X the number of commuters. Simulate 1000 surveys using the function rbinom(), storing the number of commuters in these 1000 samples. n_commuters <- rbinom(1000, size = 12, prob = 0.25)

59

(b) Approximate the probability that exactly 3 people in your sample are commuters. sum(n_commuters == 3) / 1000 ## [1] 0.248 (c) Compute the sample mean and standard deviation of the simulated values and compare with the exact values of the mean µ and standard deviation σ. mean(n_commuters) ## [1] 2.985 sd(n_commuters) ## [1] 1.57202 Actual mean is 3 Actual sd is 1.5

Chapter 5 - Continuous Distributions Exercise 1. Waiting at a ATM Machine You are waiting at your local ATM machine and as usual, you are waiting in a line. Suppose you know that your waiting time can be between 0 to 5 minutes and any value between 0 and 5 minutes is equally likely. (a) The graph below shows the density function for X, the waiting time. What is the height of this function?

1/5 (b) Find the probability you wait more than 2 minutes.

3/5 (c) Find the probability you wait between 2 and 3 minutes.

1/5

Exercise 2. Morning Wake-Up Suppose you wake up at a random time in the morning between 6 am and 12 pm. (a) Find the probability you wake up before 11 am.

1/6 (b) Find the probability you wake up between 8 and 10 am.

2/6 (c) What is an “average” or typical time you will wake up? Explain how you computed this number.

9 pm (this is the mean of this uniform distribution) (d) Find the standard deviation of the time.

60

The variance is given by the integral Z 12 V ar(X) = 6

(x − 9)2 dx 6

The standard deviation is the square root of variance – get sqrt((12 - 6) ^ 2 / 12) ## [1] 1.732051

Exercise 3. The Median Waiting Time In the “waiting for a bus” example, suppose that you record the median time T (in minutes) that you wait for the bus on the three days. The density function for this median time is given by f (t) =

6t(10 − t) , 0 < t < 10.

1000

(a) Draw a graph of this density function.

0.10 0.05 0.00

6 * x * (10 − x)/1000

0.15

curve(6 * x * (10 - x) / 1000, from = 0, to = 10)

0

2

4

6

8

x (b) Find the probability that the median time is between 5 and 7 minutes.

0.284 (c) Find the cdf F (t) for all values of t.

30t2 − 2 ∗ t3 , 0 < t < 10 1000 (d) Using the cdf you found in part c, find the probability the median time is over 6 minutes.

F (t) =

61

10

1 − F (6) = .352 (e) Find the 75% percentile of your median waiting time.

Solve F (t) = .75 — get t = 6.7

Exercise 4. The Sum of Two Spins Suppose you spin two spinners, where the location of the arrow for each spinner is equally likely to fall between 0 and 10.

If you let S be the sum of the two spins, it can be shown that the density function of S is given by ( s/100, 0 < s ≤ 10 f (s) = (20 − s)/100, 10 < s ≤ 20, and shown by the figure below. (a) Check that this function satisfies the two properties of a probability density function. (b) Find the probability the sum of the two spins is smaller than 5.

Z 5 P (S < 5) = 0

s 52 ds = = 0.125 100 200

(c) Find the cdf function F . ( s2 /200, 0 < s ≤ 10 F (s) = 1 − (20 − s)2 /200, 10 < s ≤ 20, (d) Using the cdf function, find the probability the sum of spins falls between 8 and 12.

F (12) − F (8) = 0.36 (e) Using the cdf function, find the probability the sum of spins exceeds 12.

1 − F (12) = 0.32

Exercise 5. Salaries for Professional Basketball Players Let X denote the salary (in millions of dollars) of a professional basketball player. A reasonable density function for X is given by 0.15 f (x) = 1.3 , x ≥ 0.1 x shown by the figure below. (a) What proportion of basketball players earn more than 1 million dollars?

P(X > 1) = .503 (b) What proportion of players earn between 1 and 2 million dollars?

P(1 < X < 2) = .093 (c) Find the cdf function.

F (x) =

.15 , x ≥ 0.1 .3(1/.1. 3 − 1/x. 3) 62

(d) Using the cdf function, find the probability a player earns less than one-half a million dollars.

F (0.5) (e) Find the “average” salary of a NBA player.

Find the median — around x = 1

Exercise 6. Grading on a Curve Suppose the grades on a math test are distributed according to the curve. f (x) =

x , 0 < x < 100.

5000

(a) Draw a graph of this density curve.

0.010 0.000

0.005

x/5000

0.015

0.020

curve(x / 5000, 0, 100)

0

20

40

60

80

100

x (b) Find the mean grade on this test.

E(X) = 66.67 (c) What proportion of students who take this test get a grade of 90 or higher?

1 - F(90) = .19 (d) What proportion of students get a C grade, where C is defined to be between 70 and 80?

F(80) - F(70) = .15 (e) Is this test harder or easier than the test grades in your statistics class? Explain.

63

Exercise 7. Time to Clean Your Room Suppose the time that it takes you to clean your room (in hours) is a random variable X with the cdf function given below. A graph of the cdf is also shown.   x<0 0, F (x) = 0.75(2x3 /3 − x4 /4) 0 ≤ x ≤ 2   1, x>2 (a) Find the probability you can clean your room in under one hour.

F(1) = (3/4) (2/3 - 1/4) = .3125 (b) Find the probability it takes you over one and a half hour to clean your room?

1 - F(1.5) = .175 (c) Using the graph, find a value M such that it is equally likely that X is smaller than M and X is larger than M . [Hint: M is the 50th percentile of X.] Solve F(x) = .5, get x = 1.23

Exercise 8. Time to Complete a Race Suppose a group of children are running a race. The times (in minutes) that the children complete the race can be described by the density function f (x) =

4 + (x − 3)2 , 3 < x < 6.

21

(a) Graph this density function.

0.4 0.2 0.0

(4 + (x − 3)^2)/21

0.6

curve((4 + (x - 3)^2)/21, from=3, to=6, ylim=c(0, 0.7))

3.0

3.5

4.0

4.5 x

64

5.0

5.5

6.0

(b) Looking at your graph, is it more common to have a slow time (near 6 minutes) or a fast time (near 3 minutes)?

More common to have a slow time. (c) Find the probability a child completes the race in under 4 minutes.

Z 4 P (X < 4) = 0

4 + (x − 3)2 dx = 0.206 21

(d) Find the probability that a child’s time exceeds 5 1/2 minutes.

Z 6

4 + (x − 3)2 dx = 0.276 21 5.5

P (X > 5.5) = (e) Find the median running time.

Solve the following equation for M:

Z M P (X < M ) = 0

4 + (x − 3)2 dx = 0.5 21

Numerically, I found M = 4.98.

Exercise 9. Spinning a Random Spinner Suppose you flip a coin. If the coin lands heads, you spin a spinner that is equally likely to fall at any point in the interval (0, 4). If the coin lands tails, you spin a different spinner that lands at any point in the interval (2, 6). If X denotes your spin, the density function for X is graphed below. (a) Check that this graphed function is indeed a probability density. (b) Find the probability that X is greater than 5.

P(X > 5) = .125 (c) Find the probability that X falls between 1 and 3.

P(1 < X < 3) = .125 + .25 = .375

Exercise 10. Lifetimes of Light Bulbs Suppose that a company is interested in the amount of time that a particular type of light bulb will last until it burns out. After sampling the lifetimes for a large group of light bulbs, it is decided that the lifetime X (in hours) is well-described by the exponential distribution of the form f (x) =

1 −x/100 e , x > 0.

100

The cdf for X is drawn below.

In addition, the cdf is computed for some values of X in the following table. x

F (x)

x

F (x)

0 30 60 90

0 0.2592 0.4512 0.5934

180 210 240 270

0.8347 0.8775 0.9093 0.9328

65

x

F (x)

x

F (x)

120 150

0.6988 0.7769

300

0.9502

(a) Find the probability that a lifetime of a bulb will be less than 90 hours.

F (90) = 0.5934 (b) Find the probability the lifetime is between 120 and 180 hours.

F (180) − F (120) = 0.8347 − 0.6988 = 0.1359 (c) From the table, approximate the median lifetime.

From table, the median lifetime is between 60 and 90. (d) Approximate the 95th percentile.

From table, 95th percentile is close to 300.

Exercise 11. Locations of Dart Throws Suppose you throw a dart at a circular target such that the dart is equally likely to land in any location on the target. The locations for a large number of dart throws are shown in the figure below.

Let X denote the distance of a throw from the bulls eye. It can be shown that the density function of X has the form x , 0 < x < 2.

2 (a) Find the probability your throw lands within a distance of 1 unit from the target. f (x) =

P(X < 1) = 1/4 (b) Find the probability your throw lands between .5 and 1.5 units from the target.

P(.5 < X < 1.5) = (2.25 - .25) / 4 = .5 (c) If you threw the dart many times at the target, find your average distance from the target.

E(X) = 8/6 = 1.33

Exercise 12. Heights of Men Suppose heights of American men are approximately Normally distributed with mean 70 inches and standard deviation 4 inches. (a) What proportion of men is between 68 and 74 inches? pnorm(74, 70, 4) - pnorm(68, 70, 4) ## [1] 0.5328072 (b) What proportion of men is taller than 6 feet?

1 - pnorm(72, 70, 4) ## [1] 0.3085375 (c) Find the 90th percentile of heights.

66

qnorm(0.9, 70, 4) ## [1] 75.12621

Exercise 13. Test Scores Test scores in a precalculus test are approximately Normally distributed with mean 75 and standard deviation 10. If you choose a student at random from this class (a) What is the probability he or she gets an A (over 90)?

1 - pnorm(90, 75, 10) ## [1] 0.0668072 (b) What is the probability he or she gets a C (between 70 and 80)? pnorm(80, 75, 10) - pnorm(70, 75, 10) ## [1] 0.3829249 (c) What is the letter grade of the lower quartile of the scores? qnorm(0.25, 75, 10) ## [1] 68.2551 The lower quartile is 68.25 which is a D.

Exercise 14. Body Temperatures The normal body temperature was measured for 130 subjects in an article published in the Journal of the American Medical Association. These body temperatures are approximately Normally distributed with mean µ = 98.2 degrees and standard deviation σ= 0.73. (a) Most people believe that the mean body temperature of healthy individuals is 98.6 degrees, but actually the mean body temperature is smaller than 98.6. What proportion of healthy individuals have body temperatures smaller than 98.6? pnorm(98.6, 98.2, 0.73) ## [1] 0.7081352 (b) Suppose a person has a body temperature of 96 degrees. What is the probability of having a temperature less than or equal to 96 degrees? Based on this computation, would you say that a temperature of 96 degrees is unusual? Why? pnorm(96, 98.2, 0.73) ## [1] 0.00129042 Since this probability is very small, it is a rare event. (c) Suppose that a doctor diagnoses a person as sick if his or her body temperature is above the 95th percentile of the temperature of “healthy” individuals. Find this body temperature that will give a sick diagnosis. qnorm(0.95, 98.2, 0.73) ## [1] 99.40074

67

Exercise 15. Baseball Batting Averages Batting averages of baseball players can be well approximated by a Normal curve. The figure below displays the batting averages of players during the 2003 baseball season with at least 300 at-bats (opportunities to hit). The mean and standard deviation of the matching Normal curve shown in the figure are µ = 0.274 and σ = 0.027, respectively. (a) If you choose a baseball player at random, find the probability his batting average is over 0.300. (This is a useful benchmark for a “good” batting average.)

1 - pnorm(.300, .274, 0.027) ## [1] 0.1677831 (b) Find the probability this player has a batting average between 0.200 and 0.250. pnorm(0.250, 0.274, 0.027) - pnorm(0.200, 0.274, 0.027) ## [1] 0.1839664 (c) A baseball player is said to hit below the Mendoza line (named for weak-hitting baseball player Minnie Mendoza) if his batting average is under 0.200. Given our model, find the probability that a player hits below the Mendoza line. pnorm(0.200, 0.274, 0.027) ## [1] 0.003065043 (d) Suppose that a player has an incentive clause in his contract that states that he will earn an additional $1 million if his batting average is in the top 15%. How well does the player have to hit to get this additional salary? qnorm(0.85, 0.274, 0.027) ## [1] 0.3019837

Exercise 16. Emergency Calls Suppose that the AAA reports that the average time it takes to respond to an emergency call on the highway is 25 minutes. Assume that the times to respond to emergency calls are approximately Normally distributed with mean 25 minutes and standard deviation 4 minutes. (a) If your car gets stuck on a highway and you call the AAA for help, find the probability that it will take longer than 30 minutes to get help.

1 - pnorm(30, 25, 4) ## [1] 0.1056498 (b) Find the probability that you’ll wait between 20 and 30 minutes for help. pnorm(30, 25, 4) - pnorm(20, 25, 4) ## [1] 0.7887005 (c) Find a time such that you are 90% sure that the wait will be smaller than this number. qnorm(0.9, 25, 4) ## [1] 30.12621

68

Exercise 17. Buying a Battery for your iPod Suppose you need to buy a new battery for your iPod. Brand A lasts an average of 11 hours and Brand B lasts an average of 12 hours. You plan on using your iPod for eight hours on a trip and you want to choose the battery that is most likely to last 8 hours (that is, have a life that is least as long as 8 hours). (a) Based on this information, can you decide which battery to purchase? Why or why not?

Need to know more than just the average battery lifetime. (b) Suppose that the battery lives for Brand A are Normally distributed with mean 11 hours and standard deviation 1.5 hours, and the battery lives for Brand B are Normally distributed with mean 12 hours and standard 2 hours. Compute the probability that each battery will last at least 8 hours.

(Prob_A <- 1 - pnorm(8, 11, 1.5)) ## [1] 0.9772499 (Prob_B <- 1 - pnorm(8, 12, 2)) ## [1] 0.9772499 (c) On the basis of this calculation in part (b), which battery should you purchase?

The two batteries have the same chance of lasting more than 8 hours.

Exercise 18. Lengths of Pregnancies It is known that the lengths of completed pregnancies are approximately Normally distributed with mean 266 days and standard deviation 16 days. (a) What is the probability a pregnancy will last more than 270 days?

1 - pnorm(270, 266, 16) ## [1] 0.4012937 (b) Find an interval that will contain the middle 50% of the pregnancy lengths. qnorm(c(0.25, 0.75), 266, 16) ## [1] 255.2082 276.7918 (c) Suppose a doctor wishes to tell a mother that he is 90% confident that the pregnancy will be shorter than x days. Find the value of x. qnorm(.90, 266, 16) ## [1] 286.5048

Exercise 19. Attendances at Baseball Games The following figure displays the attendance for all of the home games of the Cleveland Indians for the 2006 baseball season. This distribution can be approximated by a Normal curve with mean µ = 24,667 and standard deviation σ = 6144.

Consider the attendance for one randomly selected game during the 2006 season. (a) Find the probability the attendance exceeds 30,000.

69

1 - pnorm(30000, 24667, 6144) ## [1] 0.1926968 (b) Find the probability the attendance is between 20,000 and 30,000. pnorm(30000, 24667, 6144) - pnorm(20000, 24667, 6144) ## [1] 0.5835572 (c) Suppose that the attendance at one game in the following season is 12,000. Based on the Normal curve, compute the probability that the attendance is at most 12,000. Based on this computation, is this attendance unusual? Why? pnorm(12000, 24667, 6144) ## [1] 0.01961881 Yes, it is unusual.

Exercise 20. Coin Flipping Suppose you flip a fair coin 1000 times. (a) How many heads do you expect to get? np = 1000 ∗ 0.5 = 500 (b) Find the probability that the number of heads is between 480 and 520.

X, the number of heads is approximately normal with mean 500 and standard deviation 15.8113883 So probability is approximated by pnorm(520, 500, 15.8) - pnorm(480, 500, 15.8) ## [1] 0.7944235 (c) Suppose your friend gets 550 heads. What is the probability of getting at least 550 heads? Do you believe that your friend’s coin really was fair? Explain.

1 - pnorm(550, 500, 15.8) ## [1] 0.0007765963 Since this probability is so small, I don’t believe coin is really fair.

Exercise 21. Use of Online Banking Services Suppose that a newspaper article claims that 80% of adults currently use online banking services. You wonder if the proportion of adults who use online banking services in your community, p, is actually this large. You take a sample of 100 adults and 70 tell you they use online banking. (a) If the newspaper article is accurate, find the probability that 70 or fewer of your sample would use on-line banking. µ = np = 100 ∗ .8 = 80 p σ = n ∗ p ∗ (1 − p) = 4

70

So probability is approximated by pnorm(70, 80, 4) ## [1] 0.006209665 (b) Based on your computation, is there sufficient evidence to suggest that less than 80% of your community use online banking services? Explain.

Yes, the probability of this outcome is small if really p = 0.8.

Exercise 22. Time to Complete a Race Suppose a group of children are running a race. The times (in minutes) that the children complete the race can be described by the density function f (x) =

4 + (x − 3)2 , 3 < x < 6.

21

A graph of this density is shown below. The mean and standard deviation of this density are given by 4.83 and 0.84 minutes, respectively. (a) Suppose 25 students run this race and you find the mean completion time. Find the probability that the mean time exceeds 5 minutes. √ The sample mean ȳ will be normal with mean 4.83 and standard deviation 0.84/ 25 = 0.168. So one can approximate the probability by 1 - pnorm(5, 4.83, 0.168) ## [1] 0.1557918 (b) Find an interval that you are 90% confident contains the mean completion time for the 25 students. qnorm(c(0.05, 0.95), 4.84, 0.168) ## [1] 4.563665 5.116335

Exercise 23. Snowfall Accumulation Your local meteorologist has collected data on snowfall for the past 100 years. Based on these data, you are told that the amount of snowfall in January is approximately Normally distributed with mean 15 inches and standard deviation 4 inches. (a) Find the probability you get more than 20 inches of snow this year.

1 - pnorm(20, 15, 4) ## [1] 0.1056498 (b) In the next ten years, find the probability that the average snowfall (for these ten years) will exceed 20 inches.

The sample mean X̄ is approximately normal with mean 15 and standard deviation 4 / sqrt(10) = 1.26. So the probability average snowfall exceeds 20 inches is 1 - pnorm(20, 15, 1.26) ## [1] 3.620058e-05

71

Exercise 24. Total Waiting Time at a Bank You are waiting to be served at your bank. From past experience, you know that your time to be served has a Uniform distribution between 0 and 10 minutes. (a) Find the mean and standard deviation of your waiting time.

(I am using special formulas, but you can find these by integration.) p µ = 5, σ = (b − a)2 /12 = 2.886751 (b) The Central Limit Theorem can be also stated in terms of the sum of random variables. If the random variables X1 , ..., Xn represent a random sample drawn Pnfrom a population with mean µ and standard deviation σ, then the sum of random variables S = i=1 Xi , for √ large sample size n, will be approximately Normally distributed with mean nµ and standard deviation nσ . Suppose you wait at the bank for 30 days. Use this version of the Central Limit Theorem to find the probability that your total waiting time will exceed three hours. √ S is normal with mean 30 × 5 = 150 and standard deviation 30(2.886751) = 15.81 So probability is 1 - pnorm(180, 150, 15.81) ## [1] 0.0288788

Exercise 25. Total Errors in Check Recording Suppose you record the amount of a written check to the nearest dollar. It is reasonable to assume that the error between the actual check amount and the written amount has a Uniform distribution between −0.50 and +0.50. (a) Find the mean and standard deviation of one error. p µ = 0, σ = 1/12 = .2887 (b) Suppose you write 100 checks in a single month and S denotes the total error in recording these checks.

Find the probability that S is smaller than $5. (Use the version of the Central Limit Theorem described in Exercise 5.) √ S has mean 100(0) = 0 and standard deviation 100 × .2887 = 2.887 and so the probability is pnorm(5, 0, 2.887) ## [1] 0.9583545 (c) Find an interval of the form (−c, c) so that P (−c < S < c) = 0.95. qnorm(c(0.025, 0.975), 0, 2.887) ## [1] -5.658416

5.658416

Exercise 26. Distribution of Measurements Suppose that a group of measurements is approximately Normally distributed with mean µ and standard deviation σ. (a) Find the probability that a measurement falls within one standard deviation of the mean. pnorm(1) - pnorm(-1)

72

## [1] 0.6826895 (b) Is it likely that you collect a measurement that is larger than µ + 3σ ? Explain.

1 - pnorm(3) ## [1] 0.001349898 No it is unlikely that you collect a measurement larger than µ + 3σ. (c) Find an interval that contains the middle 50% of the measurements. qnorm(c(0.25, 0.75)) ## [1] -0.6744898

0.6744898

The interval will be (µ − 0.674σ, µ + 0.674σ)

Exercise 27. Salaries of Professional Football Players Suppose you learn that the mean salary of all professional football players this season is 7 million dollars with a standard deviation of 2 million dollars. (a) Do you believe that the distribution of salaries is approximately Normally distributed? If your answer is no, sketch a plausible distribution for the salaries.

I would think that the distribution of incomes would be strongly right-skewed. (b) From your graph, find an approximate probability that a salary is smaller than $6 million. (c) Suppose you take a random sample of 30 salaries. Find the probability that the mean salary for this sample is smaller than $6 million.

Here we apply the Central Limit Theorem. The mean salary would be approximately normal with mean 7 and standard deviation 2 / sqrt(30) = 0.365 So the probability the mean salary is smaller than 6 is approximated by pnorm(6, 7, 0.365) ## [1] 0.003074521

Exercise 28. Weights of Candies In the candy bowl example, the probability distribution of the candy weight X is given in the following table.

Table 5.1. Weight and proportion of 5 types of candies.

fruity square milk maid jelly nougat caramel candy bars

x

P (X = x)

2 5 8 14 18

0.15 0.35 0.20 0.15 0.15

Verify by calculation that the mean and standard deviation of X are given by µ = 8.4500 and σ = 5.3617, respectively.

73

x <- c(2, 5, 8, 14, 18) p <- c(.15, .35, .20, .15, .15) (mu <- sum(x * p)) ## [1] 8.45 (sigma <- sqrt(sum((x - mu) ^ 2 * p))) ## [1] 5.36167

Exercise 29. Sleeping Times Suppose sleeping times of college students are approximately Normally distributed. You are told that 25% of students sleep less than 6.5 hours and 25% of students sleep longer than 8 hours. Given this information, determine the mean and standard deviation of the Normal distribution.

For any normal curve, the 25th and 75th percentiles are given by µ − .674σ and µ + .674σ. By solving the equations µ − .674σ = 6.5, µ + .674σ = 8 one obtains µ = 7.25 and σ = 1.11

Exercise 30. A Continuous Spinner Suppose you spin a spinner where all values from 0 to 100 are equally likely. (a) Write down the density function for X, one spin from this spinner. f (x) = 1/100, 0 < x < 100 (b) Use the following command to simulate 1000 values from this Uniform distribution and store the values in the vector spinner: spinner <- runif(1000, min = 0, max = 100) (c) Construct a histogram of the simulated spins. hist(spinner)

74

80 60 40 0

20

Frequency

100

Histogram of spinner

0

20

40

60

80

100

spinner (d) Use the simulated spins to approximate the probability P (X > 70). sum(spinner > 70)

/

1000

## [1] 0.322

Exercise 31. Simulating a Normal Distribution Suppose monthly snowfalls in Rochester, New York are Normally distributed with mean 25 inches and standard deviation 10 inches. (a) Using the rnorm() function, simulate snowfalls for 1000 hypothetical months in Rochester. snowfall <- rnorm(1000, 25, 10) (b) Construct a graph of these snowfall amounts. hist(snowfall)

75

100 0

50

Frequency

150

200

Histogram of snowfall

0

10

20

30

40

50

60

snowfall (c) Approximate from the simulated values the probability that a snowfall falls in the interval (20, 30).

Compare your answer with the exact probability found using the pnorm() function. sum(snowfall > 20 & snowfall < 40) / 1000 ## [1] 0.617 Exact answer: pnorm(40, 25, 10) - pnorm(20, 25, 10) ## [1] 0.6246553 (d) From the simulated values, find an interval that contains the middle 80% of the snowfalls. Compare your answer with the exact interval found using the qnorm() function. quantile(snowfall, c(0.1, 0.9)) ## 10% 90% ## 11.60861 38.20848 Compare with exact answer: qnorm(c(.1, .9), 25, 10) ## [1] 12.18448 37.81552

Exercise 32. Waiting for a Bus In the example, the amount of time that one waits for a bus has a Uniform distribution from 0 to 10 minutes.

One waits for a bus on Monday, Wednesday, and Friday and records the minimum of the three waiting times. (a) Write a program to simulate 1000 values of this minimum waiting time.

76

monday <- runif(1000, 0, 10) wednesday <- runif(1000, 0, 10) friday <- runif(1000, 0, 10) min_waiting_time <- pmin(monday, wednesday, friday) (b) One can show that the minimum waiting time Y has density given by f (y) =

3 (10 − y)2 , 0 < y < 10.

1000

Compare a histogram of simulated values from (a) with this density function to confirm that you have indeed simulated from the correct distribution. hist(min_waiting_time, freq = FALSE ) curve(3 / 1000 * (10 - x) ^ 2, add = TRUE)

0.10 0.00

Density

0.20

Histogram of min_waiting_time

0

2

4

6

8

10

min_waiting_time Exercise 33. Weights of Candies (continued) Suppose one takes a sample of 10 candies from the distribution of candy weights shown in Exercise 28. (a) Write a function to take a random sample of 10 candies from the bowl and return the sample mean X̄. x <- c(2, 5, 8, 14, 18) p <- c(.15, .35, .20, .15, .15) one_sample_mean <- function(n){ candies <- sample(x, prob = p, size = n, replace = TRUE) mean(candies) } one_sample_mean(10)

77

## [1] 9.1 (b) Use the replicate() function to repeat this process for 1000 iterations – store the sample means in the vector xbars. xbars <- replicate(1000, one_sample_mean(10)) (c) Construct a histogram of the sample means and comment on its shape. Also find the mean and standard deviation of the sample means. hist(xbars)

150 100 0

50

Frequency

200

250

Histogram of xbars

4

6

8

10

12

14

xbars mean and sd of sample means mean(xbars); sd(xbars) ## [1] 8.4364 ## [1] 1.69999 (d) Repeat this exercise using samples of size n = 25. Are there any changes in the mean and standard deviation of the sample means? xbars <- replicate(1000, one_sample_mean(25)) mean(xbars); sd(xbars) ## [1] 8.45272 ## [1] 1.088605 the distribution of the sample means with a larger sample size have a similar mean, smaller standard deviation

78

Exercise 34. Spins and the Central Limit Theorem Suppose you are spinning a spinner with equally likely outcomes 1, 2, 3, 4, 5. X represents a single spin from this spinner. (a) Find the mean µ and standard deviation σ of X. x <- 1:5 p <- rep(1, 5) / 5 (mu <- sum(x * p)) ## [1] 3 (sd <- sqrt(sum((x - mu) ^ 2 * p))) ## [1] 1.414214 (b) Write a function to simulate 10 spins from this spinner and compute the sample mean X̄. one_sample_mean <- function(n){ spins <- sample(x, prob = p, size = n, replace = TRUE) mean(spins) } one_sample_mean(10) ## [1] 3 (c) Simulate 1000 samples of 10 spins, obtaining a vector of sample means. sample_means <- replicate(1000, one_sample_mean(10)) (d) Construct a histogram of the sample means and comment on its shape. Also find the mean and standard deviation of the sample means. hist(sample_means)

79

100 0

50

Frequency

150

Histogram of sample_means

1.5

2.0

2.5

3.0

3.5

4.0

4.5

sample_means (mu_xbar <- mean(sample_means)) ## [1] 2.9989 (sd_xbar <- sd(sample_means)) ## [1] 0.426768 (e) Check your calculations in part (d) by finding the exact mean and standard deviation of the sample mean X̄. (exact_mu_xbar <- mu) ## [1] 3 (exact_sd_xbar <- sd / sqrt(10)) ## [1] 0.4472136

Chapter 6 - Joint Probability Distributions Exercise 1. Coin Flips Suppose you flip a coin three times with eight equally likely outcomes HHH, HHT, ..., T T T . Let X denote the number of heads in the first two flips and Y the number of heads in the last two flips. (a) Find the joint probability mass function (pmf) of X and Y and put your answers in the following table. y x 0

0 1/8

1 1/8 80

2 0

y 1 2

1/8 0

2/8 1/8

X

P(X)

0 1 2

2/8 4/8 2/8

Y

P(Y)

0 1 2

2/8 4/8 2/8

1/8 1/8

(b) Find P (X > Y ).

P(X > Y) = P((1, 0), (2, 1)) = 2/8 (c) Find the marginal pmf’s of X and Y .

(d) Find the conditional pmf of X given Y = 1.

Focus on the column where Y = 1: X

P(X given Y = 1)

0 1 2

1/4 2/4 1/4

Exercise 2. Selecting Numbers Suppose you select two numbers without replacement from the set {1, 2, 3, 4, 5}. Let X denote the smaller of the two numbers and Y denote the larger of the two numbers. (a) Find the joint probability mass function of X and Y . y x 1 2 3 4

2 1/10 0 0 0

3 1/10 1/10 0 0

4 1/10 1/10 1/10 0

(b) Find the marginal pmf’s of X and Y .

X

P(X)

1 2 3

4/10 3/10 2/10 81

5 1/10 1/10 1/10 1/10

X

P(X)

4

1/10

Y

P(Y)

1 2 3 4

1/10 2/10 3/10 4/10

(c) Are X and Y independent? If not, explain why.

No, for example P(X = 3, Y = 2) is not equal to P(X = 3)P(Y = 2) (d) Find P (Y = 3 | X = 2).

P(Y = 3, X = 2) / P(X = 2) = (1/10) / (3/10) = 1/3

Exercise 3. Die Rolls You roll a die 4 times and record O, the number of ones, and T the number of twos rolled. (a) Construct the joint pmf of O and T .

O T – 0 1 2 3 4

0 – 0.196 0.196 0.074 0.012 0.001

1 – 0.196 0.148 0.037 0.003 0

2 – 0.074 0.037 0.005 0 0

(b) Find the probability P (O = T ).

0.196 + 0.148 + 0.005 = 0.349 (c) Find the conditional pmf of T given O = 1.

T

Prob

0 1 2 3 4

0.510 0.385 0.096 0.008 0

(d) Compute P (T > 0 | O = 1).

P(T > 0 | O = 1) = 1 - 0.510 = 0.490

82

3 – 0.012 0.003 0 0 0

4 – 0.001 0 0 0 0

Exercise 4. Choosing Balls Suppose you have a box with 3 red and 2 black balls. You first roll a die – if the roll is 1, 2, you sample 3 balls without replacement from the box. If you roll is 3 or higher, you sample 2 balls with replacement from the box. Let X denote the number of balls you sample and Y the number of red balls selected. (a) Find the joint pmf of X and Y .

–

–

Y

–

–

– X=2 X=3

0 1/30 0

1 6/30 6/30

2 3/30 12/30

3 0 2/30

(b) Find the probability P (X = Y ).

5/30 (c) Find the marginal pmf of Y .

Y

P(Y)

0 1 2 3

1/30 12/30 15/30 2/30

(d) Find the conditional pmf of X given Y = 2.

X

P(X given Y = 2)

2 3

3/15 12/15

Exercise 5. Baseball Hitting Suppose a player is equally likely to have 4, 5, or 6 at-bats (opportunities) in a baseball game. If N is the number of opportunities, then assume that X, the number of hits, is Binomial with probability p = 0.3 and sample size N . (a) Find the joint pmf of N and X. X

0

1

2

3

4

5

6

N=4 N=5 N=6

.080 .056 .039

.137 .120 .101

.088 .103 .108

.025 .044 .062

.003 .009 .020

0 .001 .003

0 0 0

(b) Find the marginal pmf of X. X

P(X)

0 1 2

.175 .358 .299 83

X

P(X)

3 4 5 6

.131 .032 .004 .000

(c) Find the conditional pmf of N given X = 2.

N

P(N)

4 5 6

.294 .344 .361

(d) If the player gets 3 hits, what is the probability he had exactly 5 at-bats? want P(N = 5 | X = 3) = .044 / .131 = .336

Exercise 6. Multinomial Density Suppose a box contains 4 red, 3 black, and 3 green balls. You sample eight balls with replacement from the box and let R denote the number of red and B the number of black balls selected. (a) Explain why this is a Multinomial experiment and given values of the parameters of the Multinomial distribution for (R, B).

This is a multinomial experiment where n = 8 and the probabilities of red, black, green on each selection are 4/10, 3/10, 3/10. (b) Compute P (R = 3, B = 2). factorial(8)/(factorial(3) * factorial(2) * factorial(3)) * (0.4) ^ 3 * (0.3) ^ 2 * (0.3) ^ 3 ## [1] 0.0870912 Or use the dmultinom() function: dmultinom(c(3, 2, 3), size = 8, prob = c(.4, .3, .3)) ## [1] 0.0870912 (c) Compute the probability that you sample more red balls than black balls. want P(X > Y) Need to sum the probs over the outcomes (8, 0), (7, 0), (7, 1), (6, 0), (6, 1), (6, 2), (5, 0), (5, 1), (5, 2), (5, 3), (4, 0), (4, 1), (4, 2), (4, 3), (3, 0), (3, 1), (3, 2), (2, 0), (2, 1), (1, 0). x <- c(8, 7, 7, 6, 6, 6, 5, 5, 5, 5, 4, 4, 4, 4, 3, 3, 3, 2, 2, 1) y <- c(0, 0, 1, 0, 1, 2, 0, 1, 2, 3, 0, 1, 2, 3, 0, 1, 2, 0, 1, 0) Prob <- 0 for(j in 1:length(x)){ Prob <- Prob + dmultinom(c(x[j], y[j], 8 - x[j] - y[j]), size = 8, prob = c(.4, .3, .3))

84

} Prob ## [1] 0.5543056 (d) Find the marginal distribution of B. B will be binomial with sample size 8 and probability of success 3/10 (e) If you are given that you sampled B = 4 balls, find the probability that you sampled at most 2 red balls.

If we know we sampled 4 black balls, then the number of red balls R is binomial with sample size 4 and success probability 4/7. pbinom(2, size = 4, prob = 4/7) ## [1] 0.573511

Exercise 7. Joint Density Let X and Y have the joint density f (x, y) = ky, 0 < x < 2, 0 < y < 2. (a) Find the value of k so that f () is a pdf.

Z 2Z 2 k y dx dy = 4k = 1 0

0

So k = 1/4 (b) Find the marginal density of X. Z 2 f (x) = 0

y 1 dy = , 0 < x < 2 4 2

(c) Find P (Y > X).

Z 2Z y P (Y > X) = 0

0

2 y dx dy = 4 3

(d) Find the conditional density of Y given X = x for any value 0 < x < 2. f (y|x) =

f (x, y) y/4 y = = ,0 < y < 2 f (x) 1/2 2

Exercise 8. Joint Density Let X and Y have the joint density f (x, y) = x + y, 0 < x < 1, 0 < y < 1. (a) Check that f is indeed a valid pdf. If it is not, correct the definition of f so it is valid.

85

One can confirm that

Z 1Z 1 x + y dx dy = 1 0

0

(b) Find the probability P (X > 0.5, Y < 0.5).

Z 0.5 Z 1 x + y dx dy =

P (X > 0.5, Y < 0.5) = 0

0.5

1 4

(c) Find the marginal density of X. Z 1

1 x + y dy = x + , 0 < x < 1 2 0

f (x) =

(d) Are X and Y independent? Answer by a suitable calculation.

One can show that f (x, y) is not a product of the marginal densities, so X and Y are not independent.

Exercise 9. Random Division Suppose one randomly chooses a values X on the interval (0, 2), and then random choosing a second point Y from 0 to X. (a) Find the joint density of X and Y . f (x, y) = f (x)f (y|x) =

11 , 0 < y < x < 2.

2x

(b) Are X and Y independent? Explain.

No, since the conditional density f (y|x) depends on the value of x. (c) Find the probability P (Y > 0.5).

Z 2 f (y) = y

1 1 dx = (log 2 − log y), 0 < y < 2 2x 2

(d) Find the probability P (X + Y > 2).

Z 2Z x 1

1 dydx = 1 − log(2) = 0.307 2x 2−x

Exercise 10. Choosing a Random Point in a Circle Suppose (X, Y ) denotes a random point selected over the unit circle. The joint pdf of (X, Y ) is given by ( C, x2 + y 2 ≤ 1; f (x, y) = 0, elsewhere. (a) Find the value of the constant C so f () is indeed a joint pdf.

The volume over the circle is Cπ, so C = π1 . (b) Find the marginal pdf of Y .

86

Z √1−y2 f (y) =

√

−

1 1 p dx = 2 1 − y 2 , −1 < y < 1 π 1−y 2 π

(c) Find the probability P (Y > 0.5) Z 1

1 p 2 1 − y 2 dy = 0.196 0.5 π

P (Y > 0.5) =

(d) Find the conditional pdf of X conditional on Y = 0.5. √ √ 1/π f (x|y) = 1 √ , − 1 − .25 < x < 1 − .25 2 π 2 1 − 0.5

Exercise 11. A Random Meeting Suppose John and Jill independently arrive at an airport at a random between 3 and 4 pm one afternoon.

Let X and Y denote respectively the number of minutes past 3 pm that John and Jill arrive. (a) Find the joint pdf of X and Y .

1 0 < x, y < 60 3600 (b) Find the probability that John arrives later than Jill. f (x, y) =

Z 60 Z 60 P (X > Y ) = 0

y

1 dxdy = 0.5 3600

(c) Find the probability that John and Jill meet within 10 minutes of each other.

Using some geometric reasoning . . .

P (|X − Y | < 10) =

1 6

Exercise 12. Defects in Fabric Suppose the number of defects per yard in a fabric X is assumed to have a Poisson distribution with mean λ.

That is, the conditional density of X given λ has the form f (x | λ) =

e−λ λx , x = 0, 1, 2, ... x!

The parameter λ is assumed to be uniformly distributed over the values 0.5, 1, 1.5, and 2. (a) Write down the joint pmf of X and λ. e−λ λx 1 , x = 0, 1, 2, ..., λ = 0.5, 1, 1.5, 2 x! 4 (b) Find the probability that the number of defects X is equal to 0. f (x, λ) =

87

Z ∞ P (X = 0) = 0

e−λ λ0 1 1 dλ = 0! 4 4

(c) Find the conditional pmf of λ if you know that X = 0. f (λ|X = 0) =

f (0, λ) = e−λ , λ > 0 P (X = 0)

Exercise 13. Defects in Fabric (continued) Again we assume the number of defects per yard in a fabric X given λ has a Poisson distribution with mean λ. But now we assume λ is continuous-valued with the exponential density g(λ) = exp(−λ), λ > 0. (a) Write down the joint density of X and λ. e−λ λx exp(−λ), x = 0, 1, 2, ..., λ > 0 x! (b) Find the marginal density of X. [Hint: it may be helpful to use the integral identity Z ∞ b! exp(−aλ)λb dλ = b+1 , a 0 f (x, λ) =

where b is a nonnegative integer.]

Z ∞ f (x) = 0

e−2λ λx 1 dλ = x+1 , x = 0, 1, 2, ... x!

2

(c) Find the probability that the number of defects X is equal to 0.

1 2 (d) Find the conditional density of λ if you know that X = 0.

P (X = 0) =

f (λ|X = 0) =

f (λ, 0) = 2e−2λ , λ > 0 P (X = 0)

Exercise 14. Flipping a Random Coin Suppose you plan flipping a coin twice where the probability p of heads has the density function f (p) = 6p(1 − p), 0 < p < 1.

Let Y denote the number of heads of this “random” coin. Y given a value of p is Binomial with n = 2 and probability of success p. (a) Write down the joint density of Y and p.

2 y f (y, p) = p (1 − p)2−y 6p(1 − p), 0 < p < 1, x = 0, 1, 2, y (b) Find P (Y = 2).

88

Z 1 P (Y = 2) =

p2 (6p(1 − p))dp = 0.3

0

(c) If Y = 2, then find the probability that p is greater than 0.5.

f (p|Y = 2) =

6p3 (1 − p) = 20p3 (1 − p), 0 < p < 1 0.3 Z 1

P (p > 0.5|Y = 2) =

20p3 (1 − p)dp = 0.8125

0.5

Exercise 15. Passengers on An Airport Limousine An airport limousine can accommodate up to four passengers on any one trip. The company will accept a maximum of six reservations for a trip, and a passenger must have a reservation. From previous records, 30% of all those making reservations do not appear for the trip. Answer the following questions, assuming independence whenever appropriate. (a) If six reservations are made, what is the probability that at least one individual with a reservation cannot be accommodated on the trip?

Let D denote the number who reserve and don’t appear for the trip. D is binomial with sample size 6 and probability of success 0.30.

Want P (D ≤ 1) – use R pbinom(1, size = 6, prob = 0.3) ## [1] 0.420175 (b) If six reservations are made, what is the expected number of available places when the limousine departs?

The number of expected “no shows" is np = 6 * 0.3 = 1.8, so one expects 6 - 1.8 = 4.2 to show up for their reservation. So the expected number of available places is 4 − 4.2 = −0.2. Expect the limo to be overbooked a little. (c) Suppose the probability distribution of the number of reservations made is given in the following table.

Number of reservations

3

4

5

6

Probability

0.13

0.18

0.35

0.34

Let X denote the number of passengers on a randomly selected trip. Obtain the probability mass function of X. The number of passengers is actually X = min(S, 4) where S is the number of passengers with reservations that show up. Note that S is binomial(N, p) where N is the number of reservations and p = 0.7.

This gives the distributions of X for each number of reservations R, p(x|r).

Number of reservations X=0 X=1 X=2 X=3

3

4

5

6

0.027 0.189 0.441 0.343

0.008 0.076 0.265 0.412

0.002 0.028 0.132 0.309

0.001 0.010 0.060 0.185

89

Number of reservations

3

4

5

6

X=4

0

0.240

0.528

0.745

Combining these conditional distributions with the distribution of R, you get the following marginal distribution of X. x

0

1

2

3

4

p(x)

0.006

0.051

0.172

0.290

0.481

Exercise 16. Heights of Fathers and Sons It is well-know that heights of fathers and sons are positively association. In fact, if X represents the father’s height in inches and Y represents the son’s height, then the joint distribution of (X, Y ) can be approximated by a Bivariate Normal with means µX = µY = 69, σX = σY = 3 and correlation ρ = 0.4. (a) Are X and Y independent? Why or why not?

No, since the correlation between X and Y is nonzero. (b) Find the conditional density of the sons height if you know the father’s height is 70 inches.

Y Using formula from the text, [Y |X = x] is normal with mean µY + ρ σσX (x − µX ) and variance σY2 (1 − ρ2 ).

Compute the mean and standard deviation: (mu1 <- 69 + 0.4 * (70 - 69)) ## [1] 69.4 (sigma1 <- sqrt(3 ^ 2 * (1 - 0.4 ^ 2))) ## [1] 2.749545 (c) Using the result in part (b) to find P (Y > 72 | X = 70).

1 - pnorm(72, mu1, sigma1) ## [1] 0.1721736 (d) By simulating from the Bivariate Normal distribution, approximate the probability that the son will be more than one inch taller than his father. sim_binorm <- function(mx, my, sx, sy, r){ require(ProbBayes) require(dplyr) v <- matrix(c(sx ^ 2, r * sx * sy, r * sx * sy, sy ^ 2), 2, 2) as.data.frame(rmnorm(1000, mean = c(mx, my), varcov = v))} mx <- 69; my <- 69; sx <- 3; sy <- 3; r <- 0.4 sdata <- sim_binorm(mx, my, sx, sy, r) names(sdata) <- c("X", "Y") sdata %>% summarize(Prob = mean(Y > X + 1)) ## Prob ## 1 0.398

90

Exercise 17. Instruction and Students’ Scores Twenty-two children are given a reading comprehension test before and after receiving a particular instruction method. Assume students’ pre-instructional and post-instructional scores follow a Bivariate Normal distribution with: µpre = 47, µpost = 53, σpre = 13, σpost = 15 and ρ = 0.7. (a) Find the probability that a student’s post-instructional score exceeds 60.

This score is N(53, 15), so find probability using pnorm() function. pnorm(60, 53, 15) ## [1] 0.6796308 (b) Suppose one student’s pre-instructional score is 45, find the probability that this student’s postinstructional score exceeds 70. f (y|X = 45) is normal with mean 53 + 0.7 * 15 / 13 * (45 - 47) and standard deviation 15 * sqrt(1 - 0.7 ˆ 2) (mu1 <- 53 + 0.7 * 15

/ 13 * (45 - 47))

## [1] 51.38462 (sigma1 <- 15 * sqrt(1 - 0.7 ^ 2)) ## [1] 10.71214 Probability is 1 - pnorm(70, mu1, sigma1) ## [1] 0.04112449 (c) Find the probability that a student has increased the test score by at least 10 points.

Use simulation . . . sim_binorm <- function(mx, my, sx, sy, r){ require(ProbBayes) require(dplyr) v <- matrix(c(sx ^ 2, r * sx * sy, r * sx * sy, sy ^ 2), 2, 2) as.data.frame(rmnorm(1000, mean = c(mx, my), varcov = v))} mx <- 47; my <- 53; sx <- 13; sy <- 15; r <- 0.7 sdata <- sim_binorm(mx, my, sx, sy, r) names(sdata) <- c("X", "Y") sdata %>% summarize(Prob = mean(Y > X + 10)) ## Prob ## 1 0.356

Exercise 18. Shooting Free Throws Suppose a basketball player will take N free throw shots during a game where N has the following discrete distribution.

91

N

5

6

7

8

9

10

Probability

0.2

0.2

0.2

0.2

0.1

0.1

If the player takes N = n shots, then the number of makes Y is Binomial with sample size n and probability of success p = 0.7. (a) Find the probability the player takes 6 shots and makes 4 of them.

P (N = 6, Y = 4) = P (N = 6)P (Y = 4|N = 6) 0.2 * dbinom(4, size = 6, prob = 0.7) ## [1] 0.064827 (b) From the joint distribution of (N, Y ), find the most likely (n, y) pair.

N <- 5:10 X <- 0:10 out <- NULL for(j in 1:length(N)){ probs <- round(dbinom(X, size = N[j], prob = 0.7), 3) out <- cbind(out, probs) } rownames(out) <- paste("X =", 0:10) colnames(out) <- paste("N =", N) out ## N = 5 N = 6 N = 7 N = 8 N = 9 N = 10 ## X = 0 0.002 0.001 0.000 0.000 0.000 0.000 ## X = 1 0.028 0.010 0.004 0.001 0.000 0.000 ## X = 2 0.132 0.060 0.025 0.010 0.004 0.001 ## X = 3 0.309 0.185 0.097 0.047 0.021 0.009 ## X = 4 0.360 0.324 0.227 0.136 0.074 0.037 ## X = 5 0.168 0.303 0.318 0.254 0.172 0.103 ## X = 6 0.000 0.118 0.247 0.296 0.267 0.200 ## X = 7 0.000 0.000 0.082 0.198 0.267 0.267 ## X = 8 0.000 0.000 0.000 0.058 0.156 0.233 ## X = 9 0.000 0.000 0.000 0.000 0.040 0.121 ## X = 10 0.000 0.000 0.000 0.000 0.000 0.028 It appears the most likely (n, y) pair is (5, 4). (c) Find the conditional distribution of the number of shots N if he makes 4 shots. dist_N <- out["X = 4", ] (dist_N <- round(dist_N / sum(dist_N), 3)) ## ##

N = 5 0.311

N = 6 0.280

N = 7 0.196

N = 8 0.117

N = 9 N = 10 0.064 0.032

(d) Find the expectation E(N | Y = 4). sum(N * dist_N) ## [1] 6.439

92

Exercise 19. Flipping a Random Coin Suppose one selects a probability p Uniformly from the interval (0, 1), and then flips a coin 10 times, where the probability of heads is the probability p. Let X denote the observed number of heads. (a) Find the joint distribution of p and X.

f (p, x) =

10 x p (1 − p)n−x , 0 < p < 1, x = 0, ..., 10 x

(b) Use R to simulate a sample of size 1000 from the joint distribution of (p, X). p <- runif(1000) x <- rbinom(1000, size = 10, prob = p) (c) From inspecting a histogram of the simulated values of X, guess at the marginal distribution of X. library(ProbBayes) bar_plot(x)

100

Frequency

75

50

25

0 0.0

2.5

5.0

7.5

10.0

Y It appears that X has a uniform discrete distribution on 0, 1, . . . , 10.

Exercise 20. Simulating Multinomial Probabilities Revisit Exercise 6. (a) Write an R function to simulate 10 balls sampled from replacement from the special weighted box (4 red, 3 black, and 3 green balls).

93

sample(c("red", "red", "red", "red", "black", "black", "black", "green", "green", "green"), size = 10, replace = TRUE) ## [1] "red" ## [10] "red"

"black" "black" "red"

"black" "black" "red"

"black" "black"

(b) Use the replicate() function to simulate the Multinomial experiment in Exercise 6 for 5000 iterations, and approximate P (R = 3, B = 2). one_sim <- function(){ S <- sample(c("red", "red", "red", "red", "black", "black", "black", "green", "green", "green"), size = 10, replace = TRUE) c(n_red = sum(S == "red"), n_black = sum(S == "black")) } outcomes <- replicate(5000, one_sim()) sum(outcomes[1, ] == 3 & outcomes[2, ] == 2) / 5000 ## [1] 0.0336 (c) Use the 5000 simulated Multinomial experiments to approximate the probability that you sample more red balls than black balls, i.e. P (R > B). sum(outcomes[1, ] > outcomes[2, ]) / 5000 ## [1] 0.5802 (d) Conditional on B = 4, approximate the mean number of red balls will get sampled. Compare the approximated mean value to the exact mean. [Hint: Conditional on B = 4, the distribution of R is a Binomial distribution.] cond_sample <- outcomes[1, outcomes[2, ] == 4] mean(cond_sample) ## [1] 3.426978

Exercise 21. Simulating from a Beta-Binomial Distribution Consider a box of coins where the coin probabilities vary, and the probability of a selected coin lands heads, p, follows a Beta(2, 8) distribution. Jason then continues to flip this “random" coin 10 times, and is interested in the count of heads of the 10 flips, denoted by Y . (a) Write an R function to simulate 5000 samples of (p, y). [Hint: Use rbeta() and rbinom() functions accordingly.] p <- rbeta(5000, 2, 8) y <- rbinom(5000, size = 20, prob = p) (b) Approximate the probability that Jason observes 3 heads out of 10 flips, using the simulated 5000 samples. Compare the approximated probability to the exact probability. [Hint: Write out f (y) following the work in Section 6.6, and use R to calculate the exact probability.] mean(y

== 3)

## [1] 0.1418 94

Can get exact calculation using pbetap() function. pbetap(c(2, 8), 10, 3) ## [1] 0.1486068

Exercise 22. Shooting Free Throws (continued) Consider the free throws shooting in Exercise 18. (a) Write an R function to simulate 5000 samples of (n, y).

N <- 5:10 PN <- c(0.2, 0.2, 0.2, 0.2, 0.1, 0.1) Ns <- sample(N, prob=PN, size = 5000, replace = TRUE) ys <- rbinom(5000, size = Ns, prob = 0.7) Put simulated draws in a data frame. df <- data.frame(Ns, ys) head(df) ## Ns ys ## 1 9 7 ## 2 10 7 ## 3 8 5 ## 4 7 6 ## 5 7 2 ## 6 6 6 (b) From the 5000 samples, find the most likely (n, y) pair. Compare your result to Exercise 18 part (b). df %>% group_by(Ns, ys) %>% summarize(n = n()) %>% arrange(desc(n)) %>% head(1) ## # A tibble: 1 x 3 ## # Groups: Ns [1] ## Ns ys n ## <int> <int> <int> ## 1 5 4 344 (c) Approximate the expectation E(N | Y = 4), and compare your result to Exercise 18 part (d). df %>% filter(ys == 4) %>% summarize(M = mean(Ns)) ## M ## 1 6.330018

95

Chapter 7: Learning About a Binomial Probability Exercise 1. Laymen’s Prior in the Dining Preference Example Revisit Section 7.2.1 for the laymen’s prior in Equation (7.2) and the expert’s prior in Equation (7.3). Follow the example R code (functions data.frame(), mutate() and ggplot()) to obtain the Bayes table and graph of the laymen’s prior distribution. Compare the similarities and differences between the laymen’s prior and the expert’s prior. bayes_table <- data.frame(p = seq(.3, .8, by=.1), Prior = c(1, 1, 1, 1, 1, 1)) bayes_table ## p Prior ## 1 0.3 1 ## 2 0.4 1 ## 3 0.5 1 ## 4 0.6 1 ## 5 0.7 1 ## 6 0.8 1 bayes_table %>% mutate(Prior = Prior / sum(Prior)) -> bayes_table bayes_table ## p Prior ## 1 0.3 0.1666667 ## 2 0.4 0.1666667 ## 3 0.5 0.1666667 ## 4 0.6 0.1666667 ## 5 0.7 0.1666667 ## 6 0.8 0.1666667 ggplot(data=bayes_table, aes(x=p, y=Prior)) + geom_bar(stat="identity", fill="blue", width = 0.06)

96

0.15

Prior

0.10

0.05

0.00 0.4

0.6

0.8

p Both the laymen’s prior and the expert’s prior have assigned non-zero probabilities for p = 0.3, 0.4, 0.5, 0.6, 0.7, 0.8 and assigned zero probabitlies for all other possible values for p. While the laymen’s prior believes all 6 values of p are equally likely, the expert’s prior believes that the middle 2 values are twice as likely as the remaining 4 values.

Exercise 2. Inference for the Dining Preference (Discrete Priors) Revisit Section 7.2.5 where we show how to find the posterior probability that over half of the students prefer eating out on Friday. Find the following posterior probabilities. (Be careful about the end points.)

Use bayes_table to first obtain posterior probabitlies for each value of p. bayes_table <- data.frame(p = seq(.3, .8, by=.1), Prior = c(1, 1, 2, 2, 1, 1)) bayes_table %>% mutate(Prior = Prior / sum(Prior)) -> bayes_table bayes_table$Likelihood <- dbinom(12, size=20, prob=bayes_table$p) bayesian_crank(bayes_table) -> bayes_table bayes_table ## p Prior Likelihood Product Posterior ## 1 0.3 0.125 0.003859282 0.0004824102 0.004975901 ## 2 0.4 0.125 0.035497440 0.0044371799 0.045768032 ## 3 0.5 0.250 0.120134354 0.0300335884 0.309786454 ## 4 0.6 0.250 0.179705788 0.0449264469 0.463401326 ## 5 0.7 0.125 0.114396740 0.0142995925 0.147495530 ## 6 0.8 0.125 0.022160877 0.0027701096 0.028572757 (a) The probability that more than 60% of the students prefer eating out on Friday.

97

Use the function sum to calculate. sum(bayes_table$Posterior[bayes_table$p > 0.6]) ## [1] 0.6394696 (b) The probability that less than 40% of the students prefer eating out on Friday.

Use the function sum to calculate. sum(bayes_table$Posterior[bayes_table$p < 0.4]) ## [1] 0.004975901 (c) The probability that between 20% and 40% of the students prefer eating out on Friday.

Use the function sum to calculate. sum(bayes_table$Posterior[bayes_table$p >= 0.2 & bayes_table$p <= 0.4]) ## [1] 0.05074393 (d) No more than 50% of the students prefer eating out on Friday.

Use the function sum to calculate. sum(bayes_table$Posterior[bayes_table$p < 0.5]) ## [1] 0.05074393

Exercise 3. Another Dining Survey (Discrete Priors) Suppose the restaurant owner in the college town gives another survey to a different group of students. This time he gives the survey to 30 students – among these responses 10 of them say that Friday is their preferred day to eat out. Use the owner’s prior (restated below) to calculate the following posterior probabilities. p πe (p)

= {0.3, 0.4, 0.5, 0.6, 0.7, 0.8} =

(0.125, 0.125, 0.250, 0.250, 0.125, 0.125)

Use bayes_table to first obtain posterior probabitlies for each value of p. bayes_table <- data.frame(p = seq(.3, .8, by=.1), Prior = c(1, 1, 2, 2, 1, 1)) bayes_table %>% mutate(Prior = Prior / sum(Prior)) -> bayes_table bayes_table$Likelihood <- dbinom(10, size=30, prob=bayes_table$p) bayesian_crank(bayes_table) -> bayes_table bayes_table ## p Prior Likelihood Product Posterior ## 1 0.3 0.125 1.415617e-01 1.769521e-02 4.469407e-01 ## 2 0.4 0.125 1.151854e-01 1.439817e-02 3.636649e-01 ## 3 0.5 0.250 2.798160e-02 6.995400e-03 1.766878e-01 ## 4 0.6 0.250 1.997491e-03 4.993727e-04 1.261301e-02 ## 5 0.7 0.125 2.959225e-05 3.699031e-06 9.342907e-05 ## 6 0.8 0.125 3.382768e-08 4.228460e-09 1.068012e-07 (a) The probability that 30% of the students prefer eating out on Friday.

Use the function sum to calculate.

98

sum(bayes_table$Posterior[bayes_table$p == 0.3]) ## [1] 0.4469407 (b) The probability that more than half of the students prefer eating out on Friday.

Use the function sum to calculate. sum(bayes_table$Posterior[bayes_table$p > 0.5]) ## [1] 0.01270655 (c) The probability that between 20% and 40% of the students prefer eating out on Friday.

Use the function sum to calculate. sum(bayes_table$Posterior[bayes_table$p >= 0.2 & bayes_table$p <= 0.4]) ## [1] 0.8106056

Exercise 4. Interpreting A Beta Curve Revisit Figure 7.4 where nine different Beta curves are displayed. In the context of students’ dining preference example where p is the proportion of students preferring Friday, interpret the following prior choices in terms of the opinion of p. For example, Beta(0.5, 0.5) represents the prior belief the extreme values p = 0 and p = 1 are more probable and p = 0.5 is the least probable. In the students’ dining preference example, specifying a Beta(0.5, 0.5) prior indicates the owner thinks the students’ preference of dining out on Friday is either very strong or very weak. (a) Interpret the Beta(1, 1) curve.

Beta(1, 1) is the uniform distribution between 0 and 1. This prior expresses the opinion that the unknown proportion p is equally likely to be any value between 0 and 1. (b) Interpret the Beta(0.5, 1) curve.

Beta(0.5, 1) represents the prior belief that the unknown proportion p is most likely to be close to 0, and the probability of prior values of p goes down as p approaches 1. (c) Interpret the Beta(4, 2) curve.

Beta(4, 2) represents the prior belief that the unknown proportion p is unlikely to be small (0) or large (1). It is most likely to be around 0.8. The probability of prior values of p drops down from its peak round 0.8 to both sides. (d) Compare the opinion about p expressed by the two Beta curves: Beta(4, 1) and Beta(4, 2).

Beta(4, 1) represents the prior belief that the unknown proportion p is most likely to be close to 1, and the probability of prior values of p goes down as p approaches 0. Whereas Beta(4, 2) assigns the highest prior probability around p = 0.8, and almost 0 prior probability around 0 and 1. From the angle of prior number of successes and prior number of failures, Beta(4, 2) expresses 1 more prior number of failure than Beta(4, 1), which drives down the prior probability assigned to large value of p. This makes sense as p is the proportion of success.

Exercise 5. Beta Probabilities Use the functions dbeta(), pbeta(), qbeta(), rbeta(), beta_area(), and beta_quantile() to answer the following questions about Beta probabilities.

99

(a) Compute the density of Beta(0.5, 0.5) at the values p = {0.1, 0.5, 0.9, 1.5}. Check your answers with the Beta(0.5, 0.5) curve in Figure 7.4.

Use the function dbeta() to calculate. dbeta(c(0.1,0.5,0.9,1.5), 0.5, 0.5) ## [1] 1.0610330 0.6366198 1.0610330 0.0000000 The density of Beta(0.5, 0.5) at p = {0.1, 0.5, 0.9, 1.5} are about 1.061, 0.637, 1.061, and 0.000 respectively. (b) If p ∼ Beta(6, 3), compute the probability P rob(0.2 ≤ p ≤ 0.6).

Use the function pbeta() or beta_area() to calculate. pbeta(0.6, 6, 3) - pbeta(0.2, 6, 3) ## [1] 0.3141632 beta_area(lo = 0.2, hi = 0.6, shape_par = c(6, 3))

P( 0.2 < P < 0.6 ) = 0.314

2

Density

Beta(6, 3)

1

0 0.00

0.25

0.50

0.75

1.00

P The probability P rob(0.2 ≤ p ≤ 0.6) is about 0.314 when p ∼ Beta(6, 3). (c) Compute the quantiles of the Beta(10, 10) distribution at the probability values in the set {0.1, 0.5, 0.9, 1.5}.

Use the function qbeta() to calculate. qbeta(0.1, 10, 10) ## [1] 0.3579299 qbeta(0.5, 10, 10)

100

## [1] 0.5 qbeta(0.9, 10, 10) ## [1] 0.6420701 The quantile of Beta(10, 10) at p = {0.1, 0.5, 0.9} are about 0.358, 0.5, and 0.642 respectively. There is no quantile for p = 1.5 as it is out of range [0, 1]. (d) Simulate a sample of 100 random values from Beta(4, 2).

Use the function rbeta() to simulate. beta_samps = rbeta(100, 4, 2) The R command beta_samps = rbeta(1000, 4, 2) produces a vector beta_samps containing 100 random values from Beta(4, 2).

Exercise 6. Comparing Beta Distributions Consider four Beta curves: (1) Beta(5, 5), (2) Beta(10, 10), (3) Beta(50, 50) and (4) Beta(100, 100). Think of the shape parameters a and b as counts of successes" andfailures" in a prior sample. Use one of the R beta functions (e.g. rbeta(), beta_area(), among others) to discuss the similarities and differences between these four Beta curves.

These four beta curves represent equal numbers of successes and failures in a prior sample (5 or 10 or 50 or 100), and they are all symmetric around p = 0.5.

The biggest difference among these four beta curves is that the sum a + b represents prior sample size, and the higher the sum, the stronger the prior belief. For example, when plotting these four beta curves on the same scale, we can see that Beta(100, 100) expresses the strongest belief of p around 0.5 (indicated by the peaked density curve), while Beta(5, 5) expresses the weakest belief. betapars <- matrix(c(5, 5, 10, 10, 50, 50, 100, 100), 4, 2, byrow = TRUE) p <- seq(.001, .999, length.out = 100) BETA <- NULL for (j in 1:4){ df <- data.frame(p = p, Density = dbeta(p, betapars[j, 1], betapars[j, 2])) df$Type <- paste("Beta(", betapars[j, 1], ",", betapars[j, 2], ")", sep = "") BETA <- rbind(BETA, df) } ggplot(BETA, aes(p, Density)) + geom_line(color = "blue", size = 1.5) + ylim(0, 12) + facet_wrap(~ Type, scale = "free") + increasefont() + theme(axis.text.y=element_blank()) + scale_x_continuous(breaks=seq(0, 1, 0.5))

101

Density

Beta(10,10)

0.0

0.5

Beta(100,100)

1.0 0.0

Beta(5,5)

0.0

0.5

0.5

1.0

Beta(50,50)

1.0 0.0

0.5

1.0

p Exercise 7. Specifying A Continuous Beta Prior Consider another dining survey conducted by a restaurant owner in New York. The owner is also interested in knowing about the proportion p of students prefer eating out on Friday. He believes that its 0.4 quantile is 0.7 and 0.8 quantile is 0.9. Suppose the owner plans on using a beta prior distribution. (a) Find the values of the beta shape parameters a and b to represent the restaurant owner’s belief.

Use the function beta.select() to calculate. beta.select(list(x = 0.7, p = 0.4), list(x = 0.9, p = 0.8)) ## [1] 3.21 1.24 (b) Confirm the choice of beta prior by taking a simulated sample from the prior predictive simulation.

[Hint: use the rbeta() function to simulate a sample from the selected beta distribution, and then simulate new ỹ values from the binomial data model (function rbinom()) with a sample size of 20.

Graph and/or calculate a few quantiles of the simulated ỹ sample from the predictive distribution to check the restaurant owner’s prior belief.]

Use the functions rbeta(), rbinom() to simulate. beta_samps <- rbeta(1000, 3.21, 1.24) y_pred <- rbinom(1000, 20, beta_samps) quantile(y_pred, c(0.4, 0.8))/20 ## 40% 80%

102

## 0.7 0.9 By simulating 1000 ỹ from the prior predictive distribution and checking the associated p out of n = 20, we can see that the chosen Beta(3.21, 1.24) matches with the restaurant owner’s prior belief, that 0.4 quantile is 0.7 and 0.8 quantile is 0.9.

Exercise 8. Deriving the Beta Posterior Following the derivation process of the dining preference example in Section 7.4.1, derive this more general result. If the proportion has a Beta(a, b) prior and one samples Y from a Binomial distribution with parameters n and p, then if one observes Y = y, then the posterior density of p is Beta(a + y, b + n − y).

By Bayes’ theorem, π(p | Y = y) ∝ =

π(p) × f (Y = y | p)

Γ(a + b) a−1 n y p (1 − p)b−1 p (1 − p)n−y Γ(a)Γ(b) y

∝ pa+y−1 (1 − p)b+n−y−1 , which is another beta distribution with shape parameters a + y and b + n − y. Therefore, the posterior density of p is Beta(a + y, b + n − y).

Exercise 9. Prior Sample Size and Strength of Priors Another way of specifying a Beta(a, b) prior is to imagine a pre-survey with the same question and represent the Beta shape parameters in the form of a successes and b failures in the pre-survey (see Table 7.3). This exercise explores this prior specification method.

Table 7.3. Updating the Beta prior.

Source

Successes

Failures

Prior Data/Likelihood Posterior

a y a+y

b n−y b+n−y

a (a) Recall from Section 7.3 that the mean of the Beta(a, b) distribution is a+b . Define the prior sample size to be np = a + b. Consider two Beta prior distributions: Beta(2, 2) and Beta(20, 20). Find the prior means and prior sample sizes of these two prior distributions and compare the prior beliefs of these two beta distributions.

2 The prior mean and prior sample size are 2+2 = 0.5 and 2 + 2 = 4 for Beta(2, 2) respectively. The prior mean 20 and prior sample size are 20+20 = 0.5 and 20 + 20 = 40 for Beta(20, 20) respectively. These two beta prior distributions express the same belief about the prior mean of p, although Beta(20, 20) has 10 times the prior sample size of Beta(2, 2).

(b) Suppose a survey yields 4 successes out of 10 responses. Suppose one wishes to compare the posterior inference obtained by the two different Beta priors Beta(2, 2) and Beta(20, 20). Find and compare the two posterior distributions corresponding to these two priors.

We know that a Beta(a, b) prior with a survey result y out of n produces a posterior density of Beta(a + y, b + n − y). Therefore, the two resulting posterior distributions are Beta(6, 8) and Beta(24, 26). betapars <- matrix(c(6, 8, 24, 26),

103

2, 2, byrow = TRUE) p <- seq(.001, .999, length.out = 100) BETA <- NULL for (j in 1:2){ df <- data.frame(p = p, Density = dbeta(p, betapars[j, 1], betapars[j, 2])) df$Type <- paste("Beta(", betapars[j, 1], ",", betapars[j, 2], ")", sep = "") BETA <- rbind(BETA, df) } ggplot(BETA, aes(p, Density)) + geom_line(color = "blue", size = 1.5) + ylim(0, 6) + facet_wrap(~ Type, scale = "free") + increasefont() + theme(axis.text.y=element_blank()) + scale_x_continuous(breaks=seq(0, 1, 0.5))

Beta(6,8)

Density

Beta(24,26)

0.0

0.5

1.0 0.0

0.5

1.0

p Plotting the two beta densities side-by-side, we see a much higher concentration of probability for p around 0.5 for Beta(24, 26) compared to that for Beta(6, 8), expressing a stronger belief, which corresponds to a larger sample size. (c) Consider the use of the Beta(2, 2) and Beta(20, 20) prior distributions. Show these two priors have the same prior mean, but different strengths of belief about the location of the proportion. Assuming the survey results in (b), use simulation and graphs to show how different prior sample sizes affect the posterior inference.

104

Plotting the two beta priors side-by-side, and given the numerical calculation in part (a), we can see that the two beta priors have the same prior mean of 0.5 and different strengths: Beta(20, 20) is stronger than Beta(2, 2). betapars <- matrix(c(2, 2, 20, 20), 2, 2, byrow = TRUE) p <- seq(.001, .999, length.out = 100) BETA <- NULL for (j in 1:2){ df <- data.frame(p = p, Density = dbeta(p, betapars[j, 1], betapars[j, 2])) df$Type <- paste("Beta(", betapars[j, 1], ",", betapars[j, 2], ")", sep = "") BETA <- rbind(BETA, df) } ggplot(BETA, aes(p, Density)) + geom_line(color = "blue", size = 1.5) + ylim(0, 6) + facet_wrap(~ Type, scale = "free") + increasefont() + theme(axis.text.y=element_blank()) + scale_x_continuous(breaks=seq(0, 1, 0.5))

Beta(20,20)

Density

Beta(2,2)

0.0

0.5

1.0 0.0

0.5

1.0

p Graphs in part (b) have shown how different prior sample sizes affect the posterior inference. We can also use function rbeta() to simulate a large number of beta posterior draws from the two beta posterior distributions and come to a similar conclusion.

105

(d) Suppose a survey yields 40 successes out of 100 responses. Find the two posterior distributions corresponding to the two prior distributions Beta(2, 2) and Beta(20, 20). Contrast the two posterior distributions and compare with your answer to part (c).

We know that a Beta(a, b) prior with a survey result y out of n produces a posterior density of Beta(a + y, b + n − y). Therefore, the two resulting posterior distributions are Beta(42, 62) and Beta(60, 80).

The two panels below show how different survey results affect the posterior inference, given the same prior distribution. Generally, the larger the posterior sample size a + b + n, the stronger the posterior belief. In addtion, the larger the data sample size y and n − y, the stronger the influence the data has on the posterior inference, i.e. the mean of Beta(42, 62) is closer to the sample mean 0.4 compared to that of Beta(6, 8). betapars <- matrix(c(6, 8, 42, 62), 2, 2, byrow = TRUE) p <- seq(.001, .999, length.out = 100) BETA <- NULL for (j in 1:2){ df <- data.frame(p = p, Density = dbeta(p, betapars[j, 1], betapars[j, 2])) df$Type <- paste("Beta(", betapars[j, 1], ",", betapars[j, 2], ")", sep = "") BETA <- rbind(BETA, df) } ggplot(BETA, aes(p, Density)) + geom_line(color = "blue", size = 1.5) + ylim(0, 10) + facet_wrap(~ Type, scale = "free") + increasefont() + theme(axis.text.y=element_blank()) + scale_x_continuous(breaks=seq(0, 1, 0.5))

106

Beta(6,8)

Density

Beta(42,62)

0.0

0.5

1.0 0.0

p betapars <- matrix(c(24, 26, 60, 80), 2, 2, byrow = TRUE) p <- seq(.001, .999, length.out = 100) BETA <- NULL for (j in 1:2){ df <- data.frame(p = p, Density = dbeta(p, betapars[j, 1], betapars[j, 2])) df$Type <- paste("Beta(", betapars[j, 1], ",", betapars[j, 2], ")", sep = "") BETA <- rbind(BETA, df) } ggplot(BETA, aes(p, Density)) + geom_line(color = "blue", size = 1.5) + ylim(0, 10) + facet_wrap(~ Type, scale = "free") + increasefont() + theme(axis.text.y=element_blank()) + scale_x_continuous(breaks=seq(0, 1, 0.5))

107

0.5

1.0

Beta(60,80)

Density

Beta(24,26)

0.0

0.5

1.0 0.0

0.5

1.0

p (e) Consider the two prior distributions Beta(9, 1) and Beta(45, 5). Contrast these these two Beta prior distributions with respect to the mean and strength of belief. Compare the two posterior distributions with data n = 20, y = 5, and with the data n = 200, y = 50.

The means of both beta priors are 0.9. Beta(45, 5) has a stronger prior belief due to larger prior sample size.

When n = 20, y = 5, the two corresponding posteriors are Beta(14, 16) and Beta(50, 20). The posterior inference is affected more by the prior with larger prior sample size. betapars <- matrix(c(14, 16, 50, 20), 2, 2, byrow = TRUE) p <- seq(.001, .999, length.out = 100) BETA <- NULL for (j in 1:2){ df <- data.frame(p = p, Density = dbeta(p, betapars[j, 1], betapars[j, 2])) df$Type <- paste("Beta(", betapars[j, 1], ",", betapars[j, 2], ")", sep = "") BETA <- rbind(BETA, df) } ggplot(BETA, aes(p, Density)) + geom_line(color = "blue", size = 1.5) + ylim(0, 10) + facet_wrap(~ Type, scale = "free") + increasefont() + theme(axis.text.y=element_blank()) +

108

scale_x_continuous(breaks=seq(0, 1, 0.5))

Beta(50,20)

Density

Beta(14,16)

0.0

0.5

1.0 0.0

0.5

1.0

p When n = 200, y = 50, the two corresponding posteriors are Beta(59, 151) and Beta(95, 155). The posterior inference is affected more by the prior with larger prior sample size (plot omitted for brevity).

Exercise 10. Beta Posterior Mean is a Weighted Mean If the proportion has a Beta(a, b) prior and one observes Y from a binomial distribution with parameters n and p, then if one observes Y = y, then the posterior density of p is Beta(a + y, b + n − y). a Recall that the mean of a Beta(a, b) random variable following is a+b . Show that the posterior mean of p | Y = y ∼ Beta(a + y, b + n − y) is a weighted average of the prior mean of p ∼ Beta(a, b) and the sample mean p̂ = ny . Find the two weights and explain their implication for the posterior being a combination of prior and data.

The posterior mean can be re-written in the following: a+y a a+b y n = × + × a+b+n a+b a+b+n n a+b+n a Indeed, the posterior mean is a weighted average of the prior mean a+b and of the sample mean ny . The a+b n corresponding weights are a+b+n and a+b+n respectively.

Recall that a + b can be thought of as prior sample size, and n is the data sample size. The total sample size a+b n is a + b + n, and the weights a+b+n and a+b+n reflect how the prior and the data contribute to the posterior.

As more data is collected, i.e. n increases, the contribution from data will dominate the posterior, explaining how Bayesian inference allows collected data to sharpen one’s belief from prior to posterior.

109

Exercise 11. Sequential Updating The restaurant owner’s belief about the proportion of students’ favorite dining day being Friday is represented by a Beta(15.06, 10.56) distribution. Recall that he obtained this posterior distribution from a Beta(3.06, 2.56) prior and a survey of 12 yes’s out of 20 responses. The owner is interested in conducting another dining survey a few months later with the same question and the owner is still interested in p, the proportion of all students who say Friday. (a) The second survey gives a result of 8 yes’s out of 20 responses. Use the owner’s current beliefs and this information to update the restaurant owner’s belief about the proportion p.

Starting with prior Beta(15.06, 10.56) and survey result n = 20, y = 8, we come to the posterior Beta(15.06 + 8, 10.56 + 20 − 8) = Beta(23.06, 22.56). (b) Suppose the two surveys are conducted at the same time and the results are 20 yes’s (12 + 8) out of 40 responses (20 + 20). Starting with the Beta(3.06, 2.56) prior, update the owner’s belief about the proportion of interest.

Starting with prior Beta(3.06, 2.56) and surveyr result n = 40, y = 20, we come to the posterior Beta(3.06 + 20, 2.56 + 40 − 20) = Beta(20.06, 22.56). (c) Are the two posterior distributions the same in parts (a) and (b)? Why or why not?

The two posterior distributions are the same in parts (a) and (b). Mathematically we have verified that they are the same. Intuitively, since we are essentially observing the same total number of successes and total number of failures, the resulting posteriors will be the same, regardless of two samples in a roll or one big, combined sample. (d) Suppose the two survey results are reversed. That is, the first survey gives 8 yes’s and second survey gives 12 yes’s. Do you still observe the same posterior as in part (b)? What does this tell you about the order of different pieces of information shaping the belief about a parameter?

When the two survey results are reversed, we still observe the same posterior as in part (b). This tells us that the order of different pieces of information would not affect the resulting posterior belief about a parameter. (e) What if the two survey results are slightly different? e.g., first survey gives 15 yes’s and second survey gives 5 yes’s. What is the posterior distribution in this case?

When the two survey results are slightly different, and if they produce the same total number of successes and total number of failures, as with the 15 and 5 example, the posterior distribution will still be the same Beta(20.06, 22.56). However, if they produce a different total number of successes and total number of failures, we will obtain a different posterior distribution. (f) Should we combine the two survey results together? Describe a scenario where you it would be inappropriate to combine the survey results.

Given parts (a) to (d), we can see that when combining two survey results together, mathematically we will obtain the same posterior distribution. However, such combining process is only sensible if at any stage of the study, the parameter of interest stays the same, and the two survey results are independent and measuring the same parameter of interest. Only when these conditions are satisfied, it makes sense to combine two samples into a big one to conduct our Bayesian inference.

Exercise 12. Bayesian Hypothesis Testing In the dining preference example, the restaurant owner’s posterior distribution of proportion p of students preferring Friday to eat out is Beta(15.06, 10.56). Suppose the owner’s wife claims that between 50% and 60% of the students prefer to eat out on Friday. Conduct a Bayesian hypothesis test of this claim.

Use the function beta_area() to calculate the posterior probability, P rob(0.5 ≤ p ≤ 0.6) = 0.358, which is not high. We could say that there is only 0.358 posterior probability that p is between 50% and 60%.

110

beta_area(lo = 0.5, hi = 0.6, shape_par = c(15.06, 10.56))

P( 0.5 < P < 0.6 ) = 0.358 4

Density

3

Beta(15.06, 10.56)

2

1

0 0.00

0.25

0.50

0.75

1.00

P Similarly we can use the function rbeta() to approximate this posterior probability, which gives us a reasonably close result.

S <- 1000 BetaSamples <- rbeta(S, 15.06, 10.56) sum(BetaSamples <= 0.6 & BetaSamples >= 0.5)/S ## [1] 0.383

Exercise 13. Simulation Sample Size Revisit Section 7.5.2. Use R to simulate random samples of sizes S = {10, 100, 500, 1000, 5000} of p from the Beta(15.06, 10.56) distribution. Use the quantile() function to find the approximate 90% credible interval of p for each value of S. Comment on the effect of the simulation size S on the accuracy of the simulation results. Recall that the exact middle 90% posterior interval estimate is [0.427, 0.741].

Use the functions rbeta() and quantile() to approximate, and we see that the larger the sample size S, the better the approximation.

S <- 10 BetaSamples <- rbeta(S, 15.06, 10.56) quantile(BetaSamples, c(0.05, 0.95)) ## 5% 95% ## 0.4758593 0.7329371

111

S <- 100 BetaSamples <- rbeta(S, 15.06, 10.56) quantile(BetaSamples, c(0.05, 0.95)) ## 5% 95% ## 0.4394929 0.7026323 S <- 500 BetaSamples <- rbeta(S, 15.06, 10.56) quantile(BetaSamples, c(0.05, 0.95)) ## 5% 95% ## 0.4196474 0.7416786 S <- 1000 BetaSamples <- rbeta(S, 15.06, 10.56) quantile(BetaSamples, c(0.05, 0.95)) ## 5% 95% ## 0.4285056 0.7413031 S <- 5000 BetaSamples <- rbeta(S, 15.06, 10.56) quantile(BetaSamples, c(0.05, 0.95)) ## 5% 95% ## 0.4257914 0.7415383

Exercise 14. Bayesian Credible Intervals In the dining preference example, the restaurant owner’s posterior distribution of proportion p of students preferring Friday to eat out is Beta(15.06, 10.56). Find the exact Bayesian credible intervals for the following cases. (a) The middle 95% credible interval.

Use either the function qbeta() or the function beta_interval() to calculate. qbeta(c(0.025, 0.975), 15.06, 10.56) ## [1] 0.3960802 0.7665697 beta_interval(0.95, c(15.06, 10.56))

112

P( 0.396 < P < 0.767 ) = 0.95 4

Density

3

Beta(15.06, 10.56)

2

1

0 0.00

0.25

0.50

0.75

P (b) The middle 98% credible interval.

Use either the function qbeta() or the function beta_interval() to calculate. qbeta(c(0.01, 0.99), 15.06, 10.56) ## [1] 0.3613398 0.7945660 beta_interval(0.98, c(15.06, 10.56))

113

1.00

P( 0.361 < P < 0.795 ) = 0.98 4

Density

3

Beta(15.06, 10.56)

2

1

0 0.00

0.25

0.50

0.75

1.00

P (c) The 90% credible interval of the form (0, B).

Use the function qbeta() to calculate. qbeta(c(0, 0.9), 15.06, 10.56) ## [1] 0.0000000 0.7099912 (d) The 99% credible interval of the form (A, 1) Use the function qbeta() to calculate. qbeta(c(0.1, 1), 15.06, 10.56) ## [1] 0.4625504 1.0000000

Exercise 15. Simulating the Posterior of the Log Odds Since one is able to compute exact posterior summaries using the pbeta() and qbeta() functions, what is the point of using simulation computations? To illustrate the advantage of suppose one is simulation,

p interested in finding a 90% probability interval about the logit or log odds log 1−p . One can approximate the posterior of the logit by simulation. First simulate S = 1000 values from the Beta posterior for p, and then for each simulated value of p, compute a value of the logit. The resulting vector will be a random sample from the posterior distribution of the logit. (a) If the posterior for p is Beta(12, 20), use R to simulate 1000 draws from the posterior of

distribution the logit log

p 1−p

.

114

Use the function rbeta() to simulate samples of Beta(12, 20), then obtain samples of the logit through the

function log

p 1−p

. The LogitSamples contains 1000 draws from the posterior of the logit.

BetaSamples <- rbeta(1000, 12, 20) LogitSamples <- log(BetaSamples/(1-BetaSamples)) (b) Construct a 90% interval estimate for the logit parameter.

Use the function quantile() to construct a middle 90% interval estimate for the logit parameter. quantile(LogitSamples, c(0.05, 0.95)) ## 5% ## -1.10702096

95% 0.06867986

Exercise 16. Simulating the Odds Revisit Exercise 15. Instead of the logit or log odds of the proportion p, suppose we are interested in the p odds 1−p . If the posterior distribution for p is Beta(12, 20), use R to simulate 1000 values from the posterior distribution of the odds. Construct a histogram of the simulated odds and construct a 90% interval estimate.

Experiment with different values of the simulation sample size S and comment on the effect of the value of S on the width of the 90% interval estimates.

BetaSamples <- rbeta(1000, 12, 20) OddsSamples <- BetaSamples/(1-BetaSamples) df <- as.data.frame(OddsSamples) ggplot(df, aes(OddsSamples)) + geom_histogram(fill = "blue", color = "white", bins = 15) + increasefont()

115

count

200

100

0 0.0

0.5

1.0

1.5

OddsSamples quantile(OddsSamples, c(0.05, 0.95)) ## 5% 95% ## 0.3222205 1.0757631 To experiment with different values of the simulations sample size S, BetaSamples_S100 <- rbeta(100, 12, 20) BetaSamples_S500 <- rbeta(500, 12, 20) BetaSamples_S1000 <- rbeta(1000, 12, 20) BetaSamples_S10000 <- rbeta(10000, 12, 20) df <- rbind(data.frame(S = "S = 100", odds = BetaSamples_S100/(1-BetaSamples_S100)), data.frame(S = "S = 500", odds = BetaSamples_S500/(1-BetaSamples_S500)), data.frame(S = "S = 1000", odds = BetaSamples_S1000/(1-BetaSamples_S1000)), data.frame(S = "S = 10000", odds = BetaSamples_S10000/(1-BetaSamples_S10000))) ggplot(data = data.frame(x = 0), mapping = aes(x = x)) + geom_histogram(data = df, bins = 20, aes(x = odds, y = ..density..), color="black", fill="blue") + xlim(0, 1) +

116

2.0

facet_wrap(~ S, ncol=2) + xlab("p") + ylab("Density") + increasefont()

S = 100

S = 500

S = 1000

S = 10000

4 3 2

Density

1 0 4 3 2 1 0 0.00

0.25

0.50

0.75

1.000.00

0.25

0.50

0.75

1.00

p quantile(BetaSamples_S100, c(0.05, 0.95)) ## 5% 95% ## 0.2761940 0.4872124 quantile(BetaSamples_S500, c(0.05, 0.95)) ## 5% 95% ## 0.2353459 0.5168727 quantile(BetaSamples_S1000, c(0.05, 0.95)) ## 5% 95% ## 0.2450883 0.5168163 quantile(BetaSamples_S10000, c(0.05, 0.95)) ## 5% 95% ## 0.2407100 0.5183798

Exercise 17. Teenagers and Televisions In 1998, the New York Times and CBS News polled 1048 randomly selected 13 − 17 year olds to ask them if they had a television in their room. Among this group of teenagers, 692 of them said they had a television in

117

their room. Alex and Benedict both want to use the binomial model for this dataset, but they have different prior believes about p, the proportion of teenagers having a television in their room. (a) Alex asks 10 friends the same question and 8 of them have a television in their room. Alex decides to use this information to construct his prior. Design a continuous beta prior reflecting Alex’s belief.

Interpreting the parameters a and b in Beta(a, b) as prior number of successes and prior number of failures, we can design a Beta(8, 2) prior reflecting Alex’s belief. This prior reflects a prior sample size of 10. (b) Benedict thinks the 0.2 quantile is 0.3 and the 0.9 quantile is 0.4. Design a continuous beta prior reflecting Benedict’s belief.

Using the beta_select() function, we can design a Beta(34.95, 67.98) prior reflecting Benedict’s belief. This prior reflects a prior sample size of around 103. beta.select(list(x = 0.3, p = 0.2), list(x = 0.4, p = 0.9)) ## [1] 34.95 67.98 (c) Calculate Alex’s posterior and Benedict’s posterior distributions. Plot the two priors on one graph, and plot the corresponding posteriors on another graph. In addition, obtain 95% credible intervals for Alex and Benedict.

Alex’s posterior distribution is Beta(700, 358), and Benedict’s posterior distribution is Beta(726.95, 423.98)$.

Below are the plots for the two priors and two posteriors, and the corresponding 95% credible intervals. require(reshape2) S <- 1000 y <- 692 n <- 1048 a_alex <- 8 b_alex <- 2 a_benedict <- 34.95 b_benedict <- 67.98 prior_alex <- rbeta(S, a_alex, b_alex) prior_benedict <- rbeta(S, a_benedict, b_benedict) priors <- as.data.frame(cbind(prior_alex, prior_benedict)) names(priors) <- c("Alex", "Benedict") posterior_alex <- rbeta(S, a_alex + y, b_alex + n - y) posterior_benedict <- rbeta(S, a_benedict + y, b_benedict + n - y) posteriors <- as.data.frame(cbind(posterior_alex, posterior_benedict)) names(posteriors) <- c("Alex", "Benedict") priors_long <- melt(priors) ggplot(priors_long, aes(value, color = variable)) + geom_density() + theme_grey(base_size = 20, base_family = "") + xlim(0,1) + ylim(0,10) + xlab("p") + ggtitle("Priors")

118

Priors 10.0

density

7.5

variable 5.0

Alex Benedict

2.5 0.0 0.00

0.25

0.50

0.75

1.00

p posteriors_long <- melt(posteriors) ggplot(posteriors_long, aes(value, color = variable)) + geom_density() + theme_grey(base_size = 20, base_family = "") + xlim(0.5,0.75) + ylim(0,30) + xlab("p") + ggtitle("Posteriors")

119

Posteriors

density

30

20

variable Alex Benedict

10

0 0.50

0.55

0.60

0.65

0.70

p ## Exercise 95% credible intervals ## Exercise Exact solutions beta_interval(0.95, c(a_alex + y, b_alex + n - y))

120

0.75

P( 0.633 < P < 0.69 ) = 0.95

Beta(700, 358)

Density

20

10

0 0.00

0.25

0.50

P beta_interval(0.95, c(a_benedict + y, b_benedict + n - y))

121

0.75

1.00

P( 0.604 < P < 0.659 ) = 0.95

Beta(726.95, 423.98)

Density

20

10

0 0.00

0.25

0.50

0.75

1.00

P ## Exercise Monte Carlo approximation quantile(posterior_alex, c(0.025, 0.975)) ## 2.5% 97.5% ## 0.6350033 0.6861674 quantile(posterior_benedict, c(0.025, 0.975)) ## 2.5% 97.5% ## 0.6020918 0.6580958 (d) Conduct prior predictive checks for Alex and Benedict. For each person, is the prior consistent with the television data? Explain. newy_alex <- as.data.frame(rep(NA, S)) names(newy_alex) <- c("y") for (s in 1:S){ pred_p_sim <- rbeta(1, a_alex, b_alex) pred_y_sim <- rbinom(1, n, pred_p_sim) newy_alex[s,] <- pred_y_sim } ggplot(data = newy_alex, aes(newy_alex$y)) + geom_histogram(breaks=seq(200, 1100, by=5)) + xlab("y") + ggtitle("Prior predictive checks for Alex") + annotate("point", x = y, y = 0, colour = "red", size = 5) + theme_grey(base_size = 20, base_family = "")

122

Prior predictive checks for Alex 20

count

15 10 5 0 300

600

900

y newy_benedict = as.data.frame(rep(NA, S)) names(newy_benedict) = c("y") for (s in 1:S){ pred_p_sim <- rbeta(1, a_benedict, b_benedict) pred_y_sim <- rbinom(1, n, pred_p_sim) newy_benedict[s,] = pred_y_sim } ggplot(data=newy_alex, aes(newy_benedict$y)) + geom_histogram(breaks=seq(200,1000, by=5)) + xlab("y") + ggtitle("prior predictive checks for Benedict") + annotate("point", x = y, y = 0, colour = "red", size = 5) + theme_grey(base_size = 20, base_family = "")

123

prior predictive checks for Benedict

count

40 30 20 10 0 200

400

600

800

1000

y The prior predictive checks for Alex and Benedict show that Alex’s prior is more appropriate for this teenagers and television data. We can conclude from the plots below, that the predicted values from Alex’s prior is somewhat similar to the observed y = 692, whereas that from Benedict’s prior is very different from the observed y = 692.

Exercise 18. Teenagers and Televisions (continued) Revisit Exercise 17. Consider the odds of having a television in the room. Recall that if p is the probability p of having a television in room, then the odds is 1−p . (a) Find the mean, median and 95% posterior interval of Alex’s analysis of the odds of teenagers having a television in their room. odds_alex = posterior_alex/(1-posterior_alex) mean(odds_alex) ## [1] 1.964475 median(odds_alex) ## [1] 1.960037 quantile(odds_alex, c(0.025, 0.975)) ## 2.5% 97.5% ## 1.739751 2.186413 (b) Find the mean, median and 95% posterior interval of Benedict’s analysis of the odds of teenagers having a television in their room.

124

odds_benedict = posterior_benedict/(1-posterior_benedict) mean(odds_benedict) ## [1] 1.71335 median(odds_benedict) ## [1] 1.710078 quantile(odds_benedict, c(0.025, 0.975)) ## 2.5% 97.5% ## 1.513142 1.924796 (c) Compare the two posterior summaries from parts (a) and (b).

Overall, the two posterior summaries of odds are similar to each other, with Alex’s slightly higher in terms of the mean, the median, and the credible interval.

Exercise 19. Comparing Two Proportions - Science Majors at Liberal Arts Colleges Many liberal arts colleges and other organizations have been promoting science majors in recent years because of their value on the job market. One wishes to evaluate whether such promotion has any effect on student major preference. A college student Clara is interested in this question and collects data from three liberal arts colleges, presented in Table 7.4.

Table 7.4. Data from three liberal arts colleges on science majors in 2005 and 2015.

Year

Science

Non-Science

2005 2015

264 437

1496 1495

(a) Let p2005 and p2015 denote the proportions of science majors in 2005 and 2015, respectively. Assuming that p2005 and p2015 have independent uniform priors, obtain the joint posterior distribution of p2005 and p2015 . Recall that the Beta(1, 1) distribution is equivalent to the Uniform(0, 1) distribution.

By conjugacy with Beta(1, 1) prior, the individual posterior distributions for p2005 and p2015 should be Beta(265, 1497) and Beta(438, 1496) respectively. Since the prior distributions are independent, and if we assume that the likelihoods are independent as well, the posterior distributions should be independent.

Therefore, the joint posterior distribution of p2005 and p2015 is the product of the two individual posterior distributions. π(p2005 , p2015 | y2005 , y2015 ) = π(p2005 | y2005 )π(p2015 | y2015 ) p2005 | y2005 ∼ Beta(265, 1497) p2015 | y2015 ∼ Beta(438, 1496) (b) Suppose one uses the parameter δ = p2015 − p2005 to measure the difference in proportions. Use simulation from the posterior distribution to answer the question “have the proportions of science majors changed from 2005 to 2015?”. [Hint: simulate a vector s2005 of posterior samples of p2005 and another vector s2015 of posterior samples of p2015 (make sure to use the same number of samples) and subtract s2005 from s2015 which yields a vector of simulated values from the posterior of δ.]

S <- 5000 p_2005 <- rbeta(S, 265, 1497) 125

p_2015 <- rbeta(S, 438, 1496) d1 <- p_2015 - p_2005 sum(d1 > 0)/S ## [1] 1 From the simulation result, we can conclude that the proportion of science majors changed from 2005 to 2015. (c) Did the proportion of science majors change from 2005 to 2015? Answer this question by a posterior computation. o_2005 <- p_2005/(1 - p_2005) o_2015 <- p_2015/(1 - p_2015) d2 <- o_2015 - o_2005 sum(d2 > 0)/S ## [1] 1 From the simulation result, we can conclude that the odds of science majors changed from 2005 to 2015. (d) Compile a similar dataset for your school type, and answer parts (a) through (c).

Open-ended. (e) What assumption is made about the proportions p2005 and p2015 in our assignment of priors? Do you think such assumption is justified? If not, how do you think one can adjust the approach to be more realistic?

We have made the independence assumption when assigning priors for p2005 and p2015 , and also that each proportion is uniformally distributed between 0 and 1.

Open-ended.

Exercise 20. Comparing Two Proportions - Number of Depression Cases at a Hospital Data are collected on depression cases at hospitals. For a particular hospital, in the year of 1992, there were 306 diagnosed with depression among 651 patients; in the year of 1993, there were 300 diagnosed with depression among 705 patients. One is interested in learning whether the probability of being diagnosed with depression changed between 1992 and 1993. Conduct a Bayesian analysis of this question. State clearly the inference procedure, the choice of prior distributions, the choice of data model, the posterior distributions and the conclusions.

The data can be expressed as y1992 = 306, n1992 = 651 and y1993 = 300, n1993 = 705.

Suppose we give independent uniform priors for p1992 and p1993 in the form of Beta(1, 1). Using the binomial data models and by independence of priors and likelihoods, we know that the posterior distributions for p1992 and p1993 should be Beta(307, 346) and Beta(301, 406), respectively.

Through simulation, we have the following posterior probability that p1993 > p1992 , which indicates a change in the probability of being diagnosed with depression between 1992 and 1993. a <- 1 b <- 1 y_1992 <- 306 n_1992 <- 651 y_1993 <- 300 n_1993 <- 705 S <- 5000 p_1992 <- rbeta(S, y_1992 + a, n_1992 - y_1992 + b); 126

p_1993 <- rbeta(S, y_1993 + a, n_1993 - y_1993 + b); d <- p_1993 - p_1992 sum(d > 0)/S ## [1] 0.0496

Exercise 21. Prior Predictive Checking - Pizza Popularity Suppose a restaurant is serving pizza of two varieties, cheese and pepperoni, and a manager is interested in the proportion p of customers who prefer pepperoni. After some thought, the manager’s prior beliefs about p are represented by a Beta(6, 12) distribution. (a) Suppose a random sample of 20 customers is surveyed on their pizza preference and let Y denote the number that prefer pepperoni. Compute and graph the prior predictive density of Y . a <- 6 b <- 12 n <- 20 S <- 1000 pred_p_sim <- rbeta(S, a, b) pred_y_sim <- rbinom(S, n, pred_p_sim) data.frame(Y = pred_y_sim) %>% group_by(Y) %>% summarize(N = n()) %>% mutate(Probability = N / sum(N), Type = "Prior") %>% select(Type, Y, Probability) -> S1 ggplot(S1, aes(Y, Probability)) + geom_segment(aes(xend = Y, yend = 0), size = 3, lineend = "butt", color = "blue") + facet_wrap(~ Type, ncol=1) + increasefont()

127

Prior

Probability

0.10

0.05

0.00 0

5

10

15

Y (b) Suppose 20 customers are sampled and 14 prefer pepperoni. Is the value y = 14 consistent with the Bayesian model where p has a Beta(6, 12) distribution? Explain why or why not.

Based on the graph above and calculation below, a value y = 14 does not seem to be consistent with the Bayesian model where p has a Beta(6, 12) distribution.

S1$Probability[S1$Y == 14] ## [1] 0.006

Exercise 22. Bayes Factor - Pizza Popularity In the restaurant example of Exercise 21, suppose one of the waiters has a different opinion about the popularity of pepperoni pizza. His prior belief about the proportion p preferring pizza is described by a Beta(12, 6) distribution. (a) Find and graph the prior predictive density of the number y who prefer pizza in a sample of 20 customers. a <- 12 b <- 6 S <- 1000 n <- 20 pred_p_sim <- rbeta(S, a, b) pred_y_sim <- rbinom(S, n, pred_p_sim) data.frame(Y = pred_y_sim) %>% group_by(Y) %>% summarize(N = n()) %>% mutate(Probability = N / sum(N), 128

Type = "Prior") %>% select(Type, Y, Probability) -> S1 ggplot(S1, aes(Y, Probability)) + geom_segment(aes(xend = Y, yend = 0), size = 3, lineend = "butt", color = "blue") + facet_wrap(~ Type, ncol=1) + increasefont()

Prior

Probability

0.10

0.05

0.00 5

10

15

20

Y (b) If 14 out of 20 customers prefer pepperoni, is this result consistent with the predictive distribution?

Based on the graph above and calculation below, a value y = 14 seems to be consistent with the Bayesian model where p has a Beta(12, 6) distribution.

S1$Probability[S1$Y == 14] ## [1] 0.124 (c) Compare the two Bayesian models where (1) p is distributed Beta(12, 6) distribution and (2) p is distributed Beta(6, 12) distribution by a Bayes factor. Interpret the value that you compute.

Starting with equally likely probability of the two priors, 0.5 and 0.5, we can compute the Bayes factor with the binomial.beta.mix() function. probs <- c(0.5, 0.5) beta_par1 <- c(6, 12) beta_par2 <- c(12, 6) beta_par <- rbind(beta_par1, beta_par2)

129

output <- binomial.beta.mix(probs, beta_par, c(14, 6)) (posterior_odds <- output$probs[1] / output$probs[2]) ## beta_par1 ## 0.06988142 Given the observations (14 successes in 20 trials), there is 0.070 times support for prior Beta(6, 12) than for prior Beta(12, 6).

Exercise 23. Posterior Predictive Checking - Pizza Popularity Consider the same problem as in Exercise 22 where p is the proportion of customers who prefer pepperoni and the manager’s prior beliefs are given by a Beta(6, 12) distribution. (a) Suppose 14 out of 20 customers prefer pepperoni. Using the algorithm described in Section 7.6, simulate 1000 values of ỹ (out of 20 customers) from the posterior predictive distribution. Construct a histogram of these values. a <- 6 b <- 12 n <- 20 y <- 14 S <- 1000 pred_p_sim <- rbeta(S, a + y, b + n - y) pred_y_sim <- rbinom(S, n, pred_p_sim) data.frame(Y = pred_y_sim) %>% group_by(Y) %>% summarize(N = n()) %>% mutate(Probability = N / sum(N), Type = "Posterior") %>% select(Type, Y, Probability) -> S1 ggplot(S1, aes(Y, Probability)) + geom_segment(aes(xend = Y, yend = 0), size = 3, lineend = "butt", color = "blue") + facet_wrap(~ Type, ncol=1) + increasefont()

130

Posterior

Probability

0.15

0.10

0.05

0.00 5

10

15

Y (b) Is the observation (14 preferring pepperoni) consistent with this predictive distribution? Explain.

Given the graph above, yes the observation is consistent with this predictive distribution. (c) Repeat parts (a) and (b) using the waiter’s Beta(12, 6) distribution.

Using the waiter’s Beta(12, 6) prior, the observation is consistent with the corresponding predictive distribution. a <- 12 b <- 6 n <- 20 y <- 14 S <- 1000 pred_p_sim <- rbeta(S, a + y, b + n - y) pred_y_sim <- rbinom(S, n, pred_p_sim) data.frame(Y = pred_y_sim) %>% group_by(Y) %>% summarize(N = n()) %>% mutate(Probability = N / sum(N), Type = "Posterior") %>% select(Type, Y, Probability) -> S1 ggplot(S1, aes(Y, Probability)) + geom_segment(aes(xend = Y, yend = 0), size = 3, lineend = "butt", color = "blue") + facet_wrap(~ Type, ncol=1) + increasefont()

131

Posterior

Probability

0.15

0.10

0.05

0.00 5

10

15

20

Y Exercise 24. Learning from a Multinomial Experiment In Chapter 6 Section 6.3, we discussed the multinomial distribution, an extension of the Binomial distribution where each trial has more than two outcomes. As an application of a Multinomial experiment, in an analysis of an election poll, suppose that one wants inferences for three probabilities: θA = probability of a vote for a candidate from Party A, θB = probability of a vote for a candidate from Party B and θC = probability of a vote for a candidate from Party C. One assumes θA + θB + θC = 1 as people can vote for only one party. If a random sample of n potential voters is taken, the respective voter counts YA , YB , YC has the probability mass function p(YA = yA , YB = yB , YC = yC ) =

n! θyA θyB θyC , yA !yB !yC ! A B C

(1)

where yA + yB + yC = n and yA , yB , yC ≥ 0. This is written Multinomial(n; θA , θB , θC ). (a) A convenient prior distribution for (θA , θB , θC ) is the Dirichlet distribution, which has the density function αA −1 αB −1 αC −1 p(θA , θB , θC ) = KθA θB θC , Γ(αA +αB +αC ) where (αA , αB , αC ) are positive constants, and K = Γ(α is a normalizing constant.

A )Γ(αB )Γ(αC ) This is written Dirichlet(αA , αB , αC ). Install the gtools R package and explore ddirichlet() and rdirichlet() functions to evaluate the pdf and generate random samples from Dirichlet(αA = 2, αB = 1, αC = 1).

DirichletSamples <- rdirichlet(100, c(2, 1, 1))

132

(b) Suppose the prior distribution is Dirichlet(αA , αB , αC ) and one collects from n sampled voters, where (YA , YB , YC ) ∼ Multinomial(n; θA , θB , θC ). Find the posterior distribution of (θA , θB , θC ) and show that this is a Dirichlet distribution with updated parameters.

Let ai = 1 if voter i selects Party A and ai = 0 otherwise. Let bi = 1 if voter i selects Pn Party B and Pn bi = 0 otherwise.

Let c = 1 if voter i selects Party C and c = 0 otherwise.

Note that a = y , i i A i=1 i i=1 bi = Pn yB , i=1 ci = yC .

Then, we have π(θA , θB , θC | Y ) ∝ =

f (Y | θA , θB , θC )π(θA , θB , θC )

αA −1 αB −1 αC −1 ai bi ci Πni=1 θA θB θC θA θB θC

=

yA +αA −1 yB +αB −1 yc +αC −1 θA θB θC

Hence, the posterior is also a Dirichlet distribution, Dirichlet(yA + αA , yB + αB , yc + αC ) (c) Suppose in the sample of n = 100 voters, 53 voted for Party A, 18 voted for Party B, and 29 voted for Party C (yA = 53, yB = 18, yC = 29). Using the prior distribution Dirichlet(αA = 2, αB = 1, αC = 1) and the generic results from part (b), obtain the posterior distribution for (θA , θB , θC ). Plot the prior and the posterior distributions for (θA , θB , θC ) and discuss your findings.

From the results in part (b), we know that the posterior distribution is Dirichlet(55, 19, 30).

DirichletPriorSamples <- rdirichlet(1000, c(2, 1, 1)) DirichletPosteriorSamples <- rdirichlet(100, c(55, 19, 30)) theta_A <- as.data.frame(cbind(DirichletPriorSamples[, 1], DirichletPosteriorSamples[, 1])) names(theta_A) <- c("Prior", "Posterior") theta_A_long <- melt(theta_A) ggplot(theta_A_long, aes(value, color = variable)) + geom_density() + theme_grey(base_size = 10, base_family = "") + xlim(0,1) + ylim(0,15) + xlab("theta_A") + ggtitle("theta_A")

133

theta_A 15

10

density

variable Prior Posterior 5

0 0.00

0.25

0.50

0.75

1.00

theta_A theta_B <- as.data.frame(cbind(DirichletPriorSamples[, 2], DirichletPosteriorSamples[, 2])) names(theta_B) <- c("Prior", "Posterior") theta_B_long <- melt(theta_B) ggplot(theta_B_long, aes(value, color = variable)) + geom_density() + theme_grey(base_size = 10, base_family = "") + xlim(0,1) + ylim(0,15) + xlab("theta_B") + ggtitle("theta_B")

134

theta_B 15

10

density

variable Prior Posterior 5

0 0.00

0.25

0.50

0.75

1.00

theta_B theta_C <- as.data.frame(cbind(DirichletPriorSamples[, 3], DirichletPosteriorSamples[, 3])) names(theta_C) <- c("Prior", "Posterior") theta_C_long <- melt(theta_C) ggplot(theta_C_long, aes(value, color = variable)) + geom_density() + theme_grey(base_size = 10, base_family = "") + xlim(0,1) + ylim(0,15) + xlab("theta_C") + ggtitle("theta_C")

135

theta_C 15

10

density

variable Prior Posterior 5

0 0.00

0.25

0.50

0.75

1.00

theta_C The posteriors are sharpened by the data, as we can see the shorter spread of the posterior density compared to that of the prior density in each of the plots. The posterior plots also show that θA is around 0.5 while θB and θC are around 0.25. (d) Suppose one wants to compute the ratio of odds of voting for Party A to the odds of voting for Party A) B, θθBA /(1−θ /(1−θB ) . Compute a 95% posterior interval for this odds ratio.

Below is a middle 95% posterior interval for this odds ratio.

ratio_AB <- (DirichletPosteriorSamples[, 1]/(1 - DirichletPosteriorSamples[, 1]))/(DirichletPosteriorSam quantile(ratio_AB, c(0.025, 0.975)) ## 2.5% 97.5% ## 2.278676 9.804192

Chapter 8: Modeling Measurement and Count Data Exercise 1. Another Set of Federer’s Time-to-Serve Measurements (Discrete Priors) Suppose another set of thirty Federer’s time-to-serve measurements are collected with an observed mean of 19 seconds. Assume the same uniform discrete prior on the values µ = 15, 16, . . . , 22. The prior and the likelihood function are displayed below. π(µ) =

1 , µ = 15, 16, ..., 22, 8

136

n L(µ) ∝ exp − 2 (ȳ − µ)2 .

2σ (a) Assuming σ = 4, perform the Bayes’ rule calculation to find the posterior distribution for µ. df <- data.frame(mu = seq(15, 22, 1), Prior = rep(1/8, 8)) %>% mutate(Likelihood = dnorm(mu, 19, 4 / sqrt(20))) df <- bayesian_crank(df) round(df, 4) ## mu Prior Likelihood Product Posterior ## 1 15 0.125 0.0000 0.0000 0.0000 ## 2 16 0.125 0.0016 0.0002 0.0016 ## 3 17 0.125 0.0366 0.0046 0.0366 ## 4 18 0.125 0.2387 0.0298 0.2387 ## 5 19 0.125 0.4460 0.0558 0.4460 ## 6 20 0.125 0.2387 0.0298 0.2387 ## 7 21 0.125 0.0366 0.0046 0.0366 ## 8 22 0.125 0.0016 0.0002 0.0016 (b) Using the posterior, find a “best” estimate at µ and an interval of values that contains µ with probability 0.5.

The posterior columns shows that a “best” estimate at µ is 19 with posterior probability of 0.446. One interval that contains µ with probability at least 0.5 is [18, 19].

Exercise 2. Temperature in Bismarck Suppose one is interested in learning about the average January daily temperature (in degrees Fahrenheit) in Bismarck, North Dakota. One assumes that the daily temperature Y is Normally distributed with mean µ and known standard deviation σ = 10. Suppose that one’s prior is uniformly distribution over the values µ = 5, 10, 15, 20, 25, 30, 35. Suppose one observes the temperature for one January day to be 28 degrees.

Find the posterior distribution of µ and compute the posterior probability the mean is at least as large as 30 degrees. df <- data.frame(mu = seq(5, 35, 5), Prior = rep(1/7, 7)) %>% mutate(Likelihood = dnorm(mu, 28, 10 / sqrt(1))) df <- bayesian_crank(df) round(df, 4) ## mu Prior Likelihood Product Posterior ## 1 5 0.1429 0.0028 0.0004 0.0171 ## 2 10 0.1429 0.0079 0.0011 0.0478 ## 3 15 0.1429 0.0171 0.0024 0.1037 ## 4 20 0.1429 0.0290 0.0041 0.1752 ## 5 25 0.1429 0.0381 0.0054 0.2307 ## 6 30 0.1429 0.0391 0.0056 0.2366 ## 7 35 0.1429 0.0312 0.0045 0.1889 sum(df$Posterior[df$mu >= 30]) ## [1] 0.4254612 The posterior distribution is listed above and the posterior probability the mean is at least as large as 30 degrees is about 0.425.

137

Exercise 3. Choosing A Normal Prior (a) Suppose Sam believes that the 0.25 quantile of the mean of Federer’s time-to-serve µ is 14 seconds and the 0.8 quantile is 21 seconds. Using the normal.select() function, construct a normal prior distribution to match this belief. normal.select(list(p = 0.25, x = 14), list(p = 0.8, x = 21)) ## $mu ## [1] 17.11417 ## ## $sigma ## [1] 4.617076 The normal prior distribution matching this believe is Normal(17.11, 4.62). (b) Suppose Sam also believes that the 0.10 quantile of his prior is equal to 10.5 seconds. Is this statement consistent with the normal prior chosen in part (a)? If not, how could you adjust the prior to reconcile this statement about the 0.10 quantile?

It seems that the the 0.10 quantile statement is not consistent with the normal prior chosen in part (a). We can make adjustments to the mean and standard deviation of the normal prior. Open-ended. qnorm(0.1, 17.11, 4.62) ## [1] 11.18923

Exercise 4. Choosing A Normal Prior Another way of choosing a normal prior for Federer’s mean time-to-serve µ is to specify statements about the prior predictive distribution for a future time-to-serve measurement Ỹ . Using results from Section 8.5.2, if µ has a normal prior p with mean µ0 and σ0 , then the predictive density of Ỹ is normal with mean µ0 and standard deviation σ 2 + σ02 , where we are assuming that the sampling standard deviation σ = 4 seconds. (a) Suppose your best guess at Ỹ is 15 seconds, and you are 90 percent confident that Ỹ is smaller than 25 seconds. Find the normal prior for µ that matches this prior information about the future time-to-serve.

The posterior normal distribution can be found using the normal.select() function as below. normal.select(list(p = 0.5, x = 15), list(p = 0.9, x = 25)) ## $mu ## [1] 15 ## ## $sigma ## [1] 7.803041 p Since σ 2 + σ02 = 7.803 and σ = 4, we can solve σ0 to be 6.700, which means the normal prior distribution is Normal(15, 6.700). sqrt(7.803^2-4^2) ## [1] 6.699762 (b) Suppose instead that you are 90% confident that the future time-to-serve is between 18 and 24 seconds.

Find the normal prior for µ that matches this prior belief.

Similarly to part (a),

138

normal.select(list(p = 0.05, x = 18), list(p = 0.95, x = 24)) ## $mu ## [1] 21 ## ## $sigma ## [1] 1.82387 sqrt(1.824^2-4^2) ## [1] NaN (18 + 24)/2 ## [1] 21 There is no normal prior that matches such belief.

Exercise 5. Bayesian Hypothesis Testing The posterior distribution for the mean time-to-serve µ for Federer is Normal with mean 17.4 seconds and standard deviation 0.77 seconds. (a) Using this posterior, evaluate the plausibility of the statement “Federer’s mean time-to-serve is at least 16.5 seconds.”

Using the pnorm() function, we can compute that the probability of this statement is about 0.879, which is very plausible.

1 - pnorm(16.5, 17.4, 0.77) ## [1] 0.8787642 (b) Is it reasonable to say that Federer’s mean time-to-serve falls between 17 and 18 seconds? Explain.

Using the pnorm() function, we can compute that the probability of this statement is about 0.480, not a very high probability. pnorm(18, 17.4, 0.77) - pnorm(17, 17.4, 0.77) ## [1] 0.4803623

Exercise 6. Bayesian Credible Interval The posterior distribution for the mean time-to-serve µ for Federer is Normal with mean 17.4 seconds and standard deviation 0.77 seconds. (a) Construct a central 98% credible interval for µ.

A central 98% credible interval for µ is [15.609, 19.191]. qnorm(c(0.01, 0.99), 17.4, 0.77) ## [1] 15.60871 19.19129 (b) Can you use the credible interval to test the hypothesis “Federer’s mean time-to-serve is at least 16.5 seconds”? Explain.

This 98% credible interval includes 16.5. The interval contains plausible values of the mean, therefore we could use this interval to support the hypothesis.

139

Exercise 7. Posterior Predictive Distribution Write an R script to generate S = 1000 predictions of a single time-to-serve of Federer based on the posterior predictive distribution using the results given in Equation (8.31) and Equation (8.32). (a) Compare the exact posterior predictive distribution (Equation (8.30)) with the density estimate of the simulated predictions.

The comparison is done through the plots below.

S <- 1000 sigma <- 4 mu_n <- 17.4 sigma_n <- 0.77 pred_mean <- mu_n pred_sd <- sqrt(sigma ^ 2 + sigma_n ^ 2) pred_mu_sim <- rnorm(S, mu_n, sigma_n) pred_y_sim <- rnorm(S, pred_mu_sim, sigma) dest <- density(pred_y_sim) df1 <- data.frame(Prediction = dest$x, Density = dest$y, Type = "Simulated") df2 <- data.frame(Prediction = dest$x, Density = dnorm(dest$x, pred_mean, pred_sd), Type = "Exact") ggplot(rbind(df1, df2), aes(Prediction, Density, linetype = Type)) + geom_line(size = 1.5, color = "blue") + increasefont()

140

0.100

Density

0.075

Type 0.050

Simulated Exact

0.025

0.000 10

20

30

Prediction (b) Construct a 90% prediction interval for the future time-to-serve.

Using the quantile() function, we can get a central 90% prediction interval for the future time-t0-serve below. quantile(pred_y_sim, c(0.05, 0.95)) ## 5% 95% ## 11.06913 23.96199

Exercise 8. Posterior Predictive Checking The posterior predictive distribution can be used to check the suitability of the normal sampling/normal prior model for Federer’s time-to-serve data. The function post_pred_check() simulates samples of n = 20 from the posterior predictive function, and for each sample, computes a value of the checking function T (ỹ). post_pred_check <- function(test_function){ mu_n <- 17.4 sigma_n <- 0.77 sigma <- 4 n <- 20 one_sim <- function(){ mu <- rnorm(1, mu_n, sigma_n) test_function(rnorm(n, mu, sigma)) } replicate(1000, one_sim()) }

141

The output of the function is 1000 draws from the posterior predictive distribution of T . If the checking function is max(y), then one would obtain 1000 draws from the posterior predictive distribution by typing post_pred_check(max) If the value of the checking function on the observed time-to-serves T (y) is unusual relative to this posterior predictive distribution of T , this would cast doubt on the model. The observed times-to-serve for Federer are displayed in Section 8.3.1. and repeated below.

15.1 11.8 21.0 22.7 18.6 16.2 11.1 13.2 20.4 19.2 21.2 14.3 18.6 16.8 20.3 19.9 15.0 13.4 19.9 15.3 (a) Use the function post_pred_check() with the checking function T (y) = max(y) to check the suitability of the Bayesian model.

Federer_data <- c(15.1, 11.8, 21.0, 22.7, 18.6, 16.2, 11.1, 13.2, 20.4, 19.2, 21.2, 14.3, 18.6, 16.8, 20.3, 19.9, 15.0, 13.4, 19.9, 15.3) max_y <- max(Federer_data) sd_y <- sd(Federer_data) post_pred_check_max <- function(){ mu_n <- 17.4 sigma_n <- 0.77 sigma <- 4 n <- 20 one_sim <- function(){ mu <- rnorm(1, mu_n, sigma_n) max(rnorm(n, mu, sigma)) } replicate(1000, one_sim()) } sim_max <- post_pred_check_max() sum(max_y > sim_max)/1000 ## [1] 0.149 The probability that the maximum value from the sample is greater than that from the simulated data is not very low, indicating a reasonabl level of suitability of the Bayesian model. (b) Use the function post_pred_check() with the checking function T (y) = sd(y) to check the suitability of the Bayesian model. post_pred_check_sd <- function(){ mu_n <- 17.4 sigma_n <- 0.77 sigma <- 4 n <- 20 one_sim <- function(){ mu <- rnorm(1, mu_n, sigma_n) sd(rnorm(n, mu, sigma)) } replicate(1000, one_sim()) } sim_sd <- post_pred_check_sd() sum(sd_y > sim_sd)/1000 ## [1] 0.23 Same as part (a).

142

Exercise 9. Taxi Cab Fares Suppose a city manager is interested in learning about the mean fare µ for taxi cabs in New York City. (a) Suppose the manager believes that µ is smaller than $8 with probability 0.25, and that µ is smaller than $12 with probability 0.75. Find a normal prior that matches this prior information.

Using the normal.select() function, the normal prior matching this prior information is Normal(10, 2.965). normal.select(list(p = 0.25, x = 8), list(p = 0.75, x = 12)) ## $mu ## [1] 10 ## ## $sigma ## [1] 2.965204 (b) The manager collects the fares for twenty fares and observes the values (in dollars): 7.5, 8.5, 9.5, 6.5, 7.0, 6.0, 7.0, 16.0, 8.0, 8.5, 9.5, 13.5, 4.5, 8.5, 7.5, 13.0, 6.5, 9.5, 21.0, 6.5. Assuming these fares are normally distributed with mean µ and standard deviation σ = 4, find the posterior distribution for the mean µ.

Using results from Equations (8.21) to (8.23), we have µn and σn calculated in the following. The posterior normal distribution for the mean µ is Normal(9.290, 0.856)$.

ybar <- mean(c(7.5, 8.5, 9.5, 6.5, 7.0, 6.0,7.0, 16.0, 8.0, 8.5, 9.5, 13.5, 4.5, 8.5, 7.5, 13.0,6.5,9.5, n <- 20 mu_0 <- 10 phi_0 <- 1/2.965^2 sigma <- 4 phi <- 1/sigma^2 mu_n <- (phi_0*mu_0 + n*phi*ybar)/(phi_0 + n*phi) sigma_n <- sqrt(1/(phi_0 + n*phi)) mu_n ## [1] 9.289642 sigma_n ## [1] 0.8563132 (c) Construct a 90% interval estimate for the mean fare µ.

A central 90% interval estimate is below. qnorm(c(0.05, 0.95), 9.290, 0.856) ## [1]

7.882005 10.697995

Exercise 10. Taxi Cab Fares (continued) Suppose that a visitor to New York City has little knowledge about the mean taxi cab fare. (a) Construct a weakly informative prior for µ.

We can use a weakly informative prior for µ, for example, Normal(20, 10). (b) Use the data from Exercise 9 to compute the posterior distribution for the mean fare.

Similar to Exercise 9, the posterior distribution for the mean fare is Normal(9.311, 0.891).

143

ybar <- mean(c(7.5, 8.5, 9.5, 6.5, 7.0, 6.0,7.0, 16.0, 8.0, 8.5, 9.5, 13.5, 4.5, 8.5, 7.5, 13.0,6.5,9.5, n <- 20 mu_0 <- 20 phi_0 <- 1/10^2 sigma <- 4 phi <- 1/sigma^2 mu_n <- (phi_0*mu_0 + n*phi*ybar)/(phi_0 + n*phi) sigma_n <- sqrt(1/(phi_0 + n*phi)) mu_n ## [1] 9.310516 sigma_n ## [1] 0.8908708 (c) Construct a 90% interval estimate for the mean fare and compare your interval with the interval computed in Exercise 9 using an informative prior.

A central 90% interval estimate is below, which is very similar to the interval computed in Exercise 9. This shows how the data is driving the posterior inference. qnorm(c(0.05, 0.95), 9.311, 0.891) ## [1]

7.845435 10.776565

Exercise 11. Taxi Cab Fares (continued) (a) In Exercise 9, one finds the posterior distribution for the mean fare µ. Write an R function to simulate a sample of twenty fares from the posterior predictive distribution. pred_y_sim contains a sample of twenty fares from the posterior predictive distribution. n <- 20 sigma <- 4 pred_mu_sim <- rnorm(1, 9.290, 0.856) pred_y_sim <- rnorm(n, pred_mu_sim, sigma) (b) Looking at the observed data, one sees an unusually large fare of $21. To see if this fare is unusual for our model, first revise your function in part (a) to simulate the maximum fare of a sample of twenty fares from the posterior predictive distribution. Then repeat this process 1000 times, collecting the maximum fares for 1000 predictive samples. pred_y_sim_max contains 1000 predictive samples of the maximum fares. n <- 20 sigma <- 4 S <- 1000 pred_y_sim_max <- rep(NA, S) for (s in 1:S){ pred_mu_sim <- rnorm(1, 9.290, 0.856) pred_y_sim_max[s] <- max(rnorm(n, pred_mu_sim, sigma)) } (c) Construct a graph of the maximum fares. Is the fare of $21 large relative to the prediction distribution of maximum fares?

A graph of the maximum fares is sh own below. The fare of $21 is not large relative to this predictiion distribution of maximum fares.

144

df <- as.data.frame(pred_y_sim_max) names(df) <- c("max") ggplot(data = df, aes(df$max)) + geom_histogram(breaks = seq(0, 25, by = 1)) + xlab("y") + ggtitle("Posterior predictive of maximum fares") + annotate("point", x = 21, y = 0, colour = "red", size = 5) + theme_grey(base_size = 20, base_family = "")

Posterior predictive of maximum fares

count

150

100

50

0 0

5

10

15

20

25

y (d) Based on the answer to part (c), what does that say about the suitability of our model?

Using maximum fares as the evaluation measure, our model is suitable for the data analysis.

Exercise 12. Student Sleeping Times How many hours do college students sleep, on the average? Recently, some introductory students were asked when they went to bed and when they woke the following morning. A following random sample of 14 sleeping times (in hours) were recorded: 9.0, 7.5, 7.0, 8.0, 5.0, 6.5, 8.5, 7.0, 9.0, 7.0, 5.5, 6.0, 8.5, 7.5. Assume that these measurements follow a Normal sampling distribution with mean µ and standard deviation σ, where we are given that σ = 1.5. (a) Suppose that John believes a priori that the mean amount of sleep µ is normal with mean 8 hours and standard deviation 1 hour. Find the posterior distribution of µ.

The posterior distribution of µ given John’s prior is $Normal(7.385, 0.372). ybar <- mean(c(9.0, 7.5, 7.0, 8.0, 5.0, 6.5, 8.5, 7.0, 9.0, 7.0, 5.5, 6.0, 8.5, 7.5)) n <- 14 mu_0 <- 8 145

phi_0 <- 1/1^2 sigma <- 1.5 phi <- 1/sigma^2 mu_n <- (phi_0*mu_0 + n*phi*ybar)/(phi_0 + n*phi) sigma_n <- sqrt(1/(phi_0 + n*phi)) mu_n ## [1] 7.384615 sigma_n ## [1] 0.3721042 (b) Construct a 90% interval estimate for the mean µ.

A central 90% interval estimate is below. qnorm(c(0.05, 0.95), 7.385, 0.372) ## [1] 6.773114 7.996886 (c) Let y ∗ denote the sleeping time for a randomly selected student. Find the predictive distribution for y ∗ and use this to construct a 90% prediction interval.

Given results in Equation (8.30), we know that the predictive distribution for y ∗ is Normal(7.385, 1.545). pred_mean <- mu_n pred_sd <- sqrt(sigma ^ 2 + sigma_n ^ 2) pred_mean ## [1] 7.384615 pred_sd ## [1] 1.545465 A 90% prediction interval is below. qnorm(c(0.05, 0.95), 7.385, 1.545) ## [1] 4.843701 9.926299

Exercise 13. Student Sleeping Times (continued) Suppose two other people are interested in learning about the mean sleeping times of college students. Mary’s prior is normal with mean 8 hours and standard deviation 0.1 – she is pretty confident that the mean sleeping time is close to 8 hours. In contrast, Larry is very uncertain about the location of µ and assigns a normal prior with mean 8 hours and standard deviation 3 hours. (a) Find the posterior distributions of µ using Mary’s prior and using Larry’s prior.

The posterior distribution of µ using Mary’s prior is $Normal(7.958, 0.097). ybar <- mean(c(9.0, 7.5, 7.0, 8.0, 5.0, 6.5, 8.5, 7.0, 9.0, 7.0, 5.5, 6.0, 8.5, 7.5)) n <- 14 mu_mary <- 8 phi_mary <- 1/0.1^2 sigma <- 1.5 phi <- 1/sigma^2 mu_n_mary <- (phi_mary*mu_mary + n*phi*ybar)/(phi_mary + n*phi)

146

sigma_n_mary <- sqrt(1/(phi_mary + n*phi)) mu_n_mary ## [1] 7.958159 sigma_n_mary ## [1] 0.09702693 The posterior distribution of µ using Larry’s prior is $Normal(7.298, 0.397). ybar <- mean(c(9.0, 7.5, 7.0, 8.0, 5.0, 6.5, 8.5, 7.0, 9.0, 7.0, 5.5, 6.0, 8.5, 7.5)) n <- 14 mu_larry <- 8 phi_larry <- 1/3^2 sigma <- 1.5 phi <- 1/sigma^2 mu_n_larry <- (phi_larry*mu_larry + n*phi*ybar)/(phi_larry + n*phi) sigma_n_larry <- sqrt(1/(phi_larry + n*phi)) mu_n_larry ## [1] 7.298246 sigma_n_larry ## [1] 0.3973597 (b) Construct 90% interval estimates for µ using Mary’s and Larry’s priors.

A central 90% interval estimate using Mary’s prior is below. qnorm(c(0.05, 0.95), 7.958, 0.097) ## [1] 7.798449 8.117551 A central 90% interval estimate using Larry’s prior is below. qnorm(c(0.05, 0.95), 7.298, 0.397) ## [1] 6.644993 7.951007 (c) Compare the interval estimates with the interval estimates constructed in Exercise 12(b) using Mary’s prior. Is the location of the interval estimate sensitive to the choice of prior? If so, explain the differences.

The interval estimates with Mary’s prior are shorter, and overall greater than that constructed in Exercise 12(b); moreover, the center is larger. This shows that the location of the interval estimate is sensitive to the choice of prior, even if the difference in prior choices only exists for the standard deviation.

Exercise 14. Comparing Two Means - IQ Tests on School Children Do teachers’ expectations impact academic development of children? To find out, researchers gave an IQ test to a group of 12 elementary school children. They randomly picked six children and told teachers that the test predicts them to have high potential for accelerated growth (accelerated group); for the other six students in the group, the researchers told teachers that the test predicts them to have no potential for growth (no growth group). At the end of school year, they gave IQ tests again to all 12 students, and the change in IQ scores of each student is recorded. The following table shows the IQ score change of students in the accelerated group and the no growth group.

147

Table 8.8 Data from IQ score change of 12 students; 6 are in the accelerated group, and 6 are in the no growth group.

Group

IQ score change

Accelerated No growth

20, 10, 19, 15, 9, 18 3, 2, 6, 10, 11, 5

The sample means of the accelerated group and the no growth group are respectively ȳA = 15.2 and ȳN = 6.2.

Consider independent sampling models, where the IQ scores for the accelerated group (no growth group) are assumed Normal with mean µA (µN ) with known standard deviation σ = 4.

YA,i YN,i

i.i.d.

∼

Normal(µA , 4), for i = 1, · · · nA ,

i.i.d.

Normal(µN , 4), for i = 1, · · · nN ,

∼

where nA = nN = 6. (a) Assuming independent sampling, write down the likelihood function of the means (µA , µN ).

Assuming independent sampling, the likelihood function of (µA , µN ) is:

L(µA , µN )

= =

L(µA )L(µN )

nN

nA Y −(yA,i − µA )2 Y −(yN,i − µN )2 1 1 √ √ exp exp .

2σ 2 2σ 2 2πσ 2 2πσ 2 i=1 i=1

One can further simply the expression because the two sampling densities share the same standard deviation σ = 4. (b) Assume that one’s prior beliefs about µA and µN are independent, where µA ∼ Normal(γA , τA ) and µN ∼ Normal(γN , τN ). Show that the posterior distributions for µA and µN are independent normal and find the mean and standard deviation parameters for each distribution.

The joint posterior distribution of (µA , µN ) is:

π(µA , µN | yA , yN ) ∝ π(µA , µN )L(µA , µN ) = π(µA )π(µN )L(µA )L(µN ).

Focusing on the posterior distribution of µA :

π(µA | yA , yN )

= π(µA | yA ) ∝ π(µA )L(µA ).

Given what we know about the Normal-Normal conjugacy, the posterior distribution of µA is a normal distribution (in terms of precision φA = 1/τA2 and φ = 1/σ 2 ):

µA | yA , φA , φ ∼ Normal

φA γA + nA φȳA , φ A + nA φ

148

r

1 φA + n A φ

.

2 Similarly, the posterior distribution of µN is a normal distribution (in terms of precision φN = 1/τN and 2 φ = 1/σ ):

µN | yN , φN , φ ∼ Normal

φN γN + nN φȳN , φN + n N φ

r

1 φN + n N φ

.

Exercise 15. Comparing Two Means - IQ Tests on School Children (continued) In Exercise 14, you should have established that the mean IQ score changes µA and µN have independent normal posterior distributions. Assume that one has vague prior beliefs and µA ∼ Normal(0, 20) and µN ∼ Normal(0, 20). (a) Is the average improvement for the accelerated group larger than that for the no growth group? Consider the parameter delta = mu_A - mu_N to measure the difference in means. The question now becomes finding the posterior probability of delta > 0, i.e. p(mu_A - mu_N > 0 | y_A, y_N), where yA and yN are the vectors of recorded IQ score change. [Hint: simulate a vector sA of posterior samples of µA and another vector sN of posterior samples of µN (make sure to use the same number of samples) and subtract sN from sA , which gives us a vector of posterior differences between sN and sA . This vector of posterior differences serves as an approximation to the posterior distribution of δ.]

We can use Monte Carlo approximation by first simulating S draws of µA from its posterior distribution, and S draws of µN from its posterior distribution. Second, comparing element-by-element of the two vectors, sA and sN and approximate the probability p(µA − µN | yA , yN ), which is 1 with S = 1000. This means with probability 1, the average of IQ score change in the accelerated group is higher than that of the no growth group. yA <- c(20, 10, 19, 15, 9, 18) yN <- c(3, 2, 6, 10, 11, 5) sigma <- 4 phi <- 1/sigma^2 mu_A0 <- 0 sigma_A0 <- 20 phi_A0 <- 1/sigma_A0^2 yAbar <- mean(yA) nA <- 6 mu_An <- (phi_A0*mu_A0 + nA*yAbar*phi)/(phi_A0 + nA*phi) sd_An <- sqrt(1/(phi_A0 + nA*phi)) mu_N0 <- 0 sigma_N0 <- 20 phi_N0 <- 1/sigma_N0^2 yNbar <- mean(yN) nN <- 6 mu_Nn <- (phi_N0*mu_N0 + nN*yNbar*phi)/(phi_N0 + nN*phi) sd_Nn <- sqrt(1/(phi_N0 + nN*phi)) S <- 1000 mu_Apost <- rnorm(S, mean = mu_An, sd = sd_An) mu_Npost <- rnorm(S, mean = mu_Nn, sd = sd_Nn) sum(mu_Apost > mu_Npost)/S ## [1] 1

149

(b) What is the probability that a randomly selected child assigned to the accelerated group will have larger improvement than a randomly selected child assigned to the no growth group? Consider ỸA and ỸN to be random variables for predicted IQ score change for the accelerated group and the no growth group, respectively. The question now becomes finding the posterior predictive probability of ỸA > ỸN , i.e. p(ỸA > ỸN | yA , yN ), where yA and yN are the vectors of recorded IQ score change, each of length 6. [Hint: Show that the posterior predictive distributions of ỸA and ỸN are independent. Simulate predicted IQ score changes from the posterior predictive distributions for the two groups, then simulate the posterior predictive distribution of ỸA − ỸN by taking the difference of simulated draws.]

We can use Monte Carlo approximation by first simulating S draws of µA from its posterior distribution, and S draws of µN from its posterior distribution. Second, simulating S draws of ỹA from the sampling model, and simulating S draws of ỹN from the sampling model. Third, comparing element-by-element of the two vectors and approximate the probability p(ỸA > ỸN | yA , yN )), which is above 0.9 with S = 1000. This means with this probability value, the predicted IQ score change in the accelerated group is higher than that of the no growth group. newy_A <- rnorm(S, mu_Apost, 4) newy_N <- rnorm(S, mu_Npost, 4) sum(newy_A > newy_N)/S ## [1] 0.925

Exercise 16. Comparing Two Means - Prices of Diamonds Weights of diamonds are measured in carats. The difference between the size of a 0.99 carat diamond and a 1 carat diamond is most likely undetectable to the naked human eye, but the price of a 1 carat diamond tends to be much higher than the price of a 0.99 carat diamond. To find out if it is truly the case, data on point prices (the prices of 0.99 carat diamonds divided by 99, and the prices of 1 carat diamonds divided by 100) of n99 = 23 of 0.99 carat diamonds and n100 = 25 of 1 carat diamonds were collected and stored in the files pt99price.csv and pt100price.csv. (a) Explore the two datasets by making plots and computing summary statistics. What are the findings?

All summary statistics of the 0.99 carat diamonds are lower than that of the 1 carat diamond. The plot shows that 1 carat diamonds are generally more expensive than 0.99 carat diamonds. summary(pt99price$price) ## ##

Min. 1st Qu. 18.07 35.00

Median 48.28

Mean 3rd Qu. 44.51 55.34

Max. 61.56

Mean 3rd Qu. 58.74 72.60

Max. 105.62

summary(pt100price$price) ## ##

Min. 1st Qu. 32.93 46.49

Median 55.44

sd(pt99price$price) ## [1] 13.32311 sd(pt100price$price) ## [1] 18.27428 price <- as.data.frame(cbind(pt99price$price, pt100price$price)) names(price) <- c("pt99", "pt100") price_long <- melt(price) ggplot(price_long, aes(value, color = variable)) + 150

geom_density() + theme_grey(base_size = 20, base_family = "") + xlab("Price") + ggtitle("Density plots of diamond price")

Density plots of diamond price

density

0.02

variable pt99 pt100

0.01

0.00 25

50

75

100

Price (b) Consider independent normal sampling models for these datasets with a fixed and known value of the standard deviation. From your exploratory work, choose a value for the standard deviation.

We can choose 18 for the standard deviation. sd(price_long$value) ## [1] 17.73233 (c) Choose appropriate weakly informative prior distributions, and use posterior simulation to answer whether the average point price of the 1 carat diamonds is higher than that of the 0.99 diamonds.

Suppose weakly informative prior, Normal(15, 10), for the averages µpt99 and µpt100 . Based on the calculation below, the average point price of the 1 carat diamonds is higly likely higher than that of the 0.99 diamonds. y_99 <- pt99price$price y_100 <- pt100price$price sigma <- 18 phi <- 1/sigma^2 mu_99_0 <- 18 sigma_99_0 <- 10 phi_99_0 <- 1/sigma_99_0^2 y_99bar <- mean(y_99) n_99 <- length(y_99) mu_99_n <- (phi_99_0*mu_99_0 + n_99*y_99bar*phi)/(phi_99_0 + n_99*phi)

151

sd_99_n <- sqrt(1/(phi_99_0 + n_99*phi)) mu_100_0 <- 18 sigma_100_0 <- 10 phi_100_0 <- 1/sigma_100_0^2 y_100bar <- mean(y_100) n_100 <- length(y_100) mu_100_n <- (phi_100_0*mu_100_0 + n_100*y_100bar*phi)/(phi_100_0 + n_100*phi) sd_100_n <- sqrt(1/(phi_100_0 + n_100*phi)) S <- 1000 mu_99_post <- rnorm(S, mean = mu_99_n, sd = sd_99_n) mu_100_post <- rnorm(S, mean = mu_100_n, sd = sd_100_n) sum(mu_99_post > mu_100_post)/S ## [1] 0.003 (d) Perform posterior predictive checks of the Bayesian inferences obtained in part (c).

To perform posterior predictive checks for the 0.99 carat diamonds analysis using maximum price as the testing statistic, it seems that the model is consistent with the price data. Similar checks can be performed for other statistics, and for the 1 carat diamonds. max_y <- max(y_99) n <- n_99 sigma <- 18 S <- 1000 pred_y_sim_max <- rep(NA, S) for (s in 1:S){ pred_mu_sim <- rnorm(1, mu_99_n, sd_99_n) pred_y_sim_max[s] <- max(rnorm(n, pred_mu_sim, sigma)) } df <- as.data.frame(pred_y_sim_max) names(df) <- c("max") ggplot(data = df, aes(df$max)) + geom_histogram(breaks = seq(0, 150, by = 5)) + xlab("y") + ggtitle("Posterior predictive of maximum prices") + annotate("point", x = max_y, y = 0, colour = "red", size = 5) + theme_grey(base_size = 20, base_family = "")

152

Posterior predictive of maximum prices 200

count

150 100 50 0 0

50

100

150

y Exercise 17. Gamma-Poisson Conjugacy Derivation Section 8.8.3 presents the Bayesian update results for Poisson sampling with the use of the Gamma conjugate prior. (a) Verify the equation for the likelihood in Equation (8.37). [Hint:

f (Y1 = y1 , ..., Yn = yn | λ)

= =

n Y

f (yi | λ)

i=1 n Y

1 yi −λ λ e , y ! i=1 i

the joint sampling density of n i.i.d. Poisson distributed random variables.]

Given the hint, we can write the likelihood function for λ as

L(λ) = f (Y1 = y1 , ..., Yn = yn | λ)

=

n Y i=1 n Y

f (yi | λ)

1 yi −λ λ e y ! i=1 i Pn ∝ λ i=1 yi e−nλ . =

153

(b) Assuming that the Poisson parameter λ has a gamma prior with shape α and rate β, show that the posterior distribution of λ has a gamma functional form and find the parameters of this gamma distribution.

With prior π(λ) ∝ λα−1 e−βλ , and likelihood

Pn L(λ) ∝ λ i=1 yi e−nλ ,

using Bayes’ rule, we have Pn π(λ | y1 , · · · , yn ) ∝ π(λ)L(λ) = λα+ i=1 yi −1 e(β+n)λ , which is another gamma distribution: λ | y1 , · · · , yn ∼ Gamma(α +

n X

yi , β + n).

i=1

Exercise 18. The Number of ER Visits: the Prior Suppose two people, Pedro and Mia, have different prior beliefs about the average number of ER visits during the 10pm - 11pm time period. Pedro’s prior information is matched to a gamma distribution with parameters α = 70 and β = 10, and Mia’s beliefs are matched to a gamma distribution with α = 33.3 and β = 3.3. The two gGamma priors are displayed in Figure 8.12. (a) Compare the priors of Pedro and Mia with respect to average value and spread. Which person believes that there will be more ER visits, on average? Which person is more confident of his/her “best guess” at the average number of ER visits?

Mia believes that there will be more ER visits on average, because the density of her prior is concentrated at a higher value of average ER visit. Pedro seems more confident of his “best guess” at the average number of ER visits, because his prior has a smaller spread comparing to that of Mia’s. (b) Using the qgamma() function, construct 90% interval estimates for λ using Pedro’s prior and Mia’s prior.

90% interval estimates using Pedro’s prior and Mia’s prior are constructed below. a_Pedro <- 70 b_Pedro <- 10 a_Mia <- 33.3 b_Mia <- 3.3 qgamma(c(0.05, 0.95), a_Pedro, b_Pedro) ## [1] 5.682967 8.430648 qgamma(c(0.05, 0.95), a_Mia, b_Mia) ## [1]

7.396876 13.128930

(c) After some thought, Pedro believes that his best prior guess at λ is correct, but he is less confident in this guess. Explain how Pedro can adjust the parameters of his gamma prior to reflect this new prior belief.

Pedro can adjust the parameters, α = 70 and β = 10 in his gamma prior to keep the ratio of α β as 7, but increase the ratio βα2 to show less confidence. For example, he can set α = 7 and β = 1.

154

(d) Mia also revisits her prior. Her best guess at the average number of ER visits is now 3 larger than her previous best guess, but the degree of confidence in this guess hasn’t changed. Explain how Mia can adjust the parameters of her gamma prior to reflect this new prior belief.

Mia could increase the average by 3 without adjusting the spread by choosing a gamma with shape 43.3 and the same rate parameter.

Exercise 19. The Number of ER Visits A hospital collects the number of patients in the emergency room (ER) admitted between 10 pm and 11 pm for each day of a week. Table 8.9 records the day and the number of ER visits for the given day.

Table 8.9 Number of ER visits during the period 10 pm to 11 pm for each day for a particular week at a hospital.

Day

Number of ER visits

Sunday Monday Tuesday Wednesday Thursday Friday Saturday

8 6 6 9 8 9 7

Suppose one assumes Poisson sampling for the counts, and a conjugate gamma prior with parameters α = 70 and β = 10 for the Poisson rate parameter λ. (a) Given the sample shown in Table 8.9, obtain the posterior distribution for λ through the gamma-Poisson conjugacy. Obtain a 95% posterior credible interval for λ.

We know by the gamma-Poisson conjugacy:

λ ∼ Gamma(α, β)Y1 , · · · , Yn | λ ∼ Poisson(λ) → λ | Y1 , · · · , Yn ∼ Gamma(α +

n X

yi , β + n)

i=1

Therefore, with data {8, 6, 6, 9, 8, 9, 7}, the posterior distribution for λ is: λ | Y1 , · · · , Y7 ∼ Gamma(70 + 53, 10 + 7) = Gamma(123, 17).

A 95% posterior credible interval for λ is given below. qgamma(c(0.025, 0.975), 123, 17) ## [1] 6.013237 8.568716 (b) Suppose a hospital administrator states that the average number of ER visits during any evening hour does not exceed 6. By computing a posterior probability, evaluate the validity of the administrator’s statement.

Using the pgamma() function, the posterior probability that λ does not exceed 6 is 0.024, which is very low.

The data provides evidence against the hospital administrator’s statement. pgamma(6, 123, 17) ## [1] 0.02368644 155

(c) The hospital is interested in predicting the number of ER visits between 10 pm and 11 pm for another week. Use simulations to generate posterior predictions of the number of ER visits for another week (seven days). y_pred contains posterior predictions of the number of ER visits for another week.

S <- 7 lambda_pred <- rgamma(1, 123, 17) y_pred <- rpois(S, lambda_pred)

Exercise 20. Times Between Traffic Accidents The exponential distribution is often used as a model to describe the time between events, such as traffic accidents. A random variable Y has an exponential distribution if its pdf is as follows. ( f (y | λ) =

λ exp(−λy), if y ≥ 0.

0, if y < 0.

(2)

Here, the parameter λ > 0, considered as the rate of event occurrences. This is a one-parameter model. (a) The gamma distribution is a conjugate prior distribution for the rate parameter λ in the exponential data model. Use the prior distribution λ ∼ Gamma(a, b), and find its posterior distribution π(λ | y1 , · · · , yn ), i.i.d.

where yi ∼ Exponential(λ) for i = 1, · · · , n.

With prior π(λ) ∝ λα−1 e−βλ , and likelihood L(λ) =

n Y

λe−λyi = λn e−λ

Pn i=1

yi

,

i=1

using Bayes’ rule, we have Pn π(λ | y1 , · · · , yn ) ∝ π(λ)L(λ) = λα+n−1 e(β+ i=1 yi )λ , which is another gamma distribution: λ | y1 , · · · , yn ∼ Gamma(α + n, β +

n X

yi ).

i=1

(b) Suppose 10 times between traffic accidents are collected: 1.5, 15, 60.3, 30.5, 2.8, 56.4, 27, 6.4, 110.7, 25.4 (in minutes). With the posterior distribution derived in part (a), use Monte Carlo approximation to calculate the posterior mean, median, and a middle 95% credible interval for the rate λ. [Hint: choose the appropriate R functions from dgamma(), pgamma(), qgamma(), and rgamma().]

If we use a Gamma(1, 1) prior, we can obtain the posterior mean, median, and a middle 95% credible interval as below. y <- c(1.5, 15, 60.3, 30.5, 2.8, 56.4, 27, 6.4, 110.7, 25.4) n <- length(y) alpha_0 <- 1 beta_0 <- 1 alpha_n <- alpha_0 + n beta_n <- beta_0 + sum(y)

156

S <- 1000 LambdaSamples <- rgamma(S, alpha_n, beta_n) mean(LambdaSamples) ## [1] 0.03251512 median(LambdaSamples) ## [1] 0.03160791 quantile(LambdaSamples, c(0.025, 0.975)) ## 2.5% 97.5% ## 0.01622082 0.05246945 (c) Use Monte Carlo approximation to generate another set of 10 predicted times between events. [Hint: rexp() generates random draws from an exponential distribution.] pred_y_sim contains a set of 10 predicted times between events. pred_lambda_sim <- rgamma(1, alpha_n, beta_n) pred_y_sim <- rexp(10, pred_lambda_sim) pred_y_sim ## ##

[1] 3.697323 91.477688 32.527328 11.707172 [8] 21.575390 89.720610 20.529127

9.807830

8.778517

1.183561

Exercise 21 Modeling Survival Times The Weibull distribution is often used as a model for survival times in biomedical, demographic, and engineering analyses. A random variable Y has a Weibull distribution if its pdf is as follows. f (y | α, λ) = λαy α−1 exp(−λy α )

for y > 0.

(3)

Here, α > 0 and λ > 0 are parameters of the distribution. For this problem, assume that α = α0 is known, but λ is not known, i.e. a simplified case of a one-parameter model. Also assume that software routines for simulating from Weibull distributions are available (e.g. rweibull()) (a) Assuming a prior distribution π(λ | α = α0 ) ∝ 1, find its posterior π(λ | y1 , . . . , yn , α = α0 ), where i.i.d.

yi ∼ Weibull(λ, α = α0 ) for i = 1, · · · , n. Write the name of the distribution and expressions for its parameter values.

π(λ | α0 , y1 , . . . , yn ) ∝ f (y1 , . . . , yn | α0 , λ)π(λ | α0 ) n X = α0n λn Πni=1 yiα0 −1 exp(−λ yiα0 ) i=1

∝ λn exp(−λ

n X

yiα0 ).

i=1

The last step is justified because the terms that do not depend on λ are constants and hence will cancel from the numerator and denominator by P the proportionality constant. The final expression is the kernel of the n gamma distribution, Gamma(n + 1, i=1 yiα0 ). (b) Using the posterior distribution derived in part (a), explain step-by-step how you would use Monte Carlo simulation to approximate the posterior median survival time, assuming that α = α0 .

Pn • Step 1: Sample a value of λ from Gamma(n + 1, i=1 yiα0 ). Call this λ∗ .

157

• Step 2: Sample one value of Ỹ from a Weibull distribution with parameters α = α0 and λ = λ∗ .

• Step 3: Repeat steps 1 and 2 for 1000 or more times, and find the median of the simulated Ỹ values. (c) What family of distributions represents the conjugate prior distributions for λ, assuming that α = α0 .

For gamma posterior distributions, the conjugate prior distribution is a gamma distribution.

Chapter 9: Simulation by Markov Chain Monte Carlo Exercise 1. Normal and Cauchy Priors In the example in Section 9.1.2, it was assumed that the prior for the average snowfall µ was Normal with mean 10 inches and standard deviation 3 inches. (a) Confirm that the 25th and 75th percentiles of this prior are equal to 8 and 12 inches, respectively.

Use the qnorm() function to confirm. qnorm(c(0.25, 0.75), 10, 3) ## [1]

7.976531 12.023469

(b) Show that under this Normal prior, it is unlikely that the mean µ is at least as large as 26.75 inches.

1 - pnorm(26.75, 10, 3) ## [1] 1.179759e-08 (c) Confirm that a Cauchy distribution with location 10 inches and scale parameter 2 inches also have 25th and 75th percentiles equal to 8 and 12 inches, respectively. qcauchy(c(0.25, 0.75), 10, 2) ## [1]

8 12

Exercise 2. A Random Walk The following matrix represents the transition matrix for a random walk on the integers {1, 2, 3, 4, 5}.  .2 .2  P = 0 0 0

.8 .2 .2 0 0

0 .6 .6 .6 0

0 0 .2 .2 .8

 0 0  0  .2 .2

(a) Suppose one starts walking at the state value 4. Find the probability of landing at each location after a single step.

Look at the fourth row of the table. Starting at value 4, he will be at states 3, 4, 5 with respective probabilities 0.6, 0.2, 0.2. (b) Starting at value 4, find the probability of landing at each location after three steps.

Compute P ∗ P ∗ P using R. P <- matrix(c(0.2, 0.8, 0, 0, 0, 0.2, 0.2, 0.6, 0, 0, 0, 0.2, 0.6, 0.2, 0,

158

0, 0, 0.6, 0.2, 0.2, 0, 0, 0, 0.8, 0.2), 5, 5, byrow = TRUE) (P3 <- P %*% P %*% P) ## [,1] [,2] [,3] [,4] [,5] ## [1,] 0.104 0.320 0.480 0.096 0.000 ## [2,] 0.080 0.224 0.552 0.120 0.024 ## [3,] 0.040 0.184 0.552 0.184 0.040 ## [4,] 0.024 0.120 0.552 0.224 0.080 ## [5,] 0.000 0.096 0.480 0.320 0.104 Starting at location 4, after 3 steps, the traveler will be states 1, 2, 3, 4, 5 with respective probabilities P3[4, ] ## [1] 0.024 0.120 0.552 0.224 0.080 (c) Explain what is means for this Markov Chain to be irreducible and aperiodic.

Irreducible means that it is possible to get to any state from any other student. Periodic means that there is exists some state that can be visited only at multiples of some integer greater than 1. Aperiodic means that there exists no prediodic states.

Exercise 3. A Random Walk (continued) Consider the random walk Markov chain described in Exercise 2. (a) Suppose one starts at the location 1. Using an R script with the “‘sample()} function (see example script Section ??), simulate 1000 steps of the Markov chain using the probabilities given in the transition matrix. Store the locations of the walk in a vector.

The matrix P has already been defined in the previous exercise. s <- vector("numeric", 1000) s[1] <- 1 for (j in 2:1000){ s[j] <- sample(1:5, size=1, prob=P[s[j - 1], ]) } S <- data.frame(Iteration = 1:1000, Location = s) (b) Compute the relative frequencies of the walker in the five states from the simulation output. From this computation, guess at the value of the stationary distribution vector w. table(S$Location) ## ## ##

1 2 3 4 43 153 533 196

5 75

I am guessing that the stationary distribution is (0.05, 0.20, 0.50, 0.20, 0.05) (c) Confirm that your guess is indeed the stationary distribution by using the matrix computation w %*% P. w <- matrix(c(0.05, 0.20, 0.50, 0.20, 0.05), 1, 5) w %*% P

159

## [,1] [,2] [,3] [,4] [,5] ## [1,] 0.05 0.18 0.54 0.18 0.05 Actually the stationary distribution is given by (0.045, 0.182, 0.545, 0.182, 0.045)

Exercise 4. Weird Weather Suppose a city in Alaska has interesting weather. The four possible weather states are “sunny” (SU ), “rainy” (R), “cloudy” (C), and “snow” (SN ). If it is sunny one day, it is equally likely to be rainy, cloudy, and snow on the next day. If is currently rainy, then the probabilities of sunny, rain, cloudy, and snow on the next day are respectively 1/2, 1/6, 1/6, and 1/6. The following matrix gives the transitions of weather from one day to the next day.   SU   R  C SN

SU 0 1/2 0 0

 R C SN 1/3 1/3 1/3   1/6 1/6 1/6   1/4 1/2 1/4  1/4 1/4 1/2

(a) If the weather is rainy today, find the probability that is rainy two days later.

P <- matrix(c(0, 1/3, 1/3, 1/3, 1/2, 1/6, 1/6, 1/6, 0, 1/4, 1/2, 1/4, 0, 1/4, 1/4, 1/2), 4, 4, byrow = TRUE) P %*% P ## [,1] [,2] [,3] [,4] ## [1,] 0.16666667 0.2222222 0.3055556 0.3055556 ## [2,] 0.08333333 0.2777778 0.3194444 0.3194444 ## [3,] 0.12500000 0.2291667 0.3541667 0.2916667 ## [4,] 0.12500000 0.2291667 0.2916667 0.3541667 The probability of interest is 0.278. (b) Starting with a sunny day, write an R script to simulate 1000 days of weather using this Markov Chain. s <- vector("numeric", 1000) s[1] <- 1 for (j in 2:1000){ s[j] <- sample(1:4, size=1, prob=P[s[j - 1], ]) } S <- data.frame(Iteration = 1:1000, Location = s) (c) Find the relative frequencies of the four states. Are these values approximately the stationary distribution of the Markov chain? table(S$Location) ## ## 1 2 3 4 ## 120 242 320 318 The relative frequencies are

160

table(S$Location) / 1000 ## ## 1 2 3 4 ## 0.120 0.242 0.320 0.318 These are approximately the stationary distribution.

Exercise 5. Ehrenfest Urn Model Grinstead and Snell (2006) describe a model used to explain diffusion of gases. One version of this model is described in the setting of two urns that, between them, contains four balls. A state is the number of balls in the first urn. There are five possible states 0, 1, 2, 3, and 4. At each step, one ball is chosen at random and moved from the urn it is located to the other urn. The transition matrix for this Markov chain is shown below: 

0 1/4  P =  0  0 0

1 0 0 3/4 1/2 0 0 3/4 0 0

 0 0 0 0   1/2 0   0 1/4 1 0

P <- matrix(c(0, 1, 0, 0, 0, 1/4, 0, 3/4, 0, 0, 0, 1/2, 0, 1/2, 0, 0, 0, 3/4, 0, 1/4, 0, 0, 0, 1, 0), 5, 5, byrow = TRUE) (a) Starting at state 1, find the probabilities of each state after two steps.

P2 <- P %*% P P2[1, ] ## [1] 0.25 0.00 0.75 0.00 0.00 (b) Starting at state 1, find the probabilities of each state after three steps.

P3 <- P %*% P %*% P P3[1, ] ## [1] 0.000 0.625 0.000 0.375 0.000 (c) Explain why this Markov Chain is not aperiodic.

It can only reach state 1 on even visits. (d) Does a stationary distribution exist for this Markov Chain? Why or why not?

No, in this case, a stationary distribution does not exist.

Exercise 6. Metropolis Sampling in a Random Walk Suppose the variable X takes on values from 1 to 9 with respective probabilities that are proportional to the values 9, 7, 5, 3, 1, 3, 5, 7, 9. This probability distribution displayed in Figure 9.19 has a “bathtub” shape. (a) Write an R function that computes this probability distribution for any value of X.

161

pd <- function(x){ values <- c(9, 7, 5, 3, 1, 3, 5, 7, 9) ifelse(x %in% 1:length(values), values[x], 0) } (prob_dist <- data.frame(x = 1:8, prob = pd(1:8))) ## x prob ## 1 1 9 ## 2 2 7 ## 3 3 5 ## 4 4 3 ## 5 5 1 ## 6 6 3 ## 7 7 5 ## 8 8 7 (b) Using the Metropolis algorithm described in Section 9.3.1 as programmed in the function random_walk(), simulate 10,000 draws from this probability distribution starting at the value X = 2. library(ProbBayes) svalues <- random_walk(pd, 2, 10000) (c) Collect the simulated draws and find the relative frequencies of the values 1 through 9. Compare these approximate probabilities with the exact probabilities. table(svalues) / 10000 ## svalues ## 1 2 3 4 5 6 7 8 9 ## 0.1327 0.1069 0.0807 0.0555 0.0234 0.0719 0.1196 0.1751 0.2342 Here are the exact probabilities: prob_dist$prob / sum(prob_dist$prob) ## [1] 0.225 0.175 0.125 0.075 0.025 0.075 0.125 0.175

Exercise 7. Metropolis Sampling of a Binomial Distribution (a) Using the Metropolis algorithm described in Section 9.3 as programmed in the function random_walk(), simulate 1000 draws from a Binomial distribution with parameters n = 20 and p = 0.3. mybinom <- function(x) dbinom(x, size = 20, prob = 0.3) svalues <- random_walk(mybinom, 1, 1000) (b) Collect the simulated draws and find the relative frequencies of the values 0 through 20. Compare these approximate probabilities with the exact probabilities. table(svalues) / 1000 ## svalues ## 1 2 3 4 5 6 7 8 9 10 11 12 13 ## 0.001 0.021 0.075 0.138 0.187 0.201 0.172 0.105 0.048 0.031 0.011 0.007 0.003 Here are exact probabilities data.frame(x = 0:20, prob = round(dbinom(0:20, size = 20, prob = 0.3), 3))

162

## x prob ## 1 0 0.001 ## 2 1 0.007 ## 3 2 0.028 ## 4 3 0.072 ## 5 4 0.130 ## 6 5 0.179 ## 7 6 0.192 ## 8 7 0.164 ## 9 8 0.114 ## 10 9 0.065 ## 11 10 0.031 ## 12 11 0.012 ## 13 12 0.004 ## 14 13 0.001 ## 15 14 0.000 ## 16 15 0.000 ## 17 16 0.000 ## 18 17 0.000 ## 19 18 0.000 ## 20 19 0.000 ## 21 20 0.000 (c) Using the simulated values, estimate the mean µ and standard deviation σ of the distribution and compare these estimates with the known values of µ and σ of a binomial distribution. c(Mean = mean(svalues), SD = sd(svalues)) ## Mean SD ## 5.991000 1.993211 The exact values are c(Mean = 20 * .3, SD = sqrt(20 * .3 * .7)) ## Mean SD ## 6.00000 2.04939

Exercise 8. Metropolis Sampling - Poisson-Gamma Model Suppose we observe y1 , ..., yn from a Poisson distribution with mean λ, and the parameter λ has a Gamma(a, b) distribution. The posterior density is proportional to " n # Y a−1

yi π(λ | y1 , · · · , yn ) ∝ exp(−λ)λ λ exp(−bλ) . i=1

(a) Write a function to compute the logarithm of the posterior density. Assume that one observes the sample 2, 5, 10, 5, 6, and the prior parameters are a = b = 1. log_poisson_gamma <- function(lambda, y, a, b){ sum(dpois(y, lambda, log = TRUE)) + dgamma(lambda, shape = a, rate = b, log = TRUE) } y <- c(2, 5, 10, 5, 6) a <- 1 b <- 1 log_poisson_gamma(3, y, a, b) 163

## [1] -19.19065 (b) Use the metropolis() function in Section 9.3.3 to collect 1000 draws from the posterior distribution.

Use a starting value of λ = 5 and a neighborhood scale value of C = 2. y <- c(2, 5, 10, 5, 6) a <- 1 b <- 1 out <- metropolis(log_poisson_gamma, 5, 2, 1000, y, a, b) (c) Inspect MCMC diagnostic graphs to assess if the simulated sample approximates the posterior density of λ. library(coda) plot(mcmc(out$S))

Density of var1

0.0

3

4

0.1

5

0.2

6

0.3

7

0.4

8

Trace of var1

0

200

600

1000

2

Iterations

4

6

8

N = 1000 Bandwidth = 0.2416

Exercise 9. Metropolis Sampling from a Bimodal Distribution Suppose we observe a random sample y1 , ..., yn from a Cauchy distribution with location θ and scale parameter 1 with density 1 f (yi | θ) = .(#eq : cauchy) (4) π [1 + (yi − θ)2 ] If a Uniform prior is placed on θ, then the posterior density of θ is proportional to π(θ | y1 , · · · , yn ) ∝

n Y

1 (#eq : cauchypost) 2 π [1 + (y i − θ) ] i=1

164

(5)

If we observe the values 3, 6, 7, 8, 15, 14, 16, 17, Figure 9.20 displays the bimodal shape of the posterior density. (a) Write a function to compute the logarithm of the posterior density. log_cauchy_post <- function(theta, y){ sum(dcauchy(y, theta, 1, log = TRUE)) } (b) Using the metropolis() function in Section 9.3.3, collect a simulated sample of 1000 from the posterior distribution. Run the sampler twice, once using a starting value of θ = 10 and a neighborhood scale value of C = 3, and a second time with the same starting value and a scale value of C = 0.2. psample1 <- metropolis(log_cauchy_post, 10, 3, 1000, y) psample2 <- metropolis(log_cauchy_post, 10, 0.2, 1000, y) (c) By inspecting MCMC diagnostic graphs, which value of C appears to result in a simulated sample that is a better approximation to the posterior distribution? Explain. plot(mcmc(psample1$S))

Density of var1

0.0

4

0.1

0.2

6

0.3

8

0.4

0.5

10

0.6

Trace of var1

0

200

600

1000

2

Iterations

4

6

8

10

N = 1000 Bandwidth = 0.1981

plot(mcmc(psample2$S))

165

Density of var1

0.00

4

5

0.10

6

7

0.20

8

9

0.30

10

Trace of var1

0

200

600

1000

4

Iterations

6

8

10

N = 1000 Bandwidth = 0.3282

Due to the strong snake-like appearance in the 2nd set of plots, I would prefer the first sampler with a proposal width of C = 3.

Exercise 10. Gibbs Sampling - Poisson-Gamma Model Suppose a single observation Y conditional on λ is Poisson with mean λ, and λ has a Gamma(a, b) prior with density equal to ba a−1 π(λ) = λ exp(−bλ). Γ(a) (a) Write down the joint density of Y and λ.

f (λ, y) ∝ e−λ λy λa−1 exp(−bλ) (b) Identify the conditional distribution Y conditional on λ, and the conditional distribution of λ conditional on Y = y.

We have [y|λ] ∼ P oisson(λ) and [λ|y] ∼ Gamma(y + a, b + 1). (c) Use the information from part (b) to construct a Gibbs sampling algorithm to sample from the joint distribution of (Y, λ). gibbs_pg <- function(a, b, y = 1, iter = 1000){ x <- matrix(0, iter, 2) for(k in 1:iter){ lam <- rgamma(1, y + a, b + 1 ) y <- rpois(1, lambda = lam ) x[k, ] <- c(lam, y) } x } 166

(d) Write an R function to implement one cycle of Gibbs sampling, and run 1000 iterations of Gibbs sampling for the case where a = 3 and b = 3. out <- gibbs_pg(3, 3) (e) By integration, find the marginal density of Y . Compare the exact values of the marginal density with the simulated draws of Y found using Gibbs sampling.

Table of simulated draws of Y – convert frequencies to relative frequencies: table(out[, 2]) / 1000 ## ## 0 1 2 3 4 5 6 7 8 ## 0.391 0.326 0.177 0.077 0.015 0.008 0.004 0.001 0.001 Here is the exact marginal density of Y for comparison: f (y) =

ba Γ(y + a) , y = 0, 1, 2, ... Γ(a) (b + 1)y+a

Exercise 11. Gibbs Sampling - Coin Flips Suppose one observes the outcomes of four fair coin flips W1 , ..., W4 where Wi = 1 if the outcome is heads and Wi = 0 otherwise. Let X = W1 + W2 + W3 denote the number of heads in the first three flips and Y = W2 + W3 + W4 is the number of heads in the last three flips. The joint probability of X and Y is given in Table 9.3.

Table 9.3. Table of number of heads X in the first three flips and number of heads Y in last three flips in four flips of a fair coin.

X

0 1 2 3

Y 0 1/16 1/16 0 0

1 1/16 3/16 2/16 0

2 0 2/16 3/16 1/16

3 0 0 1/16 1/16

The joint probability mass function f (x, y) of the number of heads in the first three flips X and the number of heads in the last three flips Y in four tosses of a fair coin.

Put the probabilty distribution in a matrix: P <- matrix(c(1, 1, 0, 0, 1, 3, 2, 0, 0, 2, 3, 1, 0, 0, 1, 1) / 16, 4, 4, byrow = TRUE) rownames(P) <- 0:3 colnames(P) <- 0:3 P ## 0 1 2 3 ## 0 0.0625 0.0625 0.0000 0.0000 ## 1 0.0625 0.1875 0.1250 0.0000 ## 2 0.0000 0.1250 0.1875 0.0625

167

## 3 0.0000 0.0000 0.0625 0.0625 (a) Find the conditional distribution f (x | Y = 1). (px_1 <- P[, "1"] / sum(P[, "1"])) ## 0 1 2 3 ## 0.1666667 0.5000000 0.3333333 0.0000000 (b) Find the conditional distribution f (y | X = 2). (py_x <- P["2", ] / sum(P["2", ])) ## 0 1 2 3 ## 0.0000000 0.3333333 0.5000000 0.1666667 (c) Describe how Gibbs sampling can be used to simulate from the joint distribution of X and Y . (d) Using the gibbs_discrete() function in Section 9.5.1, simulate 1000 iterations of Gibbs sampling using this probability distribution. By tabulating the (X, Y ) output and computing relative frequencies, confirm that the relative frequencies are good approximations to the actual probabilities. out <- gibbs_discrete(P, i = 1, iter = 1000) output <- table(out[, 1], out[, 2]) / 1000 rownames(output) <- 0:3 colnames(output) <- 0:3 output ## ## ## ## ## ##

0 1 2 3 0 0.070 0.054 0.000 0.000 1 0.059 0.166 0.126 0.000 2 0.000 0.120 0.213 0.075 3 0.000 0.000 0.061 0.056

Exercise 12. Normal Sampling with Both Parameters Unknown The heights in inches of 20 college women were collected, observing the following measurements: 47 63.5

64 65

61 62

61 64

63 61

61 56

64 63

66 65

63 64

67 59

Suppose one assumes that the Normal mean and precision parameters are independent with µ distributed Normal(62, 1) and φ distributed Gamma with parameters a = 1 and b = 1. measurements <- c(47, 64, 61, 61, 63, 61, 64, 66, 63, 67, 63.5, 65, 62, 64, 61, 56, 63, 65, 64, 59 ) (a) Using the gibbs_normal() function in Section 9.5.3, collect a sample of 5000 from the joint posterior distribution of (µ, φ). s <- list(y = measurements, mu0 = 62, sigma0 = 1, a = 1, b = 1) out <- gibbs_normal(s, P = 0.02, iter = 5000) √ (b) Find a 90% interval estimate for the standard deviation σ = 1/ φ.

168

sigma <- 1 / sqrt(out[, 2]) quantile(sigma, c(0.05, 0.95)) ## 5% 95% ## 3.285780 5.431444 (c) Suppose one is interested in estimating the 90th percentile of the height distribution P90 = µ + 1.645σ.

Collect simulated draws from the posterior of P90 and construct a density estimate.

P90 <- out[, 1] + 1.645 * (1 / sqrt(out[, 2])) ggplot(data.frame(P90 = P90), aes(P90)) + geom_density()

0.3

density

0.2

0.1

0.0 65.0

67.5

70.0

72.5

75.0

P90

Exercise 13. Normal Sampling with Both Parameters Unknown (continued) In Exercise 12, one learned about the mean and precision of the heights by use of a Gibbs sampling algorithm.

Use JAGS and the runjags package to collect MCMC draws from this model. Write a JAGS script for this Normal sampling problem and use the run.jags() function. Answer questions from parts (c) and (d) from Exercise 12. (Note that the sample JAGS script in Section 9.7.1 returns samples of µ and σ.)

The JAGS script for this model is contained in the function JAGS_script() with the “normal” inpout.. library(runjags) the_data <- list(N = 20, y = measurements, mu0 = 62, phi0 = 1, a = 1, b = 1) posterior <- run.jags(JAGS_script("normal"),

169

n.chains = 1, data = the_data, monitor = c("mu", "sigma"), adapt = 1000, burnin = 5000, sample = 5000) ## Compiling rjags model... ## Calling the simulation using the rjags method... ## Note: the model did not require adaptation ## Burning in the model for 5000 iterations... ## Running the model for 5000 iterations... ## Simulation complete ## Calculating summary statistics... ## Finished running the simulation 90% interval estimate for σ: posterior %>% as.mcmc() %>% as.data.frame() -> post quantile(post$sigma, c(0.05, 0.95)) ## 5% 95% ## 3.266391 5.449563 Density estimate of 90th quantile µ + 1.645σ: P90 <- post$mu + 1.645 * post$sigma ggplot(data.frame(P90 = P90), aes(P90)) + geom_density()

0.3

density

0.2

0.1

0.0 67.5

70.0

72.5

P90 170

75.0

Exercise 14. Normal Sampling with Both Parameters Unknown (continued) If one graphs the height data from Exercise 12, one notes that there is one unusually small height value, 47.

One wonders if this minimum height is consistent with the fitted model. (a) Write a function to simulate a sample of size 20 from the posterior predictive distribution. You can use either the gibbs_normal() function in Section 9.5.3 or the JAGS sample script in Section 9.7.1 to generate a sample from the posterior distribution of (µ, φ) or (µ, σ). For each sample, compute the minimum value T (ỹ).

I am using the output from the JAGS simulation stored in the data frame post. pred_check <- function(j){ ys <- rnorm(20, mean = post$mu[j], sd = post$sigma[j]) min(ys) } (b) Repeat the procedure 1000 times, collecting a sample of the predictive distribution of the minimum observation. sample_min <- sapply(1:1000, pred_check) (c) Graph the predictive distribution. From comparing the observed minimum height with this distribution, what can you conclude about the suitability of the model? ggplot(data.frame(Min = sample_min), aes(Min)) + geom_histogram(color = "white", fill = "tan") + geom_vline(xintercept = min(measurements), color = "red")

count

90

60

30

0 45

50

55

60

Min Since the observed mininum value is small relative to the predictive distribution of T (y), there appears to be 171

a problem with the model.

Exercise 15. Comparing Proportions In Section 9.7.4, the problem of comparing proportions of high visits to Facebook from male and female students was considered.

Use the same JAGS script as in the book, but keep tract of the two proportions. library(runjags) modelString = " model{ ## sampling yF ~ dbin(pF, nF) yM ~ dbin(pM, nM) logit(pF) <- theta - lambda / 2 logit(pM) <- theta + lambda / 2 ## priors theta ~ dnorm(mu0, phi0) lambda ~ dnorm(0, phi) } " the_data <- list("yF" = 75, "nF" = 151, "yM" = 39, "nM" = 93, "mu0" = 0, "phi0" = 0.001, "phi" = 2) posterior <- run.jags(modelString, data = the_data, n.chains = 1, monitor = c("pF", "pM"), adapt = 1000, burnin = 5000, sample = 5000) ## Compiling rjags model... ## Calling the simulation using the rjags method... ## Adapting the model for 1000 iterations... ## Burning in the model for 5000 iterations... ## Running the model for 5000 iterations... ## Simulation complete ## Calculating summary statistics... ## Finished running the simulation (a) Using the same prior, use JAGS to take a simulated sample of size 5000 from the posterior of pF and pM . Construct a 90% probability interval estimate for the difference is proportions δ = pW − pM . posterior %>% as.mcmc() %>% as.data.frame() -> post post %>% mutate(delta = pF - pM) %>% summarize(LO = quantile(delta, 0.05), HI = quantile(delta, 0.95)) ## LO HI ## 1 -0.03574132 0.1678118 (b) Use the same simulated sample to perform inferences about the ratio of proportions R = pW /pM .

Construct a density estimate of R and construct a 90% probability interval estimate.

172

posterior %>% as.mcmc() %>% as.data.frame() -> post ggplot(post, aes(pF / pM)) + geom_density()

density

2

1

0 1.0

1.5

pF/pM post %>% mutate(R = pF / pM) %>% summarize(LO = quantile(R, 0.05), HI = quantile(R, 0.95)) ## LO HI ## 1 0.9263371 1.462862

Exercise 16. Comparing Poisson Rates Suppose the number of customers yj arriving at a bank during a half-hour period in the morning is Poisson with mean λM , and the number of customers wj arriving in an afternoon half-hour period is Poisson with mean λA . Suppose one observes the counts 3, 3, 6, 3, 2, 3, 7, 6 for the morning periods, and the counts 11, 3, 9, 10, 10, 5, 8, 7 for the afternoon periods. Assume that λM and λA have independent Gamma(1, 1) priors.

Use JAGS to obtain a simulated sample from the joint posterior of (λM , λA ) and use the output to obtain a 90% posterior interval estimate for the ratio of means R = λA /λM . library(runjags) modelString = " model{ for(i in 1:M){ y[i] ~ dpois(lam_M) } 173

for(i in 1:N){ w[i] ~ dpois(lam_A) } lam_M ~ dgamma(1, 1) lam_A ~ dgamma(1, 1) } " the_data <- list("y" = c(3, 3, 6, 3, 2, 3, 7, 6), "w" = c(11, 3, 9, 10, 10, 5, 8, "M" = 8, "N" = 8) posterior <- run.jags(modelString, data = the_data, n.chains = 1, monitor = c("lam_M", "lam_A"), adapt = 1000, burnin = 5000, sample = 5000)

7),

## Compiling rjags model... ## Calling the simulation using the rjags method... ## Note: the model did not require adaptation ## Burning in the model for 5000 iterations... ## Running the model for 5000 iterations... ## Simulation complete ## Calculating summary statistics... ## Finished running the simulation posterior %>% as.mcmc() %>% as.data.frame() mutate(R = lam_A / lam_M) %>% summarize(LO = quantile(R, 0.05), HI = quantile(R, 0.95))

%>%

## LO HI ## 1 1.340739 2.688935

Exercise 17. Normal Sampling with a Cauchy Prior In Section 9.4, we considered the problem of estimating the mean snowfall amount in Buffalo with a Cauchy prior. The sample mean ȳ is Normal with mean µ and standard error se and µ is Cauchy with location 10 and scale 2. In our problem, ȳ = 26.785 and se = 3.236. Write a JAGS script for this Bayesian model. Use the run.jags() function to simulate 1000 draws of the posterior distribution for µ. Compute the posterior mean and posterior standard deviation for µ. library(runjags) modelString = " model{ y ~ dnorm(mu, phi) mu ~ dt(t_loc, p_scale, 1) } " the_data <- list("y" = 26.785, "phi" = (1 / 3.236) ^ 2, "t_loc" = 10, "p_scale" = (1 / 2) ^ 2) posterior <- run.jags(modelString, data = the_data,

174

n.chains = 1, monitor = c("mu"), adapt = 1000, burnin = 5000, sample = 5000) ## Compiling rjags model... ## Calling the simulation using the rjags method... ## Adapting the model for 1000 iterations... ## Burning in the model for 5000 iterations... ## Running the model for 5000 iterations... ## Simulation complete ## Calculating summary statistics... ## Finished running the simulation posterior %>% as.mcmc() %>% as.data.frame() %>% summarize(post_mean = mean(mu), post_sd = sd(mu)) ## post_mean post_sd ## 1 25.3488 3.514195

Exercise 18. Normal Sampling with a Cauchy Prior (continued) In Exercise 17, one used JAGS to simulate values from the posterior of µ from a single MCMC chain. Instead use two chains with the different starting values of µ = 0 and µ = 50. Run JAGS with two chains and estimate the posterior mean and posterior standard deviation using output from each of the two chains.

Based on the output, comment on the sensitivity of the MCMC run with the choice of the starting value.

Here is the JAGS script and data: library(runjags) modelString = " model{ y ~ dnorm(mu, phi) mu ~ dt(t_loc, p_scale, 1) } " the_data <- list("y" = 26.785, "phi" = (1 / 3.236) ^ 2, "t_loc" = 10, "p_scale" = (1 / 2) ^ 2) We’ll try two diverse starting values, one at µ = 0 and one at µ = 100.

InitialValues <- list(list(mu = 0), list(mu = 100)) posterior <- run.jags(modelString, data = the_data, n.chains = 2, monitor = c("mu"), adapt = 1000, burnin = 5000, sample = 5000) ## Compiling rjags model... ## Calling the simulation using the rjags method... ## Adapting the model for 1000 iterations...

175

## Burning in the model for 5000 iterations... ## Running the model for 5000 iterations... ## Simulation complete ## Calculating summary statistics... ## Calculating the Gelman-Rubin statistic for 1 variables.... ## Finished running the simulation Compare posterior mean and posterior sds for two chains.

Chain 1 with starting value µ = 0. c(mean(posterior$mcmc[[1]]), sd(posterior$mcmc[[1]])) ## [1] 25.367536

3.426401

Chain 2 with starting value µ = 100. c(mean(posterior$mcmc[[2]]), sd(posterior$mcmc[[2]])) ## [1] 25.347397

3.353697

Exercise 19. Bivariate Normal Section 6.7 introduced the Bivariate Normal distribution. Suppose we wish to use Gibbs sampling to simulate from this distribution. In the following assume (X, Y ) is Bivariate Normal with parameters (µX , µY , σX , σY , ρ). (a) Using results from Section 6.7, identify the two conditional distributions f (x | y) and f (y | x) and write down a Gibbs sampling algorithm for simulating from the joint distribution of (X, Y ).

Section 6.7 gives expressions for the conditional distributions [X|Y ] and [Y |X]. (b) Write an R function to simulate a sample from the distribution using Gibbs sampling.

Here is a function to implement Gibbs sampling. gibbs_bn <- function(x = 0, iter = 1000, mu_x, mu_y, s_x, s_y, rho){ xy <- matrix(0, iter, 2) for(k in 1:iter){ mean_y <- mu_y + rho * s_y / s_x * (x - mu_x) sd_y <- s_y * sqrt(1 - rho ^ 2) y <- rnorm(1, mean_y, sd_y) mean_x <- mu_x + rho * s_x / s_y * (y - mu_y) sd_x <- s_x * sqrt(1 - rho ^ 2) x <- rnorm(1, mean_x, sd_x) xy[k, ] <- c(x, y) } xy } (c) Assume µX = 0, µY = 0, σX = 1, σY = 1, ρ = 0.5 and run the simulation for 1000 iterations. Compare the means, standard deviations, and correlation computed from the simulation with the true values of the parameters.

Running the Gibbs sampler for 1000 iterations. out <- gibbs_bn(x = 0, iter = 1000, 0, 0, 1, 1, 0.5) Computing posterior means.

176

apply(out, 2, mean) ## [1] -0.007728359 -0.022740319 Computing posterior standard deviations. apply(out, 2, sd) ## [1] 1.000341 0.995200 Computing correlation. cor(out) ## [,1] [,2] ## [1,] 1.0000000 0.5253451 ## [2,] 0.5253451 1.0000000 (d) Repeat part (c) using the correlation value ρ = 0.99 and again compare the simulation estimates with the true values. Explain why Gibbs sampling does not appear to work as well in this situation.

I am going to use the more extreme value ρ = 0.99, so the conclusion might be more obvious. out <- gibbs_bn(x = 0, iter = 1000, 0, 0, 1, 1, 0.99) Posterior means: apply(out, 2, mean) ## [1] 0.2460595 0.2476703 Posterior stanard deviations: apply(out, 2, sd) ## [1] 0.9756832 0.9760998 Correlation: cor(out) ## [,1] [,2] ## [1,] 1.0000000 0.9881828 ## [2,] 0.9881828 1.0000000 You should see that the summaries from the simulated output are a poorer match to the true values when the value of the correlation is high. When the correlation is high, the Gibbs sampler does not mix well, due to the high autocorrelation structure of the iterates.

Exercise 20. A Normal Mixture Model Consider a three-component mixture distribution, where the density for x has the form f (x) = 0.45 × φ(x, −3, 1/3) + 0.1 × φ(x, 0, 1/3) + 0.45 × φ(x, 3, 1/3), (#eq : normalmix)

(6)

where φ(x, µ, σ) is the Normal density with mean µ and standard deviation σ. Consider the following two ways of simulating from this mixture density.

Approach 1: Monte Carlo: Introduce a “mixture component indicator”, δ, an unobserved latent variable.

The variable z is equal to 1, 2, and 3 with respective probabilities 0.45, 0.1, and 0.45. The density for

177

x conditional on z is normal where [x | z = 1] ∼ Normal(−3, 1/3), [x | z = 2] ∼ Normal(0, 1/3), and [x | z = 3] ∼ Normal(3, 1/3).

One simulates x by first simulating a value of z from its discrete distribution and then simulating a value of x from the corresponding conditional distribution. By repeating this method, one obtains a Monte Carlo simulated sample from the exact mixture distribution.

Approach 2: Gibbs Sampling: An alternative way of simulating from the mixture density is based on Gibbs sampling. Introduce the latent variable z and consider the two conditional distributions [x | z] and [z | x]. The conditional distribution [x | z] will be a Normal density where the Normal parameters depend on the value of the latent variable. The conditional distribution [z | x] is discrete on the values 1, 2, 3 where the probabilities are proportional to 0.45 × φ(x, −3, 1/3), 0.1 × φ(x, 0, 1/3), 0.45 × φ(x, 3, 1/3) respectively.

Write R scripts to use both the Monte Carlo and Gibbs sampling methods to simulate 1000 draws from this mixture density. set.seed(123) m <- c(-3, 0, 3) s <- sqrt(rep(1/3, 3)) mixwt <- c(.45, .10, .45) ### direct sampling # sample a mixture component indicator (1, 2, or 3) # from the marginal posterior distribution on components # given the mixture component, sample mu S <- 1000 # number of MC samples Delta.MC <- sample(1:3, size = S, replace = TRUE, prob = mixwt) Mu.MC <- rnorm(S, m[Delta.MC], s[Delta.MC]) ### Gibbs sampler Mu.MCMC <- Delta.MCMC <- rep(NA, S) delta <- 1 mu <- -6 Delta.MCMC[1] <- delta Mu.MCMC[1] <- mu for (i in 2:S) { delta <- sample(1:3, size = 1, prob = mixwt * dnorm(mu, m, s) / sum(mixwt * dnorm(mu, m, s))) mu <- rnorm(1, m[delta], s[delta])

}

Delta.MCMC[i] <- delta Mu.MCMC[i] <- mu

Exercise 21. A Normal Mixture Model – MCMC Diagnostics Figure 9.21 displays histograms of simulated draws from the mixture distribution using the Monte Carlo and Gibbs sampling algorithms, and the exact mixture density is overlaid on top. It is clear from the figure that the Gibbs sampling does not appear to perform as well as the Monte Carlo method in simulating from this distribution. Using MCMC diagnostic graphs, explore the Gibbs sampling output. Are there particular features in these diagnostic graphs that would indicate problems in the convergence of the Gibbs sampling algorithm?

The two figures in Figure 9.21 show that the 1000 samples of µ from MCMC/Gibbs do not include enough draws from the 2nd component (Normal(0, (1/3)2 ), which should comprise 10% of the total number of draws,

178

as p(δ = 2|y) = 0.1) of the three-component mixture model. At the same time, they contain more samples in the other two components (Normal(-3, (1/3)2 ) and Normal(3, (1/3)2 )). This indicates the potential problems of the MCMC/Gibbs sampler not fully exploring the parameter space of µ.

Comparing the two scatter plots below, we can see similar problems as in Figure 9.21 for Gibbs: fewer than 10% draws of µ around 0, and the sudden jump of draws of µ from around -3 to 3. This again indicates the potential problems of the MCMC/Gibbs sampler not fully exploring the parameter space of µ. df_MC <- as.data.frame(cbind(seq(1:1000), Mu.MC)) names(df_MC) <- c("index", "mu.MC") ggplot(df_MC, aes(x = index, y = mu.MC)) + geom_point(alpha = 0.2) + increasefont() + ggtitle("Monte Carlo")

Monte Carlo 5.0

mu.MC

2.5

0.0

−2.5

−5.0

0

250

500

index df_MCMC <- as.data.frame(cbind(seq(1:1000), Mu.MCMC)) names(df_MCMC) <- c("index", "mu.MCMC") ggplot(df_MCMC, aes(x = index, y = mu.MCMC)) + geom_point(alpha = 0.2) + increasefont() + ggtitle("Markov chain Monte Carlo")

179

750

1000

Markov chain Monte Carlo

mu.MCMC

3

0

−3

−6 0

250

500

750

1000

index The traceplots below show that draws from Monte Carlo approximation it has adequately explored the parameter space (ranging from -4 to 4). However, draws from MCMC show a clear jump from µ around -3 to µ around 0 then to around 3. The MCMC/Gibbs mixing has stickiness issues, and the chain easily gets stuck in the local parameter space. plot(Mu.MC, type = "l", ylab=mu, main="Monte Carlo")

180

2 0 −2 −4

3.66995543701198

4

Monte Carlo

0

200

400

600

800

1000

Index plot(Mu.MCMC, type = "l", ylab=mu, main="Markov Chain Monte Carlo")

2 0 −2 −4 −6

3.66995543701198

4

Markov Chain Monte Carlo

0

200

400

600

800

1000

Index The ACF plots below show close to 0 autocorrelations between adjacent parameter draws for the draws from Monte Carlo approximation. However, the draws from MCMC show very high autocorrelations (still greater than 0.8 after lag 30), indicating severe stickiness in the chain. Draws are highly dependent from one another.

This suggests the possibility of the chain not exploring the entire parameter space yet.

181

acf(Mu.MC, main= "Monte Carlo")

0.4 0.0

0.2

ACF

0.6

0.8

1.0

Monte Carlo

0

5

10

15

20

25

30

25

30

Lag acf(Mu.MCMC, main ="Markov Chain Monte Carlo")

0.4 0.2 0.0

ACF

0.6

0.8

1.0

Markov Chain Monte Carlo

0

5

10

15 Lag

182

20

To improve the MCMC convergence, we should run the MCMC/Gibbs sampler longer, add appropriate length of burnin and perform thinning to reduce the autocorrelations between adjacent parameter draws. We should summarize the posterior of µ using independent parameter draws, which have explored the entire parameter space well. We could also consider the approach in Explaining the Gibbs sampler paper, where one runs many MCMC/Gibbs samplers at different starting values, and only saves the last draw from each chain.

Exercise 22. Change Point Analysis The World Meteorological Association collects data on tropical storms, and scientists want to find out whether the distribution of storms changed over time, and if so, when. Data on the number of storms per year has been collected for n years, and let yi be the number of storms in year i, where i = 1, · · · , n. Let M be the year in which the distribution of Y changes, where M ∈ {1, · · · , n − 1}.

A reasonable sampling model for Y is: yi | λ1 , M

∼

Poisson(λ1 ), i = 1, · · · , M ;

yi | λ2 , M

∼

Poisson(λ2 ), i = M + 1, · · · , n.

Suppose one gives a Uniform prior for M over integers from 1 to n − 1 to represent complete uncertainty about change point: M | λ1 , λ2 ∼ Discrete(

1 1 ,··· , ), M ∈ {1, · · · , n − 1}. n−1 n−1

Equivalently, you can think of the Uniform prior as: P rob(M = m) =

1 , M ∈ {1, · · · , n − 1}. n−1

Recall that Gamma distributions are conjugate prior distributions for Poisson data model. Suppose one uses independent conjugate Gamma priors for λ1 and λ2 : λ1 | a1 , b1

∼ Gamma(a1 , b1 ),

λ2 | a2 , b2

∼ Gamma(a2 , b2 ).

(a) Write the joint posterior distribution, π(λ1 , λ2 , M | y1 , · · · , yn ), up to a constant.

• The likelihood: L(λ1 , λ2 , M )

=

M Y λyi e−λ1

n Y

λy2i e−λ2 yi !

(7)

λ2 M +1 e−(M λ1 +(n−M )λ2 )

(8)

1

yi !

i=1

PM ∝ λ1

i=1

yi

i=M +1

Pn

yi

• The joint prior: π(λ1 , λ2 , M ) ∝ λa1 1 −1 e−b1 λ1 λa2 2 −1 e−b2 λ2

(9)

• The joint posterior: π(λ1 , λ2 , M | y1 , · · · , yn )

Pn PM yi yi ∝ λ1 i=1 λ2 M +1 e−(M λ1 +(n−M )λ2 ) λa1 1 −1 e−b1 λ1 λ2a2 −1 e−b2 λ2 (10) Pn PM yi +a2 −1 yi +a1 −1 −(M +b )λ i=1 1 1 = λ1 e λ2 M +1 e−(n−M +b2 )λ2 (11)

(b) Find the full conditional posterior distribution for λ1 and λ2 . Write the name of the distributions and expressions for their parameter values.

183

• Full condtional posterior for λ1 : PM π(λ1 | λ2 , M, y1 , · · · , yn ) ∝ λ1 i=1

yi +a1 −1 −(M +b )λ 1 1

e

,

(12)

which corresponds to λ1 | λ2 , M, y1 , · · · , yn ∼ Gamma(

M X

yi + a1 , M + b1 ).

i=1

• Full condtional posterior for λ2 : Pn yi +a2 −1 e−(n−M +b2 )λ2 , π(λ2 | λ1 , M, y1 , · · · , yn ) ∝ λ2 M +1

(13)

which corresponds to λ2 | λ1 , M, y1 , · · · , yn ∼ Gamma(

n X

yi + a2 , n − M + b1 ).

i=M +1

(c) Find the full conditional posterior distribution for M , which should be a discrete distribution over m = 1, · · · , n − 1.

Pn PM yi yi i=1 π(M | λ1 , λ2 , y1 , · · · , yn ) ∝ λ1 λ2 M +1 e−M (λ1 −λ2 ) We can then use Bayes rule for discrete variables, M = 1, · · · , n. (d) Describe how you would design a Gibbs sampling to simulate posterior draws of the set of parameters, (λ1 , λ2 , M ).

Open-ended.

Chapter 10: Bayesian Hierarchical Modeling Exercise 1. Time-to-Serve for Six Tennis Players Table 10.3 displays the sample size nj and the mean time-to-serve ȳj (in seconds) for six professional tennis players. Assume that the sample mean for the i-th player ȳi is normally distributed with mean µi and √ standard deviation σ/ ni where we assume σ = 5.5 seconds.

Table 10.3. Number of serves and mean time-to-serve for six professional tennis players.

Player

n

ȳ

Murray Simon Federer Ferrer Isner Kyrgios

731 570 491 456 403 274

23.56 18.07 16.21 21.70 22.32 14.11

(a) (Separate estimate) Suppose one is interested in estimating Murray’s mean time-to-serve µ1 using only Murray’s time-to-serve data. Assume that one’s prior beliefs about µ1 are represented by a normal density with mean 20 and standard deviation 10 seconds. Use results from Chapter 8 to find the

184

posterior distribution of µ1 and construct a 90% interval estimate for µ1 .

The posterior distribution of µ1 is Normal(23.559, 0.203) based on the calculation below. ybar <- 23.56 n <- 731 mu_0 <- 20 phi_0 <- 1/10^2 sigma <- 5.5 phi <- 1/sigma^2 mu_n <- (phi_0*mu_0 + n*phi*ybar)/(phi_0 + n*phi) sigma_n <- sqrt(1/(phi_0 + n*phi)) mu_n ## [1] 23.55853 sigma_n ## [1] 0.2033828 A central 90% interval estimate is below. qnorm(c(0.05, 0.95), 23.559, 0.203) ## [1] 23.22509 23.89291 (b) (Combined estimate) Suppose instead that one believes that there are no differences between players and µ1 = ... = µ6 = µ. The overall mean time-to-serve is ȳ = 19.9 seconds with a combined sample size n = 2925. Assuming that µ has a Normal(20,10) prior, find the posterior distribution of µ and construct a 90% interval estimate for µ.

The posterior distribution of µ1 is Normal(19.900, 0.102) based on the calculation below. ybar <- 19.9 n <- 2925 mu_0 <- 20 phi_0 <- 1/10^2 sigma <- 5.5 phi <- 1/sigma^2 mu_n <- (phi_0*mu_0 + n*phi*ybar)/(phi_0 + n*phi) sigma_n <- sqrt(1/(phi_0 + n*phi)) mu_n ## [1] 19.90001 sigma_n ## [1] 0.1016898 A central 90% interval estimate is below. qnorm(c(0.05, 0.95), 19.900, 0.102) ## [1] 19.73222 20.06778 (c) Which approach, part (a) or part (b), seems more reasonable in this situation? Explain.

Part (a) seems more reseaonable in this situation because Murray’s sample size is large, and the ȳ values seem vary a lot across players.

185

Exercise 2. Time-to-Serve for Six Tennis Players (continued) Suppose one wishes to estimate the mean time-to-serve values for the six players by the following hierarchical model. Remember that we are assuming σ = 5.5 seconds.

ȳi

∼

√ Normal(µi , σ/ ni ), i = 1, ..., 6.

µi

∼

Normal(µ, τ ), i = 1, ..., 6.

µ ∼

Normal(20, 1/0.001),

2

Gamma(0.1, 0.1).

1/τ

∼

(a) Use JAGS to simulate a sample of size 1000 from the posterior distribution from this hierarchical model, storing values of the means µ1 , ..., µ6 . ## write the model modelString <-" model { ## sampling for (i in 1:N){ ybar[i] ~ dnorm(mu_i[i], invsigma2[i]) } ## priors for (i in 1:N){ mu_i[i] ~ dnorm(mu, invtau2) } ## hyperpriors mu ~ dnorm(mu0, g0) invtau2 ~ dgamma(a_t, b_t) tau <- sqrt(pow(invtau2, -1)) } " ybar <- c(23.56, 18.07, 16.21, 21.70, 22.32, 14.11) N <- length(ybar) sigma <- c(5.5/sqrt(731), 5.5/sqrt(570), 5.5/sqrt(491), 5.5/sqrt(456), 5.5/sqrt(403), 5.5/sqrt(274)) invsigma2 <- 1/sigma^2 initsfunction <- function(chain){ .RNG.seed <- c(1,2)[chain] .RNG.name <- c("base::Super-Duper", "base::Wichmann-Hill")[chain] return(list(.RNG.seed=.RNG.seed, .RNG.name=.RNG.name)) } the_data <- list("ybar" = ybar, "N" = N, "invsigma2" = invsigma2, "mu0" = 20, "g0" = 1/0.001, "a_t" = 0.1, "b_t" = 0.1)

186

posterior <- run.jags(modelString, n.chains = 1, data = the_data, monitor = c("mu", "tau", "mu_i"), adapt = 1000, burnin = 5000, sample = 5000, inits = initsfunction) ## Compiling rjags model... ## Calling the simulation using the rjags method... ## Note: the model did not require adaptation ## Burning in the model for 5000 iterations... ## Running the model for 5000 iterations... ## Simulation complete ## Calculating summary statistics... ## Finished running the simulation print(posterior, digits = 3) ## ## JAGS model summary statistics from 5000 samples (adapt+burnin = 6000): ## ## Lower95 Median Upper95 Mean SD Mode MCerr MC%ofSD SSeff AC.10 ## mu 19.9 20 20.1 20 0.0319 -- 0.000475 1.5 4510 -0.00544 ## tau 2.02 3.66 6.62 3.95 1.39 -0.0197 1.4 5000 0.0201 ## mu_i[1] 23.1 23.5 23.9 23.5 0.201 -- 0.00284 1.4 5000 -0.0206 ## mu_i[2] 17.6 18.1 18.5 18.1 0.23 -- 0.00325 1.4 5000 0.00362 ## mu_i[3] 15.7 16.2 16.7 16.2 0.247 -- 0.00311 1.3 6317 0.0181 ## mu_i[4] 21.2 21.7 22.2 21.7 0.259 -- 0.00366 1.4 5000 -0.0158 ## mu_i[5] 21.8 22.3 22.8 22.3 0.274 -- 0.00373 1.4 5374 0.0205 ## mu_i[6] 13.6 14.2 14.9 14.2 0.332 -0.0047 1.4 5000 -0.0231 ## ## psrf ## mu -## tau -## mu_i[1] -## mu_i[2] -## mu_i[3] -## mu_i[4] -## mu_i[5] -## mu_i[6] -## ## Total time taken: 0.2 seconds (b) Construct a 90% interval estimate for each of the means.

A 90% interval estimate for each µi can be obtained from the printout above. (c) Compare the 90% estimate for Murray with the separate and combined interval estimates from Exercise 1.

The 90% interval estimate for Murray from the hierarchical model is closer to that of the separate interval estimate from Exercise 1.

187

Exercise 3. Random σj ’s for Movie Ratings In Section 10.2.2, consider the situation where the standard deviation of the ratings differ across movies, so σj represents the standard deviation of the ratings for movie j. (a) Write out the likelihood, the prior distributions, and hyperprior distributions for this varying means and varying standard deviations model.

• Sampling: for j = 1, · · · , 8 and i = 1, · · · , nj : i.i.d.

Yij | µj , σ ∼ Normal(µj , σj ).

(14)

• Prior for µj , Stage 1: µj , j = 1, · · · , 8: µj | µ, τ ∼ Normal(µ, τ ).

(15)

µ, τ ∼ π(µ, τ ).

(16)

1/σj2 | a, b ∼ Gamma(a, b).

(17)

a, b ∼ π(a, b)

(18)

• Prior for µj , Stage 2:

• Prior for σj , Stage 1: σj , j = 1, · · · , 8:

• Prior for σj , Stage 2: (b) Discuss the implications of specifying varying σj ’s by comparing this hierarchical model to the developed model in Section 10.2.2.

In this varying σj ’s hierarchical model, we assume that all σj ’s share the same gamma prior distribution, which means that all σj ’s are related somehow, though they are not exactly the same. (c) What prior distributions do you choose for σj ’s? Why? We could give σj ’s a commom gamma prior, for example Gamma(1, 1). (d) Carry out the simulation by MCMC using JAGS. Report and discuss the findings.

MovieRatings <- animation_ratings MovieRatings %>% mutate(Title = as.character(title), Title = recode(Title, "Shrek Forever After (a.k.a. Shrek: The Final Chapter) (2010)" = "Shrek Forever", "How to Train Your Dragon (2010)" = "Dragon", "Toy Story 3 (2010)" = "Toy Story 3", "Tangled (2010)" = "Tangled", "Despicable Me (2010)" = "Despicable Me", "Legend of the Guardians: The Owls of Ga'Hoole (2010)" = "Guardians", "Megamind (2010)" = "Megamind", "Batman: Under the Red Hood (2010)" = "Batman")) -> MovieRatings ## write the model modelString <-" model {

188

## sampling for (i in 1:N){ y[i] ~ dnorm(mu_j[MovieIndex[i]], invsigma2_j[MovieIndex[i]]) } ## priors for (j in 1:J){ mu_j[j] ~ dnorm(mu, invtau2) invsigma2_j[j] ~ dgamma(a_g, b_g) sigma_j[j] <- sqrt(pow(invsigma2_j[j], -1)) } ## hyperpriors mu ~ dnorm(mu0, g0) invtau2 ~ dgamma(a_t, b_t) tau <- sqrt(pow(invtau2, -1)) } " y <- MovieRatings$rating # The y values are in the column named rating MovieIndex <- MovieRatings$Group_Number # The MovieIndex is in the column named Group_Number N <- length(y) # Compute the number of observations J <- length(unique(MovieIndex)) # Compute the number of movies initsfunction <- function(chain){ .RNG.seed <- c(1,2)[chain] .RNG.name <- c("base::Super-Duper", "base::Wichmann-Hill")[chain] return(list(.RNG.seed=.RNG.seed, .RNG.name=.RNG.name)) } the_data <- list("y" = y, "MovieIndex" = MovieIndex, "N" = N, "J" = J, "mu0" = 3, "g0" = 1, "a_t" = 1, "b_t" = 1, "a_g" = 1, "b_g" = 1) posterior <- run.jags(modelString, n.chains = 1, data = the_data, monitor = c("mu", "tau", "mu_j", "sigma_j"), adapt = 1000, burnin = 5000, sample = 5000, inits = initsfunction) ## Compiling rjags model... ## Calling the simulation using the rjags method... ## Note: the model did not require adaptation ## Burning in the model for 5000 iterations... ## Running the model for 5000 iterations...

189

## Simulation complete ## Calculating summary statistics... ## Finished running the simulation print(posterior, digits = 3) ## ## JAGS model summary statistics from 5000 samples (adapt+burnin = 6000): ## ## Lower95 Median Upper95 Mean SD Mode MCerr MC%ofSD SSeff ## mu 3.13 3.77 4.33 3.76 0.304 -- 0.00632 2.1 2307 ## tau 0.352 0.64 1.1 0.685 0.215 -- 0.00418 1.9 2645 ## mu_j[1] 2.92 3.46 3.98 3.47 0.265 -- 0.00387 1.5 4685 ## mu_j[2] 3.35 3.8 4.27 3.8 0.235 -- 0.00342 1.5 4693 ## mu_j[3] 2.79 3.88 4.88 3.86 0.528 -- 0.00843 1.6 3918 ## mu_j[4] 3.23 3.72 4.19 3.73 0.241 -- 0.0034 1.4 5000 ## mu_j[5] 2.75 4.1 5.39 4.09 0.672 -- 0.0129 1.9 2701 ## mu_j[6] 2.57 3.85 4.95 3.83 0.605 -- 0.0101 1.7 3551 ## mu_j[7] 2.6 3.56 4.54 3.56 0.493 -- 0.00816 1.7 3647 ## mu_j[8] 3.52 4.12 4.67 4.11 0.291 -- 0.00447 1.5 4250 ## sigma_j[1] 0.613 0.904 1.37 0.938 0.211 -0.003 1.4 4942 ## sigma_j[2] 0.687 0.988 1.38 1.01 0.185 -- 0.00273 1.5 4576 ## sigma_j[3] 0.602 1.21 2.45 1.35 0.587 -- 0.00912 1.6 4143 ## sigma_j[4] 0.471 0.729 1.15 0.764 0.192 -- 0.00289 1.5 4406 ## sigma_j[5] 0.399 1.14 3.15 1.42 1.11 -- 0.0182 1.6 3749 ## sigma_j[6] 0.393 0.995 2.67 1.22 0.88 -0.013 1.5 4556 ## sigma_j[7] 0.611 1.24 2.28 1.35 0.512 -- 0.00783 1.5 4274 ## sigma_j[8] 0.601 0.937 1.43 0.975 0.229 -- 0.00357 1.6 4128 ## ## AC.10 psrf ## mu 0.00834 -## tau -0.0223 -## mu_j[1] 0.00564 -## mu_j[2] 0.00861 -## mu_j[3] 0.0149 -## mu_j[4] -0.0116 -## mu_j[5] 0.00596 -## mu_j[6] 0.0021 -## mu_j[7] 0.00875 -## mu_j[8] 0.0136 -## sigma_j[1] 0.00419 -## sigma_j[2] 0.0112 -## sigma_j[3] 0.0197 -## sigma_j[4] -0.014 -## sigma_j[5] -0.0134 -## sigma_j[6] -0.0118 -## sigma_j[7] 0.00827 -## sigma_j[8] 0.00099 -## ## Total time taken: 0.4 seconds The posterior inferences for µ, τ and µj ’s seem close to that in the fixed σ model in Section 10.2.2. However, the inferences for σj ’s in the current varying σj model seem to suggest that the σj ’s are different from each other to some degree, which supports the rationale of having varying σj ’s in the model in the first place.

190

Exercise 4. Smoothing Counts A general issue in statistical inference is how to handle situations where there are zero observed counts in a sample. This exercise illustrates several Bayesian modeling approaches to this problem. (a) Suppose one is learning about the probability p a particular player successively makes a three-point shot in basketball. One assigns a uniform prior for p. This player attempts 10 shots and one observes y = 0 successes. Derive the posterior density of p and compute the posterior mean.

Given the beta-binomial conjugate model results in Chapter 7, that if p has prior of Beta(a, b), and the data model is Binomial(n, p), then the posterior of p is Beta(a + y, b + n − y). With a = b = 1 and n = 10, y = 0, the posterior density of p is Beta(1, 11), and its posterior mean is approximated below.

BetaSamples <- rbeta(1000, 1, 11) mean(BetaSamples) ## [1] 0.08110759 (b) Suppose that one is learning about the probabilities p1 , p2 , p3 , p4 , p5 of five players making three-throw shots. You assign the following hierarchical prior on the probabilities. You assume p1 , ...., p5 are independent identically distributed beta with shape parameters α and α, and at the second stage, you assign α a uniform prior on (0, 1). Write down a graphical representation of this hierarchical model.

Open-ennded. (c) In part (b), suppose that each player attempts 10 shots and you observe 0, 2, 3, 1, 3 successes for the five players. Use JAGS to obtain posterior samples from the parameters α, p1 , p2 , p3 , p4 , p5 . Compute the posterior means of α and p1 and compare your probability estimates with the estimate of p using the single-stage prior in part (a). ## write the model modelString <-" model { ## sampling for (i in 1:N){ y[i] ~ dbin(p[i], n[i]) } ## priors for (i in 1:N){ p[i] ~ dbeta(alpha, alpha) } ## hyperpriors alpha ~ dunif(alpha_a, alpha_b) } " y <- c(0, 2, 3, 1, 3) N <- length(y) n <- rep(10, N) initsfunction <- function(chain){ .RNG.seed <- c(1,2)[chain] .RNG.name <- c("base::Super-Duper", 191

}

"base::Wichmann-Hill")[chain] return(list(.RNG.seed=.RNG.seed, .RNG.name=.RNG.name))

the_data <- list("y" = y, "N" = N, "n" = n, "alpha_a" = 0, "alpha_b" = 1) posterior <- run.jags(modelString, n.chains = 1, data = the_data, monitor = c("p", "alpha"), adapt = 1000, burnin = 5000, sample = 5000, inits = initsfunction) ## Compiling rjags model... ## Calling the simulation using the rjags method... ## Adapting the model for 1000 iterations... ## Burning in the model for 5000 iterations... ## Running the model for 5000 iterations... ## Simulation complete ## Calculating summary statistics... ## Finished running the simulation print(posterior, digits = 3) ## ## JAGS model summary statistics from 5000 samples (adapt+burnin = 6000): ## ## Lower95 Median Upper95 Mean SD Mode MCerr MC%ofSD SSeff AC.10 ## p[1] 5.85e-08 0.0323 0.188 0.0556 0.0626 -- 0.00236 3.8 702 0.0568 ## p[2] 0.0335 0.219 0.481 0.236 0.124 -- 0.00258 2.1 2301 -0.00791 ## p[3] 0.0836 0.316 0.59 0.328 0.134 -- 0.00256 1.9 2742 0.0143 ## p[4] 0.000777 0.126 0.343 0.148 0.1 -- 0.00244 2.4 1689 -0.015 ## p[5] 0.0807 0.311 0.58 0.323 0.133 -- 0.00245 1.8 2927 -0.0153 ## alpha 0.291 0.672 1 0.658 0.211 -- 0.00515 2.4 1679 0.0031 ## ## psrf ## p[1] -## p[2] -## p[3] -## p[4] -## p[5] -## alpha -## ## Total time taken: 0.4 seconds The posterior output shows a posterior mean of p1 of 0.0556, which is lower than that of part (a). The posterior mean of α is 0.658.

192

Exercise 5. Schedules and Producers in Korean Drama Ratings The Korean entertainment industry has been continuously booming. The global audience for K-drama is exploding across Asia and even spreading to other parts of the world, notably the US and Europe. This surge of Korean cultural popularity is called “Hallyu”, literally meaning the “Korean wave”. K-dramas are popular on multiple streaming websites in the US, such as Hulu, DramaFever, and even Netflix.

How are K-dramas being rated in Korea? How are the producing company and broadcasting schedule affecting the drama ratings? In one study, data were collected on 101 K-dramas from 2014 to 2016. Each drama was produced by one of the three main producers/companies, and was being broadcasted in one of four different times of the week. The ratings of dramas were collected from the AGB Nielsen Media Research Group. In particular, the national AGB TV ratings of each drama were recorded.

The data is stored in KDramaData.csv. Table 10.4 provides information about the variables in the complete dataset.

Table 10.4. Table of the variables in K-dramas application.

Name

Variable information

Drama Name Schedule

The name of the drama 1 = Mon. & Tue., 2 = Wed. & Thu., 3 = Fri., 4 = Sat. & Sun. 1 = Seoul Broadcasting System (SBS), 2 = Korean Broadcasting System (KBS), 3 = Munhwa Broadcasting Corporation (MBC) AGB national TV ratings, in percentage MM/DD/YY

Producer

Viewership Date

(a) Explore the ratings graphically by schedule and by producer. dramadata <- KDramaData dramadata$Schedule <- as.factor(dramadata$Schedule) dramadata$Producer <- as.factor(dramadata$Producer) ggplot(dramadata, aes(Rating, color = Schedule)) + geom_density() + labs(title = "Density plot of ratings by schedule") + xlim(0, 0.3) + theme_grey(base_size = 10, base_family = "")

193

Density plot of ratings by schedule 50

40

Schedule 30

density

1 2 3

20 4

10

0 0.0

0.1

0.2

0.3

Rating ggplot(dramadata, aes(Rating, color = Producer)) + geom_density() + labs(title = "Density plot of ratings by producer") + xlim(0, 0.3) + theme_grey(base_size = 10, base_family = "")

194

Density plot of ratings by producer 10.0

7.5

density

Producer 1 5.0

2 3

2.5

0.0 0.0

0.1

0.2

0.3

Rating (b) Explain how the ratings differ by schedule and by producer. Are there particular days when the ratings are high or low? Does one producer tend to have larger ratings than the other producer?

The graphs in part (a) show a wider range of variability in ratings by schedule than that by producer.

Friday dramas seem to have the lowest ratings with small variation.

SBS dramas have medium mean rating with smaller variation. KBS dramas have smallest mean rating but the ratings vary more. MBC dramas have highest mean rating with small variation. (c) Choose a subset of the KDramaData.csv dataset for a particular producer. Develop a hierarchical model to make inference about the mean ratings of dramas across different schedules. Discuss your conclusions and the advantage of using hierarchical modeling in this situation.

Working with KBS dramas.

KBSdrama <- dramadata[dramadata$Producer == 2,] KBSdrama$Schedule <- as.factor(KBSdrama$Schedule) modelString <-" model { ## likelihood for (i in 1:N){ y[i] ~ dnorm(mu_j[schedule[i]], invsigma2) } ## priors for (j in 1:J){ mu_j[j] ~ dnorm(mu, invtau2) } invsigma2 ~ dgamma(a_g, b_g)

195

sigma <- sqrt(pow(invsigma2, -1)) ## hyperpriors mu ~ dnorm(mu0, 1/g0^2) invtau2 ~ dgamma(a_t, b_t) tau <- sqrt(pow(invtau2, -1)) } " y <- KBSdrama$Rating schedule <- KBSdrama$Schedule N <- length(y) J <- length(unique(schedule)) initsfunction <- function(chain){ .RNG.seed <- c(1,2)[chain] .RNG.name <- c("base::Super-Duper", "base::Wichmann-Hill")[chain] return(list(.RNG.seed=.RNG.seed, .RNG.name=.RNG.name)) } the_data <- list("y" = y, "schedule" = schedule, "N" = N, "J" = J, "mu0" = 0.1, "g0" = 0.5, "a_t" = 1, "b_t" = 1, "a_g" = 1, "b_g" = 1) posterior <- run.jags(modelString, n.chains = 1, data = the_data, monitor = c("mu", "tau", "mu_j", "sigma"), adapt = 1000, burnin = 5000, sample = 5000, thin = 1, inits = initsfunction) ## Compiling rjags model... ## Calling the simulation using the rjags method... ## Note: the model did not require adaptation ## Burning in the model for 5000 iterations... ## Running the model for 5000 iterations... ## Simulation complete ## Calculating summary statistics... ## Finished running the simulation summary(posterior) ## Lower95 Median Upper95 Mean SD Mode ## mu -0.49047454 0.10801400 0.6682304 0.10472246 0.28838370 NA ## tau 0.35267052 0.65852607 1.2499367 0.72061727 0.27488885 NA ## mu_j[1] -0.07196554 0.07132613 0.2167506 0.07266280 0.07238919 NA ## mu_j[2] -0.05688244 0.09944587 0.2532603 0.09928626 0.07995014 NA ## mu_j[3] -0.24146558 0.04479396 0.3491075 0.04271370 0.15109887 NA ## mu_j[4] -0.02481824 0.19140944 0.3991133 0.19242821 0.10745635 NA ## sigma 0.20106992 0.26158432 0.3326434 0.26503794 0.03460242 NA ## MCerr MC%ofSD SSeff AC.10 psrf

196

## mu 0.0041809868 ## tau 0.0041977744 ## mu_j[1] 0.0010237378 ## mu_j[2] 0.0011306657 ## mu_j[3] 0.0020734837 ## mu_j[4] 0.0015196623 ## sigma 0.0005535073

1.4 1.5 1.4 1.4 1.4 1.4 1.6

4758 -0.010518456 4288 -0.015113465 5000 0.002230488 5000 0.018814005 5310 -0.007974257 5000 -0.017268731 3908 0.009879697

NA NA NA NA NA NA NA

Ind_Stats = as.data.frame(matrix(NA, J, 2)) names(Ind_Stats) = c("mean", "sd") for (j in 1:J){ Ind_Stats[j, ] = c(mean(KBSdrama$Rating[KBSdrama$Schedule == j]), sd(KBSdrama$Rating[KBSdrama$Schedule == j])) } Post_Means <- summary(posterior)[, 4] Means1 <- data.frame(Type = "Sample", Mean = Ind_Stats$mean) Means2 <- data.frame(Type = "Hierarchical", Mean = Post_Means[3:(4 + J - 2)]) Means1$Title <- c("Schedule 1", "Schedule 2", "Schedule 3", "Schedule 4") Means2$Title <- c("Schedule 1", "Schedule 2", "Schedule 3", "Schedule 4") ggplot(rbind(Means1, Means2), aes(Type, Mean, group=Title)) + geom_line(color = "blue") + geom_point() + annotate(geom = "text", x = 0.75, y = Means1$Mean + c(0.01, 0.01, 0.01, -0.01), size = 3, label = Means1$Title) + increasefont(Size = 10)

197

0.20

Schedule 4

Mean

0.15

Schedule 2 0.10

Schedule 1

0.05

Schedule 3

Sample

Hierarchical

Type The hierarchical model has certain shrinkage effects on the mean rating by schedule, though not very obvious.

Hierarchical models could be more beneficial when sample sizes are even smaller.

Exercise 6 Homework Hours for Five Schools To compare weekly hours spent on homework by students, data is collected from a sample of five different schools. The data is stored in HWhours5schools.csv. (a) Explore the weekly hours spent on homework by students from the five schools. Do the school-specific means seem significantly different from each other? What about their variances?

HWhoursdata <- HWhours5schools HWhoursdata$school <- as.factor(HWhoursdata$school) ggplot(HWhoursdata, aes(hours, color = school)) + geom_density() + labs(title = "Density plot of homework hours by school") + theme_grey(base_size = 10, base_family = "")

198

Density plot of homework hours by school 0.12

0.09

school density

1 2

0.06

3 4 5 0.03

0.00 0

5

10

hours table(HWhoursdata$school) ## ## 1 2 3 4 5 ## 25 23 20 24 24 tapply(HWhoursdata$hours, HWhoursdata$school, summary) ## $`1` ## Min. 1st Qu. ## 1.550 7.440 ## ## $`2` ## Min. 1st Qu. ## 0.290 3.405 ## ## $`3` ## Min. 1st Qu. ## 0.950 5.130 ## ## $`4` ## Min. 1st Qu. ## 0.570 2.292 ## ## $`5` ## Min. 1st Qu. ## 4.27 8.21

Median 9.610

Mean 3rd Qu. 9.464 11.650

Max. 16.380

Median 6.540

Mean 3rd Qu. 7.033 10.070

Max. 15.350

Median 6.870

Mean 3rd Qu. 7.953 11.508

Max. 13.630

Median 6.485

Mean 3rd Qu. 6.232 8.473

Max. 14.170

Median 10.51

Mean 3rd Qu. 10.77 13.75

Max. 17.47

199

15

tapply(HWhoursdata$hours, HWhoursdata$school, sd) ## 1 2 3 4 5 ## 3.885404 4.484783 3.781892 4.102477 3.659042 The mean homework hours from these 5 schools seem to differ from each other. The smallest is school 4 with 6.23 hours and the largest is school 5 with 10.77 hours. School 2 has the largest sample standard deviation and school 5 has the smallest sample standard deviation. (b) Set up a hierarchical model with common and unknown σ in the likelihood, as in Section 10.2.2. Write out the likelihood, the prior distributions and the hyperprior distributions.

• Sampling: for j = 1, · · · , 5 and i = 1, · · · , nj : i.i.d.

Yij | µj , σ ∼ Normal(µj , σj ).

(19)

• Prior for µj , Stage 1: µj , j = 1, · · · , 5: µj | µ, τ ∼ Normal(µ, τ ).

(20)

µ, τ ∼ π(µ, τ ).

(21)

1/σ 2 | a, b ∼ Gamma(a, b).

(22)

• Prior for µj , Stage 2:

• Prior for σ: (c) Use JAGS to obtain posterior samples of the parameters in the hierarchical model. Perform appropriate MCMC diagnostics. ## write the model modelString <-" model { ## sampling for (i in 1:N){ y[i] ~ dnorm(mu_j[School[i]], invsigma2) } ## priors for (j in 1:J){ mu_j[j] ~ dnorm(mu, invtau2) } invsigma2 ~ dgamma(a_g, b_g) sigma <- sqrt(pow(invsigma2, -1)) ## hyperpriors mu ~ dnorm(mu0, g0) invtau2 ~ dgamma(a_t, b_t) tau <- sqrt(pow(invtau2, -1)) } " y <- HWhoursdata$hours School <- HWhoursdata$school N <- length(y) 200

J <- length(unique(School)) initsfunction <- function(chain){ .RNG.seed <- c(1,2)[chain] .RNG.name <- c("base::Super-Duper", "base::Wichmann-Hill")[chain] return(list(.RNG.seed=.RNG.seed, .RNG.name=.RNG.name)) } the_data <- list("y" = y, "School" = School, "N" = N, "J" = J, "mu0" = 3, "g0" = 1, "a_t" = 1, "b_t" = 1, "a_g" = 1, "b_g" = 1) posterior <- run.jags(modelString, n.chains = 1, data = the_data, monitor = c("mu", "tau", "mu_j", "sigma"), adapt = 1000, burnin = 5000, sample = 5000, inits = initsfunction) ## Compiling rjags model... ## Calling the simulation using the rjags method... ## Note: the model did not require adaptation ## Burning in the model for 5000 iterations... ## Running the model for 5000 iterations... ## Simulation complete ## Calculating summary statistics... ## Finished running the simulation print(posterior, digits = 3) ## ## JAGS model summary statistics from 5000 samples (adapt+burnin = 6000): ## ## Lower95 Median Upper95 Mean SD Mode MCerr MC%ofSD SSeff AC.10 ## mu 2.52 4.6 6.87 4.59 1.12 -- 0.0274 2.4 1673 -0.0343 ## tau 1.08 3.46 6.85 3.76 1.64 -- 0.0399 2.4 1683 0.000216 ## mu_j[1] 7.52 9.17 10.7 9.17 0.796 -- 0.0122 1.5 4237 -0.0112 ## mu_j[2] 5.23 6.88 8.41 6.88 0.81 -- 0.0115 1.4 5000 0.0176 ## mu_j[3] 6.01 7.74 9.4 7.74 0.857 -- 0.0125 1.5 4691 -0.0164 ## mu_j[4] 4.57 6.14 7.7 6.16 0.801 -- 0.0111 1.4 5206 0.00437 ## mu_j[5] 8.75 10.4 12 10.4 0.842 -- 0.0142 1.7 3494 0.0116 ## sigma 3.5 3.97 4.56 3.99 0.27 -- 0.00395 1.5 4697 0.0109 ## ## psrf ## mu -## tau -## mu_j[1] --

201

## mu_j[2] -## mu_j[3] -## mu_j[4] -## mu_j[5] -## sigma -## ## Total time taken: 0.2 seconds plot(posterior, vars = "tau")

ECDF

2 4 6 8 10

tau

## Generating plots...

1.0 0.8 0.6 0.4 0.2 0.0

6000 7000 8000 9000 10000 11000

0

5

6 4 2 0 0

5

10

15

tau

15

Autocorrelation of tau

% of total

Iteration

10

tau

1.0 0.5 0.0 −0.5 −1.0 0

5

10 15 20 25 30 35

Lag

(d) Compute posterior means and 95% credible intervals for every school mean. Compute the posterior probability that the mean hour in school 1 is higher than that of school 2. Discuss your findings.

The posterior means and 95% credible intervals for every school mean are below. The posterior probability that the mean hour in school 1 is higher than that of school 2 is 0.977, a very high value. post <- as.mcmc(posterior) quantile(post[, "mu_j[1]"], c(0.025, 0.5, 0.975)) ## ##

2.5% 7.557759

50% 97.5% 9.169736 10.713218

quantile(post[, "mu_j[2]"], c(0.025, 0.5, 0.975)) ## 2.5% 50% 97.5% ## 5.275329 6.883937 8.482458 quantile(post[, "mu_j[3]"], c(0.025, 0.5, 0.975)) ## 2.5% 50% 97.5% ## 6.046316 7.739924 9.439782

202

quantile(post[, "mu_j[4]"], c(0.025, 0.5, 0.975)) ## 2.5% 50% 97.5% ## 4.588712 6.140008 7.730683 quantile(post[, "mu_j[5]"], c(0.025, 0.5, 0.975)) ## ##

2.5% 50% 97.5% 8.693433 10.380389 11.950294

sum(post[, "mu_j[1]"] > post[, "mu_j[2]"])/5000 ## [1] 0.977 2

(e) Compute and summarize the posterior distribution of the ratio R = τ 2τ+σ2 . Comment on the evidence of between-school variability for this data.

The values of ratio R are around 0.5, indicating a similar level of between-school variability for this data compared to that of within-school variability. tau_draws <- as.mcmc(posterior, vars = "tau") sigma_draws <- as.mcmc(posterior, vars = "sigma") R <- tau_draws^2/(tau_draws^2 + sigma_draws^2) df <- as.data.frame(R) quantile(R, c(0.025, 0.975)) ## 2.5% 97.5% ## 0.1197314 0.7941746 ggplot(df, aes(x=R)) + geom_density() + labs(title="Density of R") + theme(plot.title = element_text(size=15)) + theme(axis.title = element_text(size=15))

203

Density of R 2.0

density

1.5

1.0

0.5

0.0 0.00

0.25

0.50

0.75

1.00

R Exercise 7. Heart Attack Deaths - New York City In Section 10.3, the heart attack deaths dataset of thirteen hospitals in Manhattan, New York City is analyzed using a hierarchical beta-binomial model. A complete dataset of heart attack death information of 45 hospitals in all 5 boroughs of New York City (Manhattan, the Bronx, Brooklyn, Queens, and Staten Island) is stored in DeathHeartAttackDataNYCfull.csv. Table 10.5 lists the variables and their description.

Table 10.5. The list of variables in the New York City hearth attack deaths dataset and their description.

Variable

Description

Hospital Borough Type Cases Deaths

The name of the hospital The borough that the hospital is in N = Non-PCI hospital; P = PCI hospital The number of hearth attack cases The number of deaths among the heart attack cases

(a) Write out the complete hierarchical Beta-binomial model for the subset of thirteen hospitals in Brooklyn.

Sketch a graphical representation and discuss how to choose priors and hyperpriors.

• Sampling, for i = 1, · · · , 13: Yi ∼ Binomial(ni , pi ) • Prior for pi , Stage 1: for i = 1, · · · , 13: pi ∼ Beta(a, b) • Prior for pi , Stage 2: a, b ∼ π(a, b) 204

For pi prior stage 2, we could re-paramaterize it to be a = µη and b = (1 − µ)η: µ ∼ Beta(1, 1) log(η) ∼ Logistic(logn∗ , 1) (b) Use JAGS to obtain posterior samples of the parameters in the hierarchical model. Perform appropriate MCMC diagnostics. deathdata <- DeathHeartAttackDataNYCfull deathdata_Brooklyn <- deathdata[deathdata$Borough == "Brooklyn", ] ## write the model modelString <-" model { ## likelihood for (i in 1:N){ y[i] ~ dbin(p[i], n[i]) } ## priors for (i in 1:N){ p[i] ~ dbeta(a, b) } ## hyperpriors a <- mu*eta b <- (1-mu)*eta mu ~ dbeta(mua, mub) eta <- exp(logeta) logeta ~ dlogis(logn, 1) } " y = deathdata_Brooklyn$Deaths # The y values are in the column named Deaths n = deathdata_Brooklyn$Cases # The n values are in the column named Cases N = length(y) # Compute the number of observations the_data <- list("y" = y, "n" = n, "N" = N, "mua" = 1, "mub" = 1, "logn" = log(100)) initsfunction <- function(chain){ .RNG.seed <- c(1,2)[chain] .RNG.name <- c("base::Super-Duper", "base::Wichmann-Hill")[chain] return(list(.RNG.seed=.RNG.seed, .RNG.name=.RNG.name)) } posterior <- run.jags(modelString, n.chains = 1, data = the_data, monitor = c("p", "mu", "logeta"), 205

adapt = 1000, burnin = 5000, sample = 5000, inits = initsfunction) ## Compiling rjags model... ## Calling the simulation using the rjags method... ## Adapting the model for 1000 iterations... ## Burning in the model for 5000 iterations... ## Running the model for 5000 iterations... ## Simulation complete ## Calculating summary statistics... ## Finished running the simulation plot(posterior, vars = "logeta")

ECDF

8 10 6 4

logeta

## Generating plots...

1.0 0.8 0.6 0.4 0.2 0.0

6000 7000 8000 9000 10000 11000

4

6

5 4 3 2 1 0 4

6

8

10

12

logeta

10

12

Autocorrelation of logeta

% of total

Iteration

8

logeta

1.0 0.5 0.0 −0.5 −1.0 0

5

10 15 20 25 30 35

Lag

plot(posterior, vars = "mu") ## Generating plots...

206

ECDF

0.06 0.08 0.10 0.12

mu

1.0 0.8 0.6 0.4 0.2 0.0

6000 7000 8000 9000 10000 11000

0.06

0.08

2.5 2.0 1.5 1.0 0.5 0.0 0.06

0.08

0.10

0.12

mu

0.12

Autocorrelation of mu

% of total

Iteration

0.10

1.0 0.5 0.0 −0.5 −1.0 0

5

mu

10 15 20 25 30 35

Lag

plot(posterior, vars = "p[1]")

ECDF

0.06 0.10 0.14

1.0 0.8 0.6 0.4 0.2 0.0

6000 7000 8000 9000 10000 11000

0.06 0.08 0.10 0.12 0.14 0.16 0.18

Iteration

p[1]

3 2 1 0 0.05

0.10

0.15

Autocorrelation of p[1]

% of total

p[1]

## Generating plots...

p[1]

1.0 0.5 0.0 −0.5 −1.0 0

5

10 15 20 25 30 35

Lag

(c) Compute the posterior probability that the death rate of Kings County Hospital Center is higher than that of the Kingsbrook Jewish Medical Center. Report and discuss your findings.

207

p4draws <- as.mcmc(posterior, vars = "p[4]") # Kings County Hospital Center p5draws <- as.mcmc(posterior, vars = "p[5]") # Kingsbrook Jewish Medical Center diff = p4draws - p5draws sum(diff > 0)/5000 ## [1] 0.2688 The posterior probability that the death rate of Kings County Hospital Center is higher than that of the Kingsbrook Jewish Medical Center is about 27%, a relatively low probability.

Exercise 8. Heart Attack Deaths - New York City (continued) Develop a hierarchical beta-binomial model for the subset of sixteen hospitals in The Bronx and Queens.

Instead of allowing a pi for each hospital i in the subset, allow a pB to be shared among hospitals in The Bronx, and a pQ to be shared among hospitals in Queens. (a) How does the hierarchical beta-binomial model change from the specification in Exercise 7? Write out the complete hierarchical beta-binomial model, sketch a graphical representation. Discuss how to choose priors and hyperpriors.

• Sampling, for i = 1, · · · , 16, and Xi is the borough indicator (B is The Bronx, Q is Queens): Yi | Xi = B ∼ Binomial(ni , pB ) Yi | Xi = Q ∼ Binomial(ni , pQ ) • Prior for pB and pQ , Stage 1: pB , pQ ∼ Beta(a, b) • Prior for pi , Stage 2: a, b ∼ π(a, b) For pi prior stage 2, we could re-paramaterize it to be a = µη and b = (1 − µ)η: µ ∼ Beta(1, 1) log(η) ∼ Logistic(logn∗ , 1) (b) Use JAGS to obtain posterior samples of the parameters in the hierarchical model. Perform appropriate MCMC diagnostics. deathdata <- DeathHeartAttackDataNYCfull deathdata_BronxQueens <- deathdata[deathdata$Borough == "The Bronx" | deathdata$Borough == "Queens", ] ## write the model modelString <-" model { ## likelihood for (i in 1:N){ y[i] ~ dbin(p[X[i]], n[i]) } ## priors for (i in 1:2){ p[i] ~ dbeta(a, b) } 208

## hyperpriors a <- mu*eta b <- (1-mu)*eta mu ~ dbeta(mua, mub) eta <- exp(logeta) logeta ~ dlogis(logn, 1) } " y = deathdata_BronxQueens$Deaths n = deathdata_BronxQueens$Cases N = length(y) X = (deathdata_BronxQueens$Borough == "Queens") + 1 the_data <- list("y" = y, "n" = n, "N" = N, "X" = X, "mua" = 1, "mub" = 1, "logn" = log(100)) initsfunction <- function(chain){ .RNG.seed <- c(1,2)[chain] .RNG.name <- c("base::Super-Duper", "base::Wichmann-Hill")[chain] return(list(.RNG.seed=.RNG.seed, .RNG.name=.RNG.name)) } posterior <- run.jags(modelString, n.chains = 1, data = the_data, monitor = c("p", "mu", "logeta"), adapt = 1000, burnin = 5000, sample = 5000, inits = initsfunction) ## Compiling rjags model... ## Calling the simulation using the rjags method... ## Adapting the model for 1000 iterations... ## Burning in the model for 5000 iterations... ## Running the model for 5000 iterations... ## Simulation complete ## Calculating summary statistics... ## Finished running the simulation plot(posterior, vars = "logeta") ## Generating plots...

209

ECDF

10 5 0

logeta

1.0 0.8 0.6 0.4 0.2 0.0

6000 7000 8000 9000 10000 11000

0

5

logeta Autocorrelation of logeta

% of total

Iteration

5 4 3 2 1 0 0

5

10

10

1.0 0.5 0.0 −0.5 −1.0 0

5

10 15 20 25 30 35

logeta

Lag

plot(posterior, vars = "mu")

ECDF

0.0 0.2 0.4 0.6 0.8

mu

## Generating plots...

1.0 0.8 0.6 0.4 0.2 0.0

6000 7000 8000 9000 10000 11000

0.0

0.2

15 10 5 0 0.0

0.2

0.4

0.6

0.6

0.8

mu

0.8

Autocorrelation of mu

% of total

Iteration

0.4

mu

1.0 0.5 0.0 −0.5 −1.0 0

5

10 15 20 25 30 35

Lag

210

plot(posterior, vars = "p[1]")

ECDF

0.4 0.0

p[1]

0.8

## Generating plots...

1.0 0.8 0.6 0.4 0.2 0.0

6000 7000 8000 9000 10000 11000

0.0

0.2

10 5 0 0.0

0.2

0.4

0.6

0.6

0.8

1.0

p[1]

0.8

1.0

Autocorrelation of p[1]

% of total

Iteration

0.4

p[1]

1.0 0.5 0.0 −0.5 −1.0 0

5

10 15 20 25 30 35

Lag

plot(posterior, vars = "p[2]") ## Generating plots...

211

ECDF

0.09 0.07

p[2]

1.0 0.8 0.6 0.4 0.2 0.0

6000 7000 8000 9000 10000 11000

0.07

% of total

4 3 2 1 0 0.06

0.07

0.08

0.09

0.09

0.10

p[2]

0.10

Autocorrelation of p[2]

Iteration

0.08

1.0 0.5 0.0 −0.5 −1.0 0

5

p[2]

10 15 20 25 30 35

Lag

(c) Compute the posterior probability that the death rate of all hospitals in The Bronx is higher than that of all hospitals in Queens. Report and discuss your findings. p1draws <- as.mcmc(posterior, vars = "p[1]") # The Bronx p2draws <- as.mcmc(posterior, vars = "p[2]") # Queens diff = p1draws - p2draws sum(diff > 0)/5000 ## [1] 0.5582 The posterior probability that the death rate of all hospitals in The Bronx is higher than that of all hospitals in Queens is about 14%, a relatively low probability.

Exercise 9. Heart Attack Deaths - New York City (continued) Develop a hierarchical beta-binomial model for the complete dataset of 45 hospitals in New York City. Instead of allowing a pi for each hospital i in the subset, allow a pP to be shared among hospitals of Type P, and a pN to be shared among hospitals of Type N. (a) Write out the complete hierarchical beta-binomial model, sketch a graphical representation. Discuss how to choose priors and hyperpriors.

• Sampling, for i = 1, · · · , 45, and Xi is the type indicator (P is type P, and N is type N): Yi | Xi = P ∼ Binomial(ni , pP ) Yi | Xi = N ∼ Binomial(ni , pN ) • Prior for pP and pN , Stage 1: pP , pN ∼ Beta(a, b)

212

• Prior for pi , Stage 2: a, b ∼ π(a, b) For pi prior stage 2, we could re-paramaterize it to be a = µη and b = (1 − µ)η: µ ∼ Beta(1, 1) log(η) ∼ Logistic(logn∗ , 1) (b) Use JAGS to obtain posterior samples of the parameters in the hierarchical model. Perform appropriate MCMC diagnostics. deathdata <- DeathHeartAttackDataNYCfull ## write the model modelString <-" model { ## likelihood for (i in 1:N){ y[i] ~ dbin(p[X[i]], n[i]) } ## priors for (i in 1:2){ p[i] ~ dbeta(a, b) } ## hyperpriors a <- mu*eta b <- (1-mu)*eta mu ~ dbeta(mua, mub) eta <- exp(logeta) logeta ~ dlogis(logn, 1) } " y = deathdata$Deaths n = deathdata$Cases N = length(y) X = (deathdata$Type == "N") + 1 the_data <- list("y" = y, "n" = n, "N" = N, "X" = X, "mua" = 1, "mub" = 1, "logn" = log(100)) initsfunction <- function(chain){ .RNG.seed <- c(1,2)[chain] .RNG.name <- c("base::Super-Duper", "base::Wichmann-Hill")[chain] return(list(.RNG.seed=.RNG.seed, .RNG.name=.RNG.name)) } posterior <- run.jags(modelString, n.chains = 1,

213

data = the_data, monitor = c("p", "mu", "logeta"), adapt = 1000, burnin = 5000, sample = 5000, inits = initsfunction) ## Compiling rjags model... ## Calling the simulation using the rjags method... ## Adapting the model for 1000 iterations... ## Burning in the model for 5000 iterations... ## Running the model for 5000 iterations... ## Simulation complete ## Calculating summary statistics... ## Finished running the simulation plot(posterior, vars = "logeta")

ECDF

2 4 6 8 10

logeta

## Generating plots...

1.0 0.8 0.6 0.4 0.2 0.0

6000 7000 8000 9000 10000 11000

2

4

% of total

3 2 1 0 0

5

10

8

10

12

logeta Autocorrelation of logeta

Iteration

6

logeta

1.0 0.5 0.0 −0.5 −1.0 0

5

10 15 20 25 30 35

Lag

plot(posterior, vars = "mu") ## Generating plots...

214

ECDF

0.15 0.05

mu

1.0 0.8 0.6 0.4 0.2 0.0

6000 7000 8000 9000 10000 11000

0.05

% of total

8 6 4 2 0 0.00

0.05

0.10

0.15

0.15

0.20

mu

0.20

Autocorrelation of mu

Iteration

0.10

1.0 0.5 0.0 −0.5 −1.0 0

5

mu

10 15 20 25 30 35

Lag

plot(posterior, vars = "p[1]")

ECDF

0.065 0.075 0.085

1.0 0.8 0.6 0.4 0.2 0.0

6000 7000 8000 9000 10000 11000

0.065 0.070 0.075 0.080 0.085

Iteration

p[1]

2.5 2.0 1.5 1.0 0.5 0.0 0.065 0.070 0.075 0.080 0.085

Autocorrelation of p[1]

% of total

p[1]

## Generating plots...

p[1]

1.0 0.5 0.0 −0.5 −1.0 0

5

10 15 20 25 30 35

Lag

215

plot(posterior, vars = "p[2]")

ECDF

0.070.080.090.10

p[2]

## Generating plots...

1.0 0.8 0.6 0.4 0.2 0.0

6000 7000 8000 9000 10000 11000

0.07

% of total

4 3 2 1 0 0.07

0.08

0.09

0.09

0.10

p[2]

0.10

Autocorrelation of p[2]

Iteration

0.08

1.0 0.5 0.0 −0.5 −1.0

p[2]

0

5

10 15 20 25 30 35

Lag

(c) Compute the posterior probability that the death rate of all hospitals of Type P higher than that of all hospitals of Type N. Report and discuss your findings. p1draws <- as.mcmc(posterior, vars = "p[1]") # Type P p2draws <- as.mcmc(posterior, vars = "p[2]") # Type N diff = p1draws - p2draws sum(diff > 0)/5000 ## [1] 0.113 The posterior probability that the death rate of all hospitals in The Bronx is higher than that of all hospitals in Queens is about 11%, a relatively low probability. (d) Can you develop a hierarchical beta-binomial model for all 45 hospitals in New York City that takes into account Borough and Type? Describe how you would design the hierarchical model, write JAGS script to obtain posterior samples of the parameters and discuss any findings from your work.

Open-ended.

Exercise 10. Hierarchical Gamma-Poisson Modeling - Marriage Rates in Italy Annual marriage counts per 1000 of the population in Italy from 1936 to 1951 were collected and recorded in Table 10.6. Can we learn something about Italians’ marriage rates during this 16-year period? The dataset is stored in marriage_counts.csv.

Table 10.6. The year and the marriage counts per 1000 of the population in Italy from 1936 to 1951.

216

Year

Count

Year

Count

1936 1937 1938 1939 1940 1941 1942 1943

7 9 8 7 7 6 6 5

1944 1945 1946 1947 1948 1949 1950 1951

5 7 9 10 8 8 8 7

(a) Recall that with count data, a common conjugate model is the gamma-Poisson model, introduced in Section 8.8. Write out the likelihood, the prior distribution, and its posterior distribution under the gamma-Poisson model.

• Sampling: for i = 1, · · · , 16: Y | λ ∼ Poisson(λ) • Prior for λ: λ ∼ Gamma(α, β) • Posterior for λ: λ | y1 , · · · , y16 ∼ Gamma(α +

16 X

yi , β + n)

i=1

(b) Observations are considered i.i.d. in the model in part (a). Figure ?? plots the marriage rates in Italy across years. Discuss whether the i.i.d. assumption is reasonable.

The plot shows clear differences of the marriage rates before, during, and after WWII, indicating that the i.i.d. assumption might not be reasonable. (c) Suppose one believes that the mean marriage rate differs across the three time periods. Using this belief, model the Italian marriage rates in a hierarchical approach. Write out the likelihood, the prior distributions, and any hyperprior distributions under a hierarchical gamma-Poisson model. yj ∼ P oisson(λ1 ), j = 1, 2, 3, 4 yj ∼ P oisson(λ2 ), j = 5, 6, 7, 8, 9, 10 yj ∼ P oisson(λ3 ), j = 11, 12, 13, 14, 15, 16

λ1 , λ2 , λ3 ∼ Gamma(a, b) a ∼ Gamma(a1, a2), b ∼ Gamma(b1, b2) (d) Sketch a graphical representation of the hierarchical gamma-Poisson model. (e) Simulate posterior draws by MCMC using JAGS. Perform MCMC diagnostics and make sure your MCMC has converged.

JAGS model script – define a new variable that gives the period value for each year.

JAGS_poisson_gamma <- "model { for (i in 1:N){ y[i] ~ dpois(lambda_j[period[i]]) } for (i in 1:3){ lambda_j[i] ~ dgamma(a, b) 217

} a ~ dgamma(1, 1) b ~ dgamma(1, 1) }" Period <- ifelse(marriage_counts$Year < 1940, 1, ifelse(marriage_counts$Year < 1946, 2, 3)) the_data <- list(y = marriage_counts$Count, period = Period, N = 10) posterior <- run.jags(JAGS_poisson_gamma, n.chains = 1, data = the_data, monitor = c("lambda_j", "a", "b"), adapt = 1000, burnin = 5000, sample = 5000) ## Compiling rjags model... ## Calling the simulation using the rjags method... ## Adapting the model for 1000 iterations... ## Burning in the model for 5000 iterations... ## Running the model for 5000 iterations... ## Simulation complete ## Calculating summary statistics... ## Finished running the simulation summary(posterior) ## Lower95 Median Upper95 Mean SD Mode ## lambda_j[1] 5.146407e+00 7.5029417 10.3201720 7.5883731 1.3548241 NA ## lambda_j[2] 4.161078e+00 5.9431871 8.0570267 6.0022561 0.9956892 NA ## lambda_j[3] 2.682561e-17 4.9771864 20.4808695 7.4667490 17.2968621 NA ## a 1.217578e-01 1.9406937 4.8809852 2.2271352 1.3684699 NA ## b 9.686763e-03 0.3241232 0.8763961 0.3798356 0.2548313 NA ## MCerr MC%ofSD SSeff AC.10 psrf ## lambda_j[1] 0.019634357 1.4 4761 0.007960213 NA ## lambda_j[2] 0.013536579 1.4 5410 0.007721405 NA ## lambda_j[3] 0.243991319 1.4 5026 -0.003663564 NA ## a 0.043405849 3.2 994 -0.005097943 NA ## b 0.007699335 3.0 1095 -0.014968216 NA (f) Do you see clear differences between the three rate parameters in the posterior? Report and discuss your findings.

Looking at the posterior means, it appears difficult to distinguish the parameters λ1 , λ2 and λ3 .

Exercise 11. Hierarchical Gamma-Poisson Modeling - Fire Calls in Pennsylvania Table 10.7 displays the number of fire calls and the number of building fires for ten counties in Montgomery County, Pennsylvania from 2015 through 2019. This data is currently described as “Emergency - 911 Calls” from kaggle.com. Suppose that the number of building fires for the j-th zip code is Poisson with mean nj λj , where nj and λj are respectively the number of fire calls and rate of building fires for the j-th zip code.

Table 10.7. The number of fire calls and building fires for ten zip codes in Montgomery County, Pennsylvania.

218

Zip Code

Fire Calls

Building Fires

18054 18103 19010 19025 19040 19066 19116 19406 19428 19474

266 1 1470 246 1093 435 2 2092 2025 4

12 0 59 11 47 26 0 113 73 1

(a) Suppose that the building fire rates λ1 , ..., λ10 follow a common Gamma(α, β) distribution where the hyperparameters α and β follow weakly informative distributions. Use JAGS to simulate a sample of size 5000 from the joint posterior distribution of all parameters of the model.

JAGS_poisson_gamma <- "model { for (i in 1:N){ y[i] ~ dpois(n[i] * lambda[i]) } for (i in 1:N){ lambda[i] ~ dgamma(alpha, beta) } alpha ~ dgamma(0.5, 0.5) beta ~ dgamma(0.5, 0.5) }" the_data <- list(y = fire_calls$Building_Fires, n = fire_calls$Fire_Calls, N = 10) library(runjags) posterior <- run.jags(JAGS_poisson_gamma, n.chains = 1, data = the_data, monitor = c("lambda", "alpha", "beta"), adapt = 1000, burnin = 5000, sample = 10000) ## Compiling rjags model... ## Calling the simulation using the rjags method... ## Adapting the model for 1000 iterations... ## Burning in the model for 5000 iterations... ## Running the model for 10000 iterations... ## Simulation complete ## Calculating summary statistics... ## Finished running the simulation (b) The individual estimates of the building rates for zip codes 18054 and 19010 are 12/266 and 59/1470, respectively. Contrast these estimates with the posterior means of the rates λ1 and λ3 . posterior %>% as.mcmc() %>% apply(2, mean) -> post_means post_means

219

## lambda[1] lambda[2] lambda[3] lambda[4] lambda[5] lambda[6] lambda[7] ## 0.04675470 0.10297461 0.04050664 0.04643544 0.04337638 0.06063830 0.08806501 ## lambda[8] lambda[9] lambda[10] alpha beta ## 0.05420305 0.03625124 0.17501875 0.71379863 6.57799765 The estimates of the building fire rates for zip codes 18054 and 19010 are respectively 0.0451 and 0.0401 and the corresponding posterior means of the true rates are 0.0464 and 0.0405. The first zip code had the smaller sample size and is more adjusted towards the overall building fire rate. (c) The parameter µ = α/β represents the mean building fire rates across zip codes. Construct a density estimate of the posterior distribution of µ. posterior %>% as.mcmc() %>% as.data.frame() %>% mutate(mu = alpha / beta) %>% ggplot(aes(mu)) + geom_density()

density

7.5

5.0

2.5

0.0 0

1

2

3

mu (d) Suppose that the county has 50 fire calls to the zip code 19066. Use the simulated predictive distribution to construct a 90% predictive interval for the number of building fires.

Simulate from posterior of λ8 – for each λ8 , simulate y8 from a Poisson distribution. posterior %>% as.mcmc() %>% as.data.frame() -> post lam8 <- post[, 8] y8 <- rpois(10000, lam8) quantile(y8, c(0.05, 0.95)) ## ##

5% 95% 0 1

Actually the predictive distribution for y8 looks like:

220

table(y8) ## y8 ## 0 ## 9458

1 534

2 8

Exercise 12. Hierarchical Gamma-Exponential Modeling - Times Between Traffic Accidents Chapter 8 Exercise 20 describes the Exponential distribution, which is often used as a model for time between events, such as traffic accidents. The exercise also describes the Gamma distribution as a conjugate prior choice for the Exponential data model. 10 times between traffic accidents are collected: 1.5, 15, 60.3, 30.5, 2.8, 56.4, 27, 6.4, 110.7, 25.4 (in minutes). (a) Suppose the 10 collected times are observed at 5 different locations, shown in Table 10.8. Use this information, model the times between traffic accidents in a hierarchical approach. Write out the likelihood, the prior distributions, and any hyperprior distributions under a hierarchical GammaExponential model.

Table 10.8. The time between traffic accidents and recorded location.

Time

Location

Time

Location

1.5 60.3 2.8 27 110.7

1 2 3 4 5

15 30.5 56.4 6.4 25.4

1 2 3 4 5

(b) Sketch a graphical representation of the hierarchical Gamma-Exponential model. (c) Simulate posterior draws by MCMC using JAGS. Perform MCMC diagnostics and make sure your MCMC has converged.

JAGS_poisson_gamma <- "model { for (i in 1:N){ y[i] ~ dexp(lambda_j[location[i]]) } for (i in 1:5){ lambda_j[i] ~ dgamma(alpha, beta) } alpha ~ dgamma(0.5, 0.5) beta ~ dgamma(0.5, 0.5) }" the_data <- list(y = c(1.5, 60.3, 2.8, 27, 110.7, 15, 30.5, 56.4, 6.4, 25.4), location = c(1:5, 1:5), N = 10) library(runjags) posterior <- run.jags(JAGS_poisson_gamma, n.chains = 1, data = the_data, monitor = c("lambda_j", "alpha",

221

"beta"), adapt = 1000, burnin = 5000, sample = 10000) ## Compiling rjags model... ## Calling the simulation using the rjags method... ## Adapting the model for 1000 iterations... ## Burning in the model for 5000 iterations... ## Running the model for 10000 iterations... ## Simulation complete ## Calculating summary statistics... ## Finished running the simulation posterior %>% as.mcmc() %>% summary() ## ## Iterations = 6001:16000 ## Thinning interval = 1 ## Number of chains = 1 ## Sample size per chain = 10000 ## ## 1. Empirical mean and standard deviation for each variable, ## plus standard error of the mean: ## ## Mean SD Naive SE Time-series SE ## lambda_j[1] 0.12184 0.07913 0.0007913 0.0008149 ## lambda_j[2] 0.02637 0.01670 0.0001670 0.0001709 ## lambda_j[3] 0.03911 0.02526 0.0002526 0.0002526 ## lambda_j[4] 0.06652 0.04295 0.0004295 0.0004386 ## lambda_j[5] 0.01757 0.01135 0.0001135 0.0001156 ## alpha 0.48515 0.24398 0.0024398 0.0046762 ## beta 3.82859 2.73322 0.0273322 0.0428233 ## ## 2. Quantiles for each variable: ## ## 2.5% 25% 50% 75% 97.5% ## lambda_j[1] 0.019205 0.064287 0.10473 0.16072 0.32028 ## lambda_j[2] 0.004127 0.014212 0.02308 0.03518 0.06730 ## lambda_j[3] 0.006367 0.020499 0.03366 0.05189 0.10058 ## lambda_j[4] 0.010107 0.035162 0.05744 0.08811 0.17568 ## lambda_j[5] 0.002725 0.009201 0.01525 0.02359 0.04571 ## alpha 0.145698 0.308475 0.44126 0.61202 1.08199 ## beta 0.378049 1.802332 3.22677 5.20298 10.59373 (d) Do you see clear differences between the rate of traffic accidents at the 5 locations? Report and discuss your findings.

To see if there are real differences between λ1 , ..., λ5 , we can compute the posterior mean that, for example, λ1 > λ5 . posterior %>% as.mcmc() %>% as.data.frame() -> post (prob <- mean(post[, 1] > post[, 5])) ## [1] 0.9735 Yes, it is likely that λ1 > λ5 . You can do similar computations to see if other λ rates are different.

222

Exercise 13. Bird Survey Trend Estimates The North American Breeding Bird Survey (BBS) is a yearly survey to monitor the bird population. Regression models were used to estimate the change in population size for many species of birds between 1966 to 1999.

For each of 28 particular grassland species of birds, Table 10.9 displays the trend estimate β̂i and the corresponding standard error σi . This data is stored in the data file BBS_survey.csv. Assume that the trend estimates are independent with β̂i ∼ Normal(βi , σi ) where we assume that the standard errors {σi } are known.

Table 10.9. Trend estimate β̂i and associated standard error σi for 28 grassland species birds.

Species Name

Trend

SE

Species Name

Trend

SE

Upland Sandpiper Long-billed Curlew Mountain Plover Greater Prairie-Chicken Sharp-tailed Grouse Ring-necked Pheasant Northern Harrier Ferruginous Hawk Common Barn Owl Short-eared Owl Burrowing Owl Horned Lark Bobolink Eastern Meadlowlark

0.76 -0.77 -1.05 -2.54 -0.92 -1.06 -0.80 3.52 -2.00 -6.23 1.00 -1.89 -1.25 -2.69

0.39 1.01 2.24 2.33 1.43 0.32 4.00 1.31 2.14 4.55 2.74 0.22 0.31 0.17

Western Meadowlark Chestnut-col Longspur McCown’s Longspur Vesper Sparrow Savannah Sparrow Baird’s Sparrow Grasshopper Sparrow Henslow’s Sparrow LeConte’s Sparrow Cassin’s Sparrow Dickcissel Lark Bunting Sprague’s Pipit Sedge Wren

-0.75 -1.36 -9.29 -0.61 -0.34 -2.04 -3.73 -4.82 0.91 -2.10 -1.46 -3.74 -5.62 3.18

0.17 0.68 8.27 0.24 0.29 1.48 0.47 2.50 0.95 0.51 0.28 2.30 1.34 0.73

(a) Suppose one assumes that the population trend estimates are equal, that is, β1 = ... β28 = β. Using JAGS to simulate from the posterior distribution of β assuming a weakly informative prior on β. Find the posterior mean and posterior standard deviation of β and compare your answers to the trend estimates and standard errors in Table 10.9.

JAGS_pooled <- "model { for (j in 1:N){ y[j] ~ dnorm(beta, invsigma2[j]) } beta ~ dnorm(mu0, g0) }" the_data <- list(y = BBS_survey$Trend, invsigma2 = 1 / BBS_survey$SE ^ 2, N = 28, mu0 = 0, g0 = 0.001) library(runjags) posterior <- run.jags(JAGS_pooled, n.chains = 1, data = the_data, monitor = c("beta"), burnin = 5000, sample = 10000) ## Compiling rjags model... ## Calling the simulation using the rjags method...

223

## Note: the model did not require adaptation ## Burning in the model for 5000 iterations... ## Running the model for 10000 iterations... ## Simulation complete ## Calculating summary statistics... ## Finished running the simulation posterior %>% as.mcmc() %>% as.data.frame() -> post mean(post$beta) ## [1] -1.342462 sd(post$beta) ## [1] 0.07506209 The mean of β is an average value of the 28 trend estimates. The standard deviation of β is very low compared to the standard errors reported in Table 10.9. (b) Next assume that the population trend estimates β1 = ... β28 are a random sample from a Normal distribution with mean µ and standard deviation τ . Assuming weakly informative priors on µ and τ , use JAGS to simulate from the posterior distribution of all parameters. Find the posterior means of the {βj } and compare your estimates with the trend estimates in Table 10.9.

JAGS_hier <- "model { for (j in 1:N){ y[j] ~ dnorm(beta_j[j], invsigma2[j]) } for (j in 1:N){ beta_j[j] ~ dnorm(mu, invtau2) } mu ~ dnorm(mu0, g0) invtau2 ~ dgamma(a_t, b_t) tau <- sqrt(pow(invtau2, -1)) }" the_data <- list(y = BBS_survey$Trend, invsigma2 = 1 / BBS_survey$SE ^ 2, N = 28, mu0 = 0, g0 = 0.001, a_t = 0.1, b_t = 0.1) library(runjags) posterior <- run.jags(JAGS_hier, n.chains = 1, data = the_data, monitor = c("beta_j"), adapt = 1000, burnin = 5000, sample = 10000) ## Compiling rjags model... ## Calling the simulation using the rjags method... ## Note: the model did not require adaptation ## Burning in the model for 5000 iterations... ## Running the model for 10000 iterations... ## Simulation complete

224

## Calculating summary statistics... ## Finished running the simulation Comparing posterior means of βj with the trend estimates in Table 10.9 posterior %>% as.mcmc() %>% apply(2, mean) -> post_ests select(BBS_survey, Species_Name, Trend) %>% mutate(Posterior = post_ests) ## Species_Name Trend Posterior ## 1 Upland_Sandpiper 0.76 0.6526354 ## 2 Long-billed_Curlew -0.77 -0.8861179 ## 3 Mountain_Plover -1.05 -1.1102364 ## 4 Greater_Prairie-Chicken -2.54 -1.6383655 ## 5 Sharp-tailed_Grouse -0.92 -1.0403793 ## 6 Ring-necked_Pheasant -1.06 -1.0605079 ## 7 Northern_Harrier -0.80 -1.1206466 ## 8 Ferruginous_Hawk 3.52 1.6790115 ## 9 Common_Barn_Owl -2.00 -1.4837927 ## 10 Short-eared_Owl -6.23 -1.7889555 ## 11 Burrowing_Owl 1.00 -0.5609870 ## 12 Horned_Lark -1.89 -1.8740091 ## 13 Bobolink -1.25 -1.2413805 ## 14 Eastern_Meadlowlark -2.69 -2.6739661 ## 15 Western_Meadowlark -0.75 -0.7539426 ## 16 Chestnut-col_Longspur -1.36 -1.3182125 ## 17 McCown's_Longspur -9.29 -1.5023477 ## 18 Vesper_Sparrow -0.61 -0.6222869 ## 19 Savannah_Sparrow -0.34 -0.3643683 ## 20 Baird's_Sparrow -2.04 -1.6197399 ## 21 Grasshopper_Sparrow -3.73 -3.5185833 ## 22 Henslow's_Sparrow -4.82 -2.2967694 ## 23 LeConte's_Sparrow 0.91 0.3784515 ## 24 Cassin's_Sparrow -2.10 -2.0113339 ## 25 Dickcissel -1.46 -1.4519023 ## 26 Lark_Bunting -3.74 -2.0455541 ## 27 Sprague's_Pipit -5.62 -3.8252830 ## 28 Sedge_Wren 3.18 2.4428959

Exercise 14. Predicting Baseball Batting Averages The data file batting_2018.csv contains batting data for every player in the 2018 Major League Baseball season. The variables AB.x and H.x in the dataset contain the number of at-bats (opportunities) and number of hits of each player in the first month of the baseball season. One assumes that yi , the number of hits of the i-th player is Binomial(ni , pi ) where ni is the number of at-bats and pi is the probability of a hit. (a) Select a random sample of 20 players from the dataset. library(ProbBayes) library(dplyr) twenty_players <- batting_2018 %>% sample_n(size = 20) twenty_players ## ## 1 ## 2

Name AB.x H.x AB.y H.y Eduardo_Nunez 96 23 384 104 Scooter_Gennett 114 34 470 147 225

## 3 Scott_Schebler ## 4 Matt_Wisler ## 5 Ross_Stripling ## 6 Isiah_Kiner-Falefa ## 7 Mike_Zunino ## 8 John_Hicks ## 9 Matt_Boyd ## 10 Brian_Anderson ## 11 Carlos_Martinez ## 12 Austin_Barnes ## 13 Dixon_Machado ## 14 Yuli_Gurriel ## 15 Jose_Iglesias ## 16 Jacob_Nottingham ## 17 Tanner_Roark ## 18 Drew_Butera ## 19 Matt_Olson ## 20 Ian_Desmond

50 4 1 62 33 28 1 102 12 42 93 58 94 5 13 49 101 101

16 0 0 16 6 5 0 27 2 8 19 13 21 0 2 8 26 18

330 81 4 0 32 3 294 77 340 69 260 70 1 0 488 134 21 6 159 33 121 25 479 143 338 95 15 4 45 9 114 23 479 117 454 113

(b) Assume that the hitting probabilities {pi } have a common Beta(a, b) prior where a = ηµ and b = η(1−µ).

Assume that the hyper parameters η and µ are independent where µ is Uniform(0, 1) and log(η) has a logistic distribution with mean log(50) and scale 1.

The JAGS script for this model is contained in the function JAGS_script() with argument “beta_binomial”: cat(JAGS_script("beta_binomial")) ## model { ## for (i in 1:N){ ## y[i] ~ dbin(p[i], n[i]) ## } ## for (i in 1:N){ ## p[i] ~ dbeta(a, b) ## } ## a <- mu * eta ## b <- (1 - mu) * eta ## mu ~ dbeta(mua, mub) ## eta <- exp(logeta) ## logeta ~ dlogis(logn, 1) ## } (c) Use a JAGS script similar to what is presented in Section 10.3.3, draw a sample of 5000 from the posterior distribution, monitoring values of the {pi }, µ, and log(η). the_data <- list(y = twenty_players$H.x, n = twenty_players$AB.x, N = 20, mua = 1, mub = 1, logn = log(50)) library(runjags) posterior <- run.jags(JAGS_script("beta_binomial"), n.chains = 1, data = the_data, monitor = c("p", "mu", "logeta"), adapt = 1000, burnin = 5000,

226

sample = 10000) ## Compiling rjags model... ## Calling the simulation using the rjags method... ## Adapting the model for 1000 iterations... ## Burning in the model for 5000 iterations... ## Running the model for 10000 iterations... ## Simulation complete ## Calculating summary statistics... ## Finished running the simulation (d) Compare unpooled, pooled, and hierarchical estimates of the {pi } in predicting the batting averages in the remainder of the season. posterior %>% as.mcmc() %>% apply(2, mean) -> post_means output <- data.frame(player = twenty_players$Name, unpooled = round(the_data$y / the_data$n, 3), pooled = round(sum(the_data$y) / sum(the_data$n), 3), hierarchical = round(post_means[1:20], 3)) output ## player unpooled pooled hierarchical ## p[1] Eduardo_Nunez 0.240 0.23 0.233 ## p[2] Scooter_Gennett 0.298 0.23 0.252 ## p[3] Scott_Schebler 0.320 0.23 0.246 ## p[4] Matt_Wisler 0.000 0.23 0.225 ## p[5] Ross_Stripling 0.000 0.23 0.230 ## p[6] Isiah_Kiner-Falefa 0.258 0.23 0.237 ## p[7] Mike_Zunino 0.182 0.23 0.224 ## p[8] John_Hicks 0.179 0.23 0.224 ## p[9] Matt_Boyd 0.000 0.23 0.230 ## p[10] Brian_Anderson 0.265 0.23 0.240 ## p[11] Carlos_Martinez 0.167 0.23 0.226 ## p[12] Austin_Barnes 0.190 0.23 0.224 ## p[13] Dixon_Machado 0.204 0.23 0.224 ## p[14] Yuli_Gurriel 0.224 0.23 0.229 ## p[15] Jose_Iglesias 0.223 0.23 0.229 ## p[16] Jacob_Nottingham 0.000 0.23 0.224 ## p[17] Tanner_Roark 0.154 0.23 0.226 ## p[18] Drew_Butera 0.163 0.23 0.219 ## p[19] Matt_Olson 0.257 0.23 0.238 ## p[20] Ian_Desmond 0.178 0.23 0.216

Exercise 15. Estimating Kidney Cancer Death Rates This exercise is a variation of an activity described in Gelman and Nolan (2017). Suppose one is interested in estimating the kidney cancer death rates for the ten Ohio counties displayed in Table 10.10. Suppose the true death rates θ1 , ..., θ10 are a sample from a Gamma(α, β) distribution. The observed number of deaths yj in the jth county is assumed to be Poisson(nj θj ) where nj is the population size.

Table 10.10. Populations of ten Ohio counties from recent Census estimates.

227

County

Population

County

Cuyahoga Gallia Hamilton Henry Holmes

1,243,857 29,979 816,684 27,086 43,892

Jackson Knox Noble Seneca Van Wert

Population 32,384 61,893 14,354 55,207 28,281

(a) Assuming α = 27, β = 58000, simulate ten true cancer rates θ1 , ..., θ10 from a Gamma(α, β) distribution.

For each county, simulate the number of deaths in all counties. (Use the following R code.) set.seed(1234) counties <- c("Cuyahoga", "Gallia", "Hamilton", "Henry", "Holmes", "Jackson", "Knox", "Noble", "Seneca", "Van Wert") true_rates <- rgamma(10, shape = 27, rate = 58000) pop_size <- c(1243857, 29979, 816684, 27086, 43892, 32384, 61893, 14354, 55207, 28281) deaths <- rpois(10, lambda = pop_size * true_rates) (cancer_data <- data.frame(counties, true_rates, pop_size, deaths)) ## counties true_rates pop_size deaths ## 1 Cuyahoga 0.0003560431 1243857 444 ## 2 Gallia 0.0004852245 29979 18 ## 3 Hamilton 0.0004893418 816684 397 ## 4 Henry 0.0003943631 27086 10 ## 5 Holmes 0.0005029156 43892 25 ## 6 Jackson 0.0004073091 32384 10 ## 7 Knox 0.0005483474 61893 34 ## 8 Noble 0.0004033202 14354 4 ## 9 Seneca 0.0004155246 55207 24 ## 10 Van Wert 0.0003147209 28281 9 (b) Compute the observed death rates {yj /nj }. Identify the counties with the lowest and highest death rates. cancer_data %>% mutate(observed_rates = deaths / pop_size) %>% arrange(desc(observed_rates)) -> cancer_data head(cancer_data, 1) ## counties true_rates pop_size deaths observed_rates ## 1 Gallia 0.0004852245 29979 18 0.0006004203 tail(cancer_data, 1) ## counties true_rates pop_size deaths observed_rates ## 10 Noble 0.0004033202 14354 4 0.000278668 (c) Using JAGS, fit a hierarchical model to the data assuming weakly informative Gamma priors on the parameters α and β. Simulate a sample of 10,000 draws from the posterior distribution and compute the posterior means of the {θj }.

JAGS_poisson_gamma <- "model { for (i in 1:N){ y[i] ~ dpois(n[i] * lambda[i]) } 228

for (i in 1:N){ lambda[i] ~ dgamma(alpha, beta) } alpha ~ dgamma(0.5, 0.5) beta ~ dgamma(0.5, 0.5) }" the_data <- list(y = deaths, n = pop_size, N = 10) library(runjags) posterior <- run.jags(JAGS_poisson_gamma, n.chains = 1, data = the_data, monitor = c("lambda"), adapt = 1000, burnin = 5000, sample = 10000) ## Compiling rjags model... ## Calling the simulation using the rjags method... ## Adapting the model for 1000 iterations... ## Burning in the model for 5000 iterations... ## Running the model for 10000 iterations... ## Simulation complete ## Calculating summary statistics... ## Finished running the simulation (d) Identify the counties with the lowest and highest posterior means of the true rates. Compare these best" andworst" counties with the best and worst counties identified in part (b). posterior %>% as.mcmc() %>% apply(2, mean) -> cancer_data$post_means cancer_data ## counties true_rates pop_size deaths observed_rates post_means ## 1 Gallia 0.0004852245 29979 18 0.0006004203 0.0003571603 ## 2 Holmes 0.0005029156 43892 25 0.0005695799 0.0006064626 ## 3 Knox 0.0005483474 61893 34 0.0005493351 0.0004862936 ## 4 Hamilton 0.0004893418 816684 397 0.0004861121 0.0003762148 ## 5 Seneca 0.0004155246 55207 24 0.0004347275 0.0005719615 ## 6 Henry 0.0003943631 27086 10 0.0003691944 0.0003127966 ## 7 Cuyahoga 0.0003560431 1243857 444 0.0003569542 0.0005513983 ## 8 Van Wert 0.0003147209 28281 9 0.0003182349 0.0002901300 ## 9 Jackson 0.0004073091 32384 10 0.0003087945 0.0004375331 ## 10 Noble 0.0004033202 14354 4 0.0002786680 0.0003255250 cancer_data %>% arrange(desc(post_means)) -> cancer_data head(cancer_data, 1) ## counties true_rates pop_size deaths observed_rates post_means ## 1 Holmes 0.0005029156 43892 25 0.0005695799 0.0006064626 tail(cancer_data, 1)

229

## counties true_rates pop_size deaths observed_rates post_means ## 10 Van Wert 0.0003147209 28281 9 0.0003182349 0.00029013 Note that the counties with the lowest and highest posterior means are different from the counties with the lowest and highest death rates.

Exercises 16 to 20 are additional Bayesian hierarchical models with more complicated structures. These exercises are here to help the reader gain familiarity of working with joint posterior distribution and deriving full conditional posterior distributions. These skills are essential to creating Metropolis and Gibbs sampling algorithms by oneself (as opposed to using JAGS).

Exercise 16. Inference for the Binomial N parameter Suppose that we want inference about an unknown number of animals N in a fixed-size population. On five separate days, we take photographs of some areas where they reside, and count the number of animals in the photos (y1 , . . . , y5 ). Suppose further that each animal has a constant probability θ of appearing in a photograph and that appearances are independent across animals and days. A reasonable model for such data is a Binomial distribution, yi | N, θ ∼ Binomial(N, θ). In our setting, neither the number of trials N nor the probability θ are known.

To get a posterior distribution for N and θ, we propose the following system of models (Raftery 1988): yi | N, θ

∼ Binomial(N, θ)

N | θ, λ

∼ Poisson(λ/θ)

π(λ, θ) ∝ 1/λ, where λ > 0 is a continuous random variable introduced to help with computations. (a) Write down the joint posterior distribution, π(N, θ, λ | y1 , . . . , y5 ), up to a multiplicative constant.

f (N, θ, λ | y1 , . . . , y5 ) ∝ f (y1 , . . . , y5 | N, θ, λ)f (N | θ, λ)f (θ, λ) 5 Y N yi (λ/θ)N exp(−λ/θ) 1/λ = θ (1 − θ)N −yi yi N! i=1 (b) Find an expression for the conditional distribution, π(λ | y1 , . . . , y5 , N, θ). Write the name of the distribution and expressions for its parameter values.

Collecting only terms involving λ, we have p(λ | y1 , . . . , y5 , N, θ, ) ∝ (λ/θ)N exp(−λ/θ)1/λ = λN −1 exp(−λ/θ) This is the kernel of the gamma distribution, Gamma(N, 1/θ). (c) Find the posterior distribution π(N, θ | y1 , . . . , y5 ) by integrating π(N, θ, λ | y1 , . . . , y5 ) with respect to λ. You don’t need to name the distribution; just write its mathematical form.

Let K be the normalizing constant for part (b), so that part (b) integrates to one and is therefore a proper pdf. We have Z ∞

5 Y N yi (λ/θ)N exp(−λ/θ) K θ (1 − θ)N −yi 1/λ dλ yi N! 0 i=1 Z ∞ 5 Y N yi 1 N −yi λN −1 exp(−λ/θ) dλ K θ (1 − θ) yi θN N ! 0 i=1

Z ∞ f (N, θ, λ | y1 , . . . , y5 ) dλ =

0

=

230

Since the integral is the kernel of a gamma distribution, we know that its integral equals one over the Γ(N ) normalizing constant for a gamma distribution. This quantity equals 1/θ N . Hence, we have Z ∞ f (N, θ, λ | y1 , . . . , y5 ) dλ = 0

=

5 Y N yi 1 Γ(N ) θ (1 − θ)N −yi N y θ N ! 1/θN i i=1 5 Y N yi 1 Γ(N ) K θ (1 − θ)N −yi N! yi i=1

K

(d) Find the conditional distribution, π(θ | y1 , . . . , y5 , N ). Write the name of the distribution and expressions for its parameter values.

Collecting only terms involving θ from part (d), we have f (θ | N, y1 , . . . , y5 ) ∝

5 Y

θyi (1 − θ)N −yi

i=1

P5 (1 − θ)5N − i=1 yi P5 P5 This is the kernel of a beta distribution, Beta( i=1 yi + 1, 5N − i=1 yi + 1) =

θ

P5

i=1

yi

Exercise 17. Successes and Failures in Tests A standard model for success or failure in testing situations is the item response model, also called the Rasch model. Suppose that J persons are given a test with K items. For j = 1, . . . , J and k = 1, . . . , K, let yjk = 1 if person j gets item k correct, and let yjk = 0 otherwise. The Rasch model is p(Yjk = 1 | πjk ) =

πjk log = 1 − πjk

Bernoulli(πjk )

(23)

αj − βk .

(24)

Here, αj represents the ability of person j, and βk represents the difficulty of item k. For a Bayesian version of the Rasch model, we use the hierarchical model distributions, p αj ∼ Normal(0, 1/τ ), for j = 1, . . . , J p βk ∼ Normal(µ, 1/φ), for k = 1, . . . , K where τ and σ are precisions. For prior distributions, we use τ

∼ Gamma(a, b),

φ

∼ Gamma(c, d),

µ ∼ Normal(0, e), for known positive constants (a, b, c, d, e). We intend to run an MCMC to estimate the posterior distributions of all parameters. This problem asks you to outline some of the MCMC steps. (a) Write the joint posterior distribution of π(α1 , . . . , αJ , β1 , . . . , βK , τ, φ, µ | {yjk }), up to a constant.

×

h i y Πjk πjkjk (1 − πjk )1−yjk h ih i Πj τ 1/2 exp(−τ αj2 /2) Πk φ1/2 exp(−φ(βk − µ)2 /2)

×

τ a−1 exp(−bτ )φc−1 exp(−dφ) exp(−µ2 /2e2 )

p(α1 , . . . , αJ , β1 , . . . , βK , τ, φ, µ | {yjk }) ∝

exp(α −β )

j k Replace each πjk with 1+exp(α . j −βk )

231

(b) Write the steps you’d take to sample µ given (α1 , . . . , αJ , β1 , . . . , βK , τ, φ, {yjk }). If you can use a Gibbs step, write the name of the full conditional posterior distribution for µ and its parameter values. If you use a Metropolis step, write an expression for the acceptance probability and suggest a family of proposal distributions.

Πk exp(−φ(βk − µ)2 /2) exp(−µ2 /2e2 ) X exp(−(φ (βk − µ)2 + µ2 /e2 )/2)

p(µ | α1 , . . . , αJ , β1 , . . . , βK , τ, φ, {yjk }) ∝ =

k

P We can recognize this as the kernel of a normal distribution. Let β̄ = k βk /K. We have X p(µ | α1 , . . . , αJ , β1 , . . . , βK , τ, φ, Y ) ∝ exp(−(φ (βk − µ)2 + µ2 /e2 )/2) k

∝

exp(−[(Kφ + 1/e2 )µ2 − 2φµ

X

βk )]/2)

k

=

exp(−[(Kφ + 1/e2 )µ2 − 2φµK β̄)]/2)

∝

exp(−(Kφ + 1/e2 )(µ −

β̄Kφ )2 /2) Kφ + 1/e2

β̄Kφ 2 −1/2 Hence, we have a normal distribution, N ( Kφ+1/e ).

2 , (Kφ + 1/e )

We can use a Gibbs step to sample a value of µ from the normal distribution above, conditional on current values of other parameters. (c) Write the steps you’d take to sample φ given (α1 , . . . , αJ , β1 , . . . , βK , τ, µ, {yjk }). If you can use a Gibbs step, write the name of the full conditional posterior distribution for φ and its parameter values. If you use a Metropolis step, write an expression for the acceptance probability and suggest a family of proposal distributions. h

i Πk φ1/2 exp(−φ(βk − µ)2 /2) φc−1 exp(−dφ) X = φK/2+c−1 exp(−φ( (βk − µ)2 /2 + d))

p(φ | α1 , . . . , αJ , β1 , . . . , βK , τ, µ, {yjk }) ∝

k

P We can recognize this as the kernel of a gamma distribution, Gamma(K/2 + c, ( k (βk − µ)2 /2 + d)) We can use a Gibbs step to sample a value of φ from the gamma distribution above, conditional on current values of other parameters. (d) Write the steps you’d take to sample τ given (α1 , . . . , αJ , β1 , . . . , βK , φ, µ, {yjk }). If you use a Gibbs step, write the name of the full conditional distribution for τ . If you use a Metropolis step, write an expression for the acceptance probability and suggest a family of proposal distributions.  p(τ | α1 , . . . , αJ , β1 , . . . , βK , φ, µ, {yjk }) ∝



Πj τ 1/2 exp(−τ

X

αj2 /2) τ α−1 exp(−bτ )

j

=

τ

J/2+α−1

exp(−τ (

X

αj2 /2 + b))

j

P We can recognize this as the kernel of a gamma distribution, Gamma(J/2 + α, ( j αj2 /2 + b)) We can use a Gibbs step to sample a value of τ from the gamma distribution above, conditional on current values of other parameters.

232

Exercise 18. Success and Failures in Tests (continued) Continuing from Exercise 17. (a) Write the steps you’d take to sample βk given (α1 , . . . , αJ , β1 , . . . βk−1 , βk+1 , . . . , βK , τ, φ, µ, {yjk }). If you can use a Gibbs step, write the name of the full conditional posterior distribution for βk and its parameter values. f you use a Metropolis step, write an expression for the acceptance probability and suggest a family of proposal distributions.

Each βk is conditionally independent of other βk , given all other parameters. We have h

i y Πj πjkjk (1 − πjk )1−yjk φ1/2 exp(−φ(βk − µ)2 /2) " yjk 1−yjk # exp(αj − βk ) 1 exp(−φ(βk − µ)2 /2) = Πj 1 + exp(αj − βk ) 1 + exp(αj − βk )

p(βk | α1 , . . . , αJ , τ, φ, µ, {yjk }) ∝

This does not have a recognizable form. Hence, we will use a Metropolis step. For the proposal distribution (s) (s) at the (s + 1) iteration, we might consider a N (βk , c2 ) where c is a tuning parameter and βk is the value ∗ of βk at the sth iteration of the MCMC. Let βk represent a draw from the proposal distribution. We would form the ratio, " yjk 1−yjk # (s) exp(αj −βk∗ ) 1 Πj exp(−φ(s) (βk∗ − µ(s) )2 /2) (s) (s) ∗ ∗ 1+exp(αj −βk )

r= "

Πj

(s)

(s)

exp(αj −βk ) (s)

1+exp(αj −βk )

yjk

(s)

1+exp(αj −βk )

1 (s) (s) 1+exp(αj −βk )

1−yjk #

(s)

exp(−φ(s) (βk − µ(s) )2 /2) (s+1)

We sample a value u from a uniform (0,1). If u < r, we set βk

(s+1)

= β ∗ . If u > r, we set βk

(s)

= βk .

(b) Write the steps you’d take to sample αj given (α1 , . . . , αj−1 , αj+1 , . . . , αJ , β1 , . . . , βK , τ, φ, µ, {yjk }). If you use a Gibbs step, write the name of the full conditional distribution for αj . If you use a Metropolis step, write an expression for the acceptance probability and suggest a family of proposal distributions.

Similar result and approach to part (a).

Exercise 19. Success and Failures in Tests (continued) Suppose that you have 1000 approximately uncorrelated draws of the parameters from the joint posterior distribution of the Rasch model in Exercise 17. Describe how you would do the following tasks. (a) Find the posterior probability that the variability in peoples’ abilities exceeds the variability in item difficulty.

P1000 Compute s=1 I(1/τ (s) > 1/φ(s) ), where I() represents an indicator function that equals one when the condition inside the () is true and equals zero otherwise. (b) Find the item in the test that appears to be the most difficult, and attach a posterior probability that it in fact is the most difficult among all K items. (s)

For each item k = 1, . . . , K, determine the percentage of the 1000 draws for which its associated βk is the largest among all items. Find the item k which corresponds to the maximum among these K percentages.

This maximum percentage is the probability. In mathematical notation, find

argmaxk (

1000 X

(s)

(s)

I(argmaxk (β1 , . . . , βK ) = k)

s=1

233

(c) Perform a posterior predictive check of the model.

Using the 1000 draws of α1 , . . . , αJ and β1 , . . . , βJ , compute the 1000 values of each πjk for all j, k using the inverse logit. For each πjk , randomly draw Yjk from a Bernoulli distribution. These are now replicated scores for each person on each item. With these, one can perform a variety of posterior predictive checks. For example, simulate several datasets with students’ total scores, and compare percentages of replicated values with corresponding percentages of observed values. When the percentages differ substantially, the model does not describe the data well.

Exercise 20. AR(1) Models in Finance and Macroeconomics A common model in finance and macroeconomics is the AR(1) model. Suppose that we have n measurements ordered in time. For j = 1, . . . , n, let yj be the measurement at time j. Suppose we consider the measurement at time 1 as known (not a random variable). Then, for j = 2, . . . , n, a typical AR(1) model is yj = βyj−1 + j where j ∼ Normal(0, σ). Equivalently, we have p(yj | yj−1 , . . . , y1 , β, σ 2 ) = Normal(βyj−1 , σ) for j = 2, . . . , n.

(25) Note that what happens at time j only depends on what happened at time j − 1. For prior distributions, we will use 1/σ 2

∼

Gamma(a, b),

β

∼

Normal(c, d),

for known positive constants (a, b, c, d). We intend to run an MCMC sampler to estimate the posterior distribution of (β, σ 2 ). Whenever possible, we will sample directly from full conditionals. This problem asks you to outline some of the MCMC steps, and to make a prediction for a future observation. (a) Write the kernel of the joint distribution of π(β, σ 2 | y1 , . . . , yn ). [Hint: write p(y2 , . . . , yn | y1 , β, σ 2 ) = p(yn | yn−1 , . . . , y2 , y1 , β, σ 2 )p(yn−1 | yn−2 , . . . , y2 , y1 , β, σ 2 ) · · · p(y2 | y1 , β, σ 2 ).]

• The likelihood: p(y2 , · · · , yn | y1 , β, σ)

=

Πnj=2 p(yj | yj−1 , β, σ)

∝

Πnj=2 1/σ 2 exp(−(yj − βyj−1 )2 /(2σ 2 )) n X 1/σ (n−1)/2 exp(− (yj − βyj−1 )2 /(2σ 2 )).

=

j=2

• With prior, the joint posterior: p(β, σ | y1 , · · · , yn )

∝ 1/σ (n−1)/2 exp(−

n X

(yj − βyj−1 )2 /(2σ 2 ))

j=2 2

=

exp(−(β − c) /(2d2 ))1/σ (a−1)/2 exp(−β(1/σ 2 )) n X 1/σ (n−1)/2+α−1 exp(−( (yj − βyj−1 )2 + 2b)/(2σ 2 )) exp(−(β − c)2 /(2d2 )) j=2

(b) Write the steps you’d take to sample σ 2 given (β, y1 , . . . , yn ). If you use a Gibbs step, write the name of the full conditional distribution for σ and its parameter values. If you use a Metropolis step, write an expression for the acceptance probability and suggest a family of proposal distributions.

234

We can use a Gibbs sampler here. Specifically, we collect all terms involving σ 2 to get

p(1/σ 2 | y1 , · · · , yn , β) ∝

n X 1/σ (n−1)/2+a−1 exp(−( (yj − βyj−1 )2 + 2b)/(2σ 2 )) j=2

Hence, the full conditional posterior distribution of 1/σ 2 is Gamma((n − 1)/2 + a, b +

Pn

2 j=2 (yj − βyj−1 ) /2).

(c) Write the steps you’d take to sample β given (σ, y1 , . . . , yn ). If you use a Gibbs step, write the name of the full conditional distribution for β and its parameter values. If you use a Metropolis step, write an expression for the acceptance probability and suggest a family of proposal distributions.

We can use a Gibbs sampler here. Specifically, we collect all terms involving β to get

p(β | y1 , · · · , yn , σ) ∝

n X exp(−( (yj − βyj−1 )2 + 2b)/(2σ 2 )) exp(−(β − c)2 /(2d2 )) j=2

∝

exp((−1/2)(1/σ 2

n−1 X

yj2 + 1/d)β 2 − 2(1/σ 2

j=1

which is a normal distribution with precision (1/σ 2 Pn 1/d)−1 (1/σ 2 j=2 yj yj−1 + c/d).

n X

yj yj−1 + c/d)β),

j=2

Pn−1

2 −1 j=1 yj + 1/d)

and mean (1/σ 2

Pn−1

2 j=1 yj +

(d) Describe how you would make a 95% posterior interval for the future value of Yn+2 .

We can proceed in three steps. First, take 1000 samples of (β, σ) from the posterior distribution. Second, draw 1000 values of Yn+1 from a Normal(βyn , σ), using each drawn value of (β, σ) only once. Third, for each drawn (β, σ, Yn+1 ), draw a sample of Yn+2 from a Normal(βyn+1 , σ). The resulting 1000 values of Yn+2 can be summarized as a 95% interval, for example by taking the 2.5% and 97.5% values. require(dplyr) require(coda)

Chapter 11: Simple Linear Regression Exercise 1. Linear Regression Model Suppose one is interested in predicting a person’s height yi in cm from his or her arm span xi in cm. Let µi denote the mean µi = E(Yi | xi ). Are the following linear models? If not, explain why. (a) µi = β0 + xi /β1 Yes this is a linear model. (b) µi = β1 xi Yes this is a linear model. (c) µi = β0 + β1 xi Yes this is a linear model. (d) µi = exp(β0 + β1 xi ) No this is not a linear model, because the relationship between µi and xi is not linear but through the exponential function.

235

Exercise 2. Linear Regression Model Suppose a researcher collects some daily weather data for Denver, Colorado for several winter months. She considers the regression model Yi | µi , σ ∼ Normal(µi , σ) where µi = β0 + β1 xi , and xi and yi are respectively the observed temperature (in degrees Fahrenheit) and snowfall (in inches) on data collected on the i-th day. (a) Interpret the intercept parameter β0 . β0 is the expected amount of snowfall µi for an observed temperature of 0 Fahrenheit (i.e. xi = 0). (b) Interpret the slope parameter β1 . β1 is the change in the expected amount of snowfall µi when the observed temperature xi increases by 1 Fahrenheit degree. (c) Suppose β0 = 5, β1 = −0.2, and σ = 0.2. If the temperature is 10 degrees, use this model to predict the amount of snowfall. pred_y contains 5000 predicted value of the amount of snowfall with temperature 10 degrees. beta0 <- 5 beta1 <- -0.2 sigma <- 0.2 xi <- 10 S <- 5000 pred_y <- rnorm(S, beta0 + beta1*xi, sigma) (d) With the same assumptions in part (c), find a 90% interval estimate for the amount of snowfall.

A middle 90% interval estimate for the amount of snowfall is below. quantile(pred_y, c(0.05, 0.95)) ## 5% 95% ## 2.680776 3.325226

Exercise 3. Pythagorean Result in Baseball Table 11.2 displays the average runs scored R, the average runs allowed RA, the number of wins W , the number of losses L for 15 National League teams in the 2018 baseball season. This data is contained in the datafile pythag2018.csv. By the Pythagorean formula, if y = log(W/L) and x = log(R/RA), then approximately y = βx for some slope parameter β. Consider the model Yi ∼ Normal(βxi , σ) where xi and yi are the values of log(R/RA) and log(W/L) for the i-th team. (a) Suppose one assumes β and σ are independent where β ∼ Normal(0, 10) and the precision φ = 1/σ 2 is gamma with parameters 0.001 and 0.001. Write down the expression for the joint posterior density of (β, σ).

• Sampling for {Yi }: ind

Yi | β, σ ∼ Normal(βxi , σ), which produces the likelihood function of (β, φ) as:

n √ Y φ φ √ exp − (yi − βxi )2 L(β, φ) = 2 2π i=1 ( ) n n φX (yi − βxi )2 . ∝ φ 2 exp − 2 i=1 236

(26)

• Independent priors for β and φ:

π(β, φ)

=

1 2 π(β)π(φ) ∝ exp − β φ0.001−1 exp(−0.001φ) 2(10)2

(27)

• Joint posterior density of (β, φ) is:

(

)

n φX 1 2 2 β π(β, φ | {yi }) ∝ φ exp − (yi − βxi ) exp − φ0.001−1 exp(−0.001φ) 2 i=1 2(10)2 ( ! )

n n 1X 1 +0.001−1 2 2 2 β = φ exp − (yi − βxi ) + 0.001 φ exp − 2 i=1 2(10)2 n 2

(28)

(29)

(b) Using JAGS, write a script defining the Bayesian model and use MCMC to simulate a sample of 1000 draws from the posterior distribution.

The JAGS script below uses MCMC and simulated a sample of 1000 draws from the posterior distribution.

BaseballData <- pythag2018 ## write the model modelString <-" model { ## sampling for (i in 1:N){ y[i] ~ dnorm(beta*x[i], invsigma2) } ## priors beta ~ dnorm(mu1, g1) invsigma2 ~ dgamma(a, b) sigma <- sqrt(pow(invsigma2, -1)) } " y <- log(BaseballData$W/BaseballData$L) x <- log(BaseballData$R/BaseballData$RA) N <- length(y) the_data <- list("y" = y, "x" = x, "N" = N, "mu1" = 0, "g1" = 0.001, "a" = 0.001, "b" = 0.001) initsfunction <- function(chain){ .RNG.seed <- c(1,2)[chain] .RNG.name <- c("base::Super-Duper", "base::Wichmann-Hill")[chain] return(list(.RNG.seed=.RNG.seed, .RNG.name=.RNG.name)) } posterior <- run.jags(modelString, n.chains = 1, data = the_data, 237

monitor = c( "beta", "sigma"), adapt = 1000, burnin = 5000, sample = 1000, inits = initsfunction) ## Compiling rjags model... ## Calling the simulation using the rjags method... ## Note: the model did not require adaptation ## Burning in the model for 5000 iterations... ## Running the model for 1000 iterations... ## Simulation complete ## Calculating summary statistics... ## Finished running the simulation (c) From the simulated output, construct 90% interval estimates for the slope β and for the standard deviation σ.

A middle 90% interval estimate for each of the two parameters is below. post <- as.mcmc(posterior) quantile(post[, "beta"], c(0.05, 0.95)) ## 5% 95% ## 1.569610 1.963545 quantile(post[, "sigma"], c(0.05, 0.95)) ## 5% 95% ## 0.1015421 0.1540806 Table 11.2. Average runs scored, average runs scored against, wins, and losses for 15 National League baseball teams in the 2018 season.

Tm

R

RA

W

L

Tm

R

RA

W

L

MIL CHC LAD COL ATL STL PIT WSN

4.6 4.7 4.9 4.8 4.7 4.7 4.3 4.8

4 4 3.70 4.60 4.10 4.30 4.30 4.20

96 95 92 91 90 88 82 82

67 68 71 72 72 74 79 80

ARI PHI NYM SFG CIN SDP MIA

4.3 4.2 4.2 3.7 4.3 3.8 3.7

4 4.50 4.40 4.30 5.10 4.70 5

82 80 77 73 67 66 63

80 82 85 89 95 96 98

Exercise 4. Pythagorean Result in Baseball (continued) (a) Suppose a team scores on average 4.5 runs and allows, on average, 4.0 runs, so x = log(R/RA) = log(4.5/4) = 0.118. Simulate 1000 draws from the posterior distribution of µ = βx. sim_mu contains 1000 simulated draws from the posterior distribution of µ = βx. x <- 0.118 sim_mu <- x * post[, "beta"] (b) If x = 0.118, use the work from part (a) to simulate repeated draws of the posterior predictive distribution of y = log(W/L) and use the output to construct a 90% prediction interval for y.

238

sim_y contains 1000 simulated draws from the posterior predictive distribution of y = log(W/L) and a 90% prediction interval for y is below. sim_y <- rnorm(1000, sim_mu, post[, "sigma"]) quantile(sim_y, c(0.05, 0.95)) ## 5% ## -0.001154883

95% 0.425734720

(c) From the interval found in part (b), find a 90% prediction interval for the number of wins in a 162 game season.

After some algebra: y

=

W/L =

log(W/L)

(30)

exp(y)

(31)

L/W

=

1/ exp(y)

(32)

L/W + 1

=

1/ exp(y) + 1

(33)

(L + W )/W

=

1/ exp(y) + 1

(34)

W

=

(L + W )/(1/ exp(y) + 1)

(35)

Since L + W = 162, a 90% prediction interval for the number of wins in a 162 game season is below. sim_W <- 162/(1/exp(sim_y) + 1) quantile(sim_W, c(0.05, 0.95)) ## 5% 95% ## 80.95323 97.98646

Exercise 5. Pythagorean Result in Baseball (continued) A traditional least-squares of the model y = βx can be found by use of the lm() function for the 2018 team data as follows.

The “Estimate” value is the least-squares estimate of the slope β and the “Std. Error” provides an estimate at the sampling error of this estimate. Compare these estimates with the Bayesian posterior mean and standard deviation of β using a weakly informative prior.

The mean and standard deviation estimates from lm() and with the Bayesian inference using a weakly informative prior are very similar to each other. fit <- lm(I(log(W / L)) ~ 0 + I(log(R / RA)), data = pythag2018) summary(fit) ## ## Call: ## lm(formula = I(log(W/L)) ~ 0 + I(log(R/RA)), data = pythag2018) ## ## Residuals: ## Min 1Q Median 3Q Max ## -0.262720 -0.093391 0.007772 0.068908 0.284084 ## ## Coefficients: ## Estimate Std. Error t value Pr(>|t|) ## I(log(R/RA)) 1.7532 0.1142 15.36 1.83e-15 *** 239

## --## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1 ## ## Residual standard error: 0.1229 on 29 degrees of freedom ## Multiple R-squared: 0.8905, Adjusted R-squared: 0.8867 ## F-statistic: 235.8 on 1 and 29 DF, p-value: 1.834e-15 summary(posterior) ## Lower95 Median Upper95 Mean SD Mode MCerr ## beta 1.50410243 1.7525731 1.9793009 1.7561585 0.11887304 NA 0.0037590955 ## sigma 0.09159158 0.1252684 0.1552842 0.1263385 0.01649082 NA 0.0005633283 ## MC%ofSD SSeff AC.10 psrf ## beta 3.2 1000 -0.01193759 NA ## sigma 3.4 857 0.01881470 NA

Exercise 6. Height and Arm Span A person’s arm span is strongly related to one’s height. To investigate this relationship, arm spans and heights were measured (in cm) for a sample of 20 students and stored in the file ‘arm_height.csv. (This data was simulated using statistics from Mohanty, Babu, and Nair (2001).) Consider the regression model Yi ∼ Normal(µi , σ) where µi = β0 + β1 xi , and yi and xi are respectively the height and arm span for the i-th student. (a) Suppose that one assigns a weakly informative prior to the vector (β0 , β1 , σ) where β0 and β1 are independent normal with mean 0 and standard deviation 10, and the precision φ = 1/σ 2 is gamma with parameters 0.1 and 0.1. Use JAGS to obtain a simulated sample from the posterior distribution.

Find the posterior means of the regression intercept and slope and interpret these posterior means in the context of the problem.

[1] 73.65968 [1] 0.5225551 Using JAGS to fit the model and obtain posterior draws of the intercept and slope as below. On average, a student with zero arm span is expected to be 73.660 cm. On average, with every 1 cm increase in a student’s arm span, his/her/their height is expected to increase by 0.523 cm.

ArmHeightData <- arm_height ## write the model modelString <-" model { ## sampling for (i in 1:N){ y[i] ~ dnorm(beta0 + beta1*x[i], invsigma2) } ## priors beta0 ~ dnorm(mu0, g0) beta1 ~ dnorm(mu1, g1) invsigma2 ~ dgamma(a, b) sigma <- sqrt(pow(invsigma2, -1)) } " y <- ArmHeightData$height x <- ArmHeightData$arm

240

N <- length(y) the_data <- list("y" = y, "x" = x, "N" = N, "mu0" = 0, "g0" = 0.0001, "mu1" = 0, "g1" = 0.0001, "a" = 0.1, "b" = 0.1) initsfunction <- function(chain){ .RNG.seed <- c(1,2)[chain] .RNG.name <- c("base::Super-Duper", "base::Wichmann-Hill")[chain] return(list(.RNG.seed=.RNG.seed, .RNG.name=.RNG.name)) } posterior <- run.jags(modelString, n.chains = 1, data = the_data, monitor = c("beta0", "beta1", "sigma"), adapt = 1000, burnin = 5000, sample = 5000, inits = initsfunction) ## Compiling rjags model... ## Calling the simulation using the rjags method... ## Note: the model did not require adaptation ## Burning in the model for 5000 iterations... ## Running the model for 5000 iterations... ## Simulation complete ## Calculating summary statistics... ## Finished running the simulation post <- as.mcmc(posterior) mean(post[, "beta0"]) ## [1] 73.65968 mean(post[, "beta1"]) ## [1] 0.5225551 (b) Rescale the heights and arm spans and consider the alternative regression model Yi∗ ∼ Normal(µi , σ) where µi = β0 + β1 x∗i and x∗i and yi∗ are the rescaled measurements found by subtracting the respective means and dividing by the respectively standard deviations, i.e. standardized. By using similar weakly informative priors as in part (a), use JAGS to simulate from the joint posterior distribution. Find the posterior means of the regression parameters for this rescaled problem and interpret the means.

[1] 0.001632053 [1] 0.7646361 Using JAGS to fit the model with standardized values and obtain posterior draws of the intercept and slope as below. On average, a student with the average zero arm span is expected to be about 1% taller than average. On average, with every 1 standard deviation increase in a student’s arm span, his/her/their height is expected to increase by 0.765 standard deviation.

ArmHeightData <- arm_height ArmHeightData$arm_STD <- scale(ArmHeightData$arm)

241

ArmHeightData$height_STD <- scale(ArmHeightData$height) ## write the model modelString <-" model { ## sampling for (i in 1:N){ y[i] ~ dnorm(beta0 + beta1*x[i], invsigma2) } ## priors beta0 ~ dnorm(mu0, g0) beta1 ~ dnorm(mu1, g1) invsigma2 ~ dgamma(a, b) sigma <- sqrt(pow(invsigma2, -1)) } " y <- as.vector(ArmHeightData$height_STD) x <- as.vector(ArmHeightData$arm_STD) N <- length(y) the_data <- list("y" = y, "x" = x, "N" = N, "mu0" = 0, "g0" = 0.0001, "mu1" = 0, "g1" = 0.0001, "a" = 0.1, "b" = 0.1) initsfunction <- function(chain){ .RNG.seed <- c(1,2)[chain] .RNG.name <- c("base::Super-Duper", "base::Wichmann-Hill")[chain] return(list(.RNG.seed=.RNG.seed, .RNG.name=.RNG.name)) } posterior <- run.jags(modelString, n.chains = 1, data = the_data, monitor = c("beta0", "beta1", "sigma"), adapt = 1000, burnin = 5000, sample = 5000, inits = initsfunction) ## Compiling rjags model... ## Calling the simulation using the rjags method... ## Note: the model did not require adaptation ## Burning in the model for 5000 iterations... ## Running the model for 5000 iterations... ## Simulation complete ## Calculating summary statistics... ## Finished running the simulation

242

post <- as.mcmc(posterior) mean(post[, "beta0"]) ## [1] 0.001632053 mean(post[, "beta1"]) ## [1] 0.7646361

Exercise 7. Height and Arm Span (continued) Consider the problem of learning about a student’s height using his or her arm span where the measurements are both standardized and one assigns weakly informative priors on the parameters. (a) Consider a student’s whose arm span is one standard deviation above the mean so x∗i = 1. Using the simulated sample from the posterior distribution, find the posterior mean and 90% interval estimate for the expected rescaled height.

Using the one_expected function, we can generate 5000 samples for the expected rescaled height given x∗i = 1. Its mean and a middle 90% interval estimate are below. post <- as.mcmc(posterior) one_expected <- function(x){ expected_y <- post[ , "beta0"] + x * post[ , "beta1"] data.frame(Expected_height_STD = expected_y) } expected_height_STD <- one_expected(1) names(expected_height_STD) <- c("value") mean(expected_height_STD$value) ## [1] 0.7662681 quantile(expected_height_STD$value, c(0.05, 0.95)) ## 5% 95% ## 0.3956527 1.1399207 (b) For the same value x∗i = 1, simulate a sample from the posterior predictive distribution of a future standardized height yi∗ . Estimate the mean and construct a 90% prediction interval for yi∗ .

Using the one_predicted function, we can generate 5000 samples for the predicted rescaled height given x∗i = 1. Its mean and a middle 90% interval estimate are below. one_predicted <- function(x){ expected_y <- post[ , "beta0"] + x * post[ , "beta1"] predicted_y <- rnorm(5000, expected_y, post[, "sigma"]) data.frame(Predicted_Price = predicted_y) } predicted_height_STD <- one_predicted(1) names(predicted_height_STD) <- c("value") mean(predicted_height_STD$value) ## [1] 0.7668367 quantile(predicted_height_STD$value, c(0.05, 0.95)) ## 5% ## -0.4231003

95% 1.9832191

243

(c) Compare the intervals computed in parts (a) and (b) and explain how they are different.

The mean estimates in parts (a) and (b) are close, which is not surprising since a prediction in part (b) is a normal draw centered at the mean simulated in part (a). However, the interval in part (b) is wider than that in part (a) due to the added variability from prediction given a normal distribution with standard deviation sigma.

Exercise 8. Serving Size and Calories of Sandwiches McDonald’s restaurant publishes nutritional information on the sandwiches they serve. Table 11.3 displays the serving size (in grams) and the calories for some sandwiches. (This data is available from the file mcdonalds.csv.). One is interested in the model Yi ∼ Normal(µi , σ), where µi = β0 + β1 xi and yi and xi are respectively the calories and serving size for the i-th sandwich.

Table 11.3. Serving size (grams) and calories for some McDonalds sandwiches.

Sandwich

Size

Calories

Hamburger Cheeseburger Double Cheeseburger Quarter Pounder with Cheese Double Quarter Pounder with Cheese Big Mac Big N’ Tasty Filet-O-Fish McChicken Premium Grilled Chicken Classic Sandwich Premium Crispy Chicken Classic Sandwich

105 119 173 199 280 219 232 141 147 229 232

260 310 460 510 730 560 470 400 370 420 500

(a) Using a suitable weakly informative prior for the regression parameters and the standard deviation, use JAGS to obtain a simulated sample from the joint posterior distribution.

McDonaldsData <- mcdonalds ## write the model modelString <-" model { ## sampling for (i in 1:N){ y[i] ~ dnorm(beta0 + beta1*x[i], invsigma2) } ## priors beta0 ~ dnorm(mu0, g0) beta1 ~ dnorm(mu1, g1) invsigma2 ~ dgamma(a, b) sigma <- sqrt(pow(invsigma2, -1)) } " y <- McDonaldsData$Calories x <- McDonaldsData$Size N <- length(y)

244

the_data <- list("y" = y, "x" = x, "N" = N, "mu0" = 0, "g0" = 0.0001, "mu1" = 0, "g1" = 0.0001, "a" = 0.1, "b" = 0.1) initsfunction <- function(chain){ .RNG.seed <- c(1,2)[chain] .RNG.name <- c("base::Super-Duper", "base::Wichmann-Hill")[chain] return(list(.RNG.seed=.RNG.seed, .RNG.name=.RNG.name)) } posterior <- run.jags(modelString, n.chains = 1, data = the_data, monitor = c("beta0", "beta1", "sigma"), adapt = 1000, burnin = 5000, sample = 5000, inits = initsfunction) ## Compiling rjags model... ## Calling the simulation using the rjags method... ## Note: the model did not require adaptation ## Burning in the model for 5000 iterations... ## Running the model for 5000 iterations... ## Simulation complete ## Calculating summary statistics... ## Finished running the simulation (b) Construct a graph and 95% interval estimate for the regression slope β1 . Is there sufficient evidence to say that sandwiches with larger serving sizes have more calories?

A middle 95% interval estimate is strictly positive and greater than 1.5. A density plot shows that β1 is positive. Both these provide sufficient evidence that sandwiches with larger serving sizes have more calories. post <- as.mcmc(posterior) quantile(post[, "beta1"], c(0.025, 0.975)) ## 2.5% 97.5% ## 1.513525 2.793634 df <- as.data.frame(post[, "beta1"]) names(df) <- c("beta1") ggplot(df, aes(x = beta1)) + geom_density(size = 1, color = "blue") + increasefont()

245

density

1.0

0.5

0.0 1.0

1.5

2.0

2.5

3.0

beta1 (c) For a sandwich with serving size 200 grams, simulate a sample from the predictive distribution of the number of calories. Construct a 95% prediction interval for the number of calories.

Based on a simulated sample from the predictive distribution of the number of calories, a 95% prediction interval is below. one_predicted <- function(x){ expected_y <- post[ , "beta0"] + x * post[ , "beta1"] predicted_y <- rnorm(5000, expected_y, post[, "sigma"]) data.frame(Predicted_Calories = predicted_y) } predicted_calories_200 <- one_predicted(200) names(predicted_calories_200) <- c("value") quantile(predicted_calories_200$value, c(0.025, 0.975)) ## 2.5% 97.5% ## 330.5047 615.7358

Exercise 9. Serving Size and Calories of Sandwiches (continued) In Exercise 8, one obtained a simulated sample from the posterior distribution of (β0 , β1 , σ) using a weakly informative prior. (a) Suppose one is interested in learning about the expected calories µ for a sandwich where the serving size is 300 grams. From the simulated sample from the posterior, construct a sample from the posterior of µ and construct a 90% interval estimate.

A sample from the posterior of µ is in expected_calories_300 and a 90% interval estimate is below.

246

one_expected <- function(x){ expected_y <- post[ , "beta0"] + x * post[ , "beta1"] data.frame(Expected_Calories = expected_y) } expected_calories_300 <- one_expected(300) names(expected_calories_300) <- c("value") quantile(expected_calories_300$value, c(0.05, 0.95)) ## 5% 95% ## 619.2432 762.8154 (b) Suppose one is interested in learning about the value of theserving size x∗ such that the mean calories β0 + β1 x∗ is equal to 500.First write the serving size x∗ as a function of β0 and β1 . Then use this representation to find a 90% interval estimate for the serving size x∗ .

Doing some algebra, we have x∗ =

500 − β0 . β1

We can then use the following script to find a 90% interval estimate for the serving size x∗ as below. x_star <- (500 - post[ , "beta0"])/post[ , "beta1"] quantile(x_star, c(0.05, 0.95)) ## 5% 95% ## 195.0987 230.8760

Exercise 10. Movie Sales Table 11.4 displays the weekend and gross sales, in millions of dollars, for ten popular movies released in 2017.

This data is contained in the file movies2017.csv. Suppose one is interested in studying the relationship between the two variables by fitting the model Yi | β0 , β1 , σ ∼ Normal(β0 + βi xi , σ), where yi and xi are respectively the gross sales and weekend sales for the i-th movie.

Table 11.4. Weekend and gross sales (in millions of dollars) for ten popular movies released in 2017.

Movie Beauty and the Beast The Fate of the Furious Despicable Me 3 Spider-Man: Homecoming Guardians of the Galaxy, Vol 2.

Thor: Ragnarok Wonder Woman Pirates of the Caribbean It Justice League

Weekend

Gross

174 99 72 117 147 122 103 63 123 94

504 226 264 334 389 315 413 173 327 229

(a) Assuming weekly informative priors on the regression parameters and the standard deviation, use JAGS to simulate from the joint posterior distribution of (β0 , β1 , σ).

247

MoviesData <- movies2017 ## write the model modelString <-" model { ## sampling for (i in 1:N){ y[i] ~ dnorm(beta0 + beta1*x[i], invsigma2) } ## priors beta0 ~ dnorm(mu0, g0) beta1 ~ dnorm(mu1, g1) invsigma2 ~ dgamma(a, b) sigma <- sqrt(pow(invsigma2, -1)) } " y <- MoviesData$Gross x <- MoviesData$Weekend N <- length(y) the_data <- list("y" = y, "x" = x, "N" = N, "mu0" = 0, "g0" = 0.0001, "mu1" = 0, "g1" = 0.0001, "a" = 0.1, "b" = 0.1) initsfunction <- function(chain){ .RNG.seed <- c(1,2)[chain] .RNG.name <- c("base::Super-Duper", "base::Wichmann-Hill")[chain] return(list(.RNG.seed=.RNG.seed, .RNG.name=.RNG.name)) } posterior <- run.jags(modelString, n.chains = 1, data = the_data, monitor = c("beta0", "beta1", "sigma"), adapt = 1000, burnin = 5000, sample = 5000, inits = initsfunction) ## Compiling rjags model... ## Calling the simulation using the rjags method... ## Note: the model did not require adaptation ## Burning in the model for 5000 iterations... ## Running the model for 5000 iterations... ## Simulation complete ## Calculating summary statistics... ## Finished running the simulation (b) Construct a scatterplot of the simulated draws of β0 and β1 .

248

post <- as.mcmc(posterior) df <- as.data.frame(cbind(post[, "beta0"], post[, "beta1"])) names(df) <- c("beta0", "beta1") ggplot(as.data.frame(df), aes(beta0, beta1)) + geom_point(color = "blue", alpha = 0.2) + increasefont()

4

beta1

3

2

1 −200

−100

0

100

200

beta0 (c) From your output, is there significant evidence that weekend sales is a useful predictor of gross sales?

Explain.

From the scatter plot in part (b), weekend sales seems a useful predictor of gross sales due to the strictly positive values of β1 . (d) From the simulated draws, construct a 80% interval estimate for the average gross sales for all movies that have a $100 million weekend sales.

A middle 80% interval estimate for the average gross sales for all movies that have a $100 million weekend sales is below. one_expected <- function(x){ expected_y <- post[ , "beta0"] + x * post[ , "beta1"] data.frame(Expected_GrossSales = expected_y) } expected_gross_100 <- one_expected(100) names(expected_gross_100) <- c("value") quantile(expected_gross_100$value, c(0.1, 0.9)) ## 10% 90% ## 262.8337 309.6785

249

(e) Suppose you are interested in predicting the gross sales for a single move that has $100 million weekend sales. Construct a 80% prediction interval for the gross sales.

A middle 80% prediction interval for the gross sales is below. one_predicted <- function(x){ expected_y <- post[ , "beta0"] + x * post[ , "beta1"] predicted_y <- rnorm(5000, expected_y, post[, "sigma"]) data.frame(Predicted_GrossSales = predicted_y) } predicted_gross_100 <- one_predicted(100) names(predicted_gross_100) <- c("value") quantile(predicted_gross_100$value, c(0.1, 0.9)) ## 10% 90% ## 209.5179 365.6766

Exercise 11. Fog Index and Complex Words of Books The amazon.com website used to provide “text statistics” for many of the books it sold. For a particular book, the website displayed the “fog index”, the number of years of formal education required to read and understand a passage of text, and the “complex words”, the percentage of words in the book with three or more syllables. Table 11.5 displays the complex words and fog index for a selection of 21 popular books.

(This data is contained in the file book_stats.csv.) Suppose one is interested in predicting the fog index of a book given its complex words measurement. Using suitable weakly informative priors on the parameters, fit a simple regression model by simulating 5000 draws from the joint posterior distribution. Use this output to construct a 90% prediction interval for a book whose complex word measurement is 10.

Table 11.5. Measures of complex words and fog index for a selection of 20 books.

Book A Million Little Pieces The Five People You Meet in Heaven The Glass Castle The Mermaid Chair The Kite Runner Marley & Me Memoirs of a Geisha In Cold Blood Moneyball Jim Cramer’s Real Money The Da Vinci Code Power of Thinking Without Thinking A Mathematician at the Ballpark Misquoting Jesus The Tipping Point Freakonomics Curve Ball The World is Flat Confessions of an Economics Hit Man Collapse Ordinal Data Modeling

250

Complex Words

Fog Index

4.00 6.00 6.00 7.00 7.00 8.00 8.00 10.00 10.00 10.00 12.00 12.00 12.00 13.00 13.00 14.00 14.00 15.00 15.00 17.00 24.00

5.70 6.60 8.40 8.20 7.10 9.20 10.10 9.80 10.30 11.70 9.10 11.60 10.20 16.10 12.60 11.10 10.10 15.00 12.80 18.00 14.60

Using weakly informative priors, we have fit a simple linear regression model and construct a middle 90% prediction interval for a book whose complex word measurement is 10 as below.

BooksData <- book_stats ## write the model modelString <-" model { ## sampling for (i in 1:N){ y[i] ~ dnorm(beta0 + beta1*x[i], invsigma2) } ## priors beta0 ~ dnorm(mu0, g0) beta1 ~ dnorm(mu1, g1) invsigma2 ~ dgamma(a, b) sigma <- sqrt(pow(invsigma2, -1)) } " y <- BooksData$Fog.Index x <- BooksData$Complex.Words N <- length(y) the_data <- list("y" = y, "x" = x, "N" = N, "mu0" = 0, "g0" = 0.0001, "mu1" = 0, "g1" = 0.0001, "a" = 0.1, "b" = 0.1) initsfunction <- function(chain){ .RNG.seed <- c(1,2)[chain] .RNG.name <- c("base::Super-Duper", "base::Wichmann-Hill")[chain] return(list(.RNG.seed=.RNG.seed, .RNG.name=.RNG.name)) } posterior <- run.jags(modelString, n.chains = 1, data = the_data, monitor = c("beta0", "beta1", "sigma"), adapt = 1000, burnin = 5000, sample = 5000, inits = initsfunction) ## Compiling rjags model... ## Calling the simulation using the rjags method... ## Note: the model did not require adaptation ## Burning in the model for 5000 iterations... ## Running the model for 5000 iterations... ## Simulation complete ## Calculating summary statistics... ## Finished running the simulation 251

post <- as.mcmc(posterior) one_predicted <- function(x){ expected_y <- post[ , "beta0"] + x * post[ , "beta1"] predicted_y <- rnorm(5000, expected_y, post[, "sigma"]) data.frame(Predicted_FogIndex = predicted_y) } predicted_fog_10 <- one_predicted(10) names(predicted_fog_10) <- c("value") quantile(predicted_fog_10$value, c(0.05, 0.95)) ## ##

5% 95% 6.73276 13.67745

Exercise 12. Distances of Batted Balls Figure ?? displays a scatterplot of the launch speed (mph) and distance traveled (feet) for batted balls hit by the baseball player Mike Trout during the 2018 season. This data is contained in the file trout20.csv. (a) In R, rescale both the explanatory and response variables so each has mean 0 and standard deviation 1.

The launch_speed_STD and hit_distance_STD are now scaled with mean 0 and standard deviation 1.

BattedballsData <- trout20 BattedballsData$launch_speed_STD <- scale(BattedballsData$launch_speed) BattedballsData$hit_distance_sc_STD <- scale(BattedballsData$hit_distance_sc) (b) Consider the regression model on the rescaled data: Yi∗ ∼ Normal(µ∗i , σ) where µ∗i = β0 + β1 x∗i , where yi∗ and x∗i are respectively the rescaled distance traveled and rescaled launch speed of the i-th batted ball. Suppose one has little prior beliefs about the values of the parameters β0 , β1 and σ. One assumes that the parameter are independent where β0 and β1 are assigned normal priors with mean 0 and standard deviation 10, and the precision φ = 1/σ 2 is Gamma with parameters 0.1 and 0.1. Use JAGS to obtain a simulated sample of 5000 draws from the posterior distribution of (β0 , β1 , σ). ## write the model modelString <-" model { ## sampling for (i in 1:N){ y[i] ~ dnorm(beta0 + beta1*x[i], invsigma2) } ## priors beta0 ~ dnorm(mu0, g0) beta1 ~ dnorm(mu1, g1) invsigma2 ~ dgamma(a, b) sigma <- sqrt(pow(invsigma2, -1)) } " y <- as.vector(BattedballsData$hit_distance_sc_STD) x <- as.vector(BattedballsData$launch_speed_STD) N <- length(y) 252

the_data <- list("y" = y, "x" = x, "N" = N, "mu0" = 0, "g0" = 0.01, "mu1" = 0, "g1" = 0.01, "a" = 0.1, "b" = 0.1) initsfunction <- function(chain){ .RNG.seed <- c(1,2)[chain] .RNG.name <- c("base::Super-Duper", "base::Wichmann-Hill")[chain] return(list(.RNG.seed=.RNG.seed, .RNG.name=.RNG.name)) } posterior <- run.jags(modelString, n.chains = 1, data = the_data, monitor = c("beta0", "beta1", "sigma"), adapt = 1000, burnin = 5000, sample = 5000, inits = initsfunction) ## Compiling rjags model... ## Calling the simulation using the rjags method... ## Note: the model did not require adaptation ## Burning in the model for 5000 iterations... ## Running the model for 5000 iterations... ## Simulation complete ## Calculating summary statistics... ## Finished running the simulation (c) From the simulated output, construct a density estimate and 90% interval estimate for the slope parameter β1 .

A density plot and a middle 90% interval estimate for β1 are below. post <- as.mcmc(posterior) df <- as.data.frame(post[, "beta1"]) names(df) <- c("beta1") ggplot(df, aes(x = beta1)) + geom_density(size = 1, color = "blue") + increasefont()

253

5

density

4

3

2

1

0 0.8

1.0

1.2

beta1 quantile(post[, "beta1"], c(0.05, 0.95)) ## 5% 95% ## 0.8182593 1.0694786 (d) Using the output, obtain simulated draws of the expected standardized distances when the standardized speed is equal to one. Construct a 90% interval estimate for the expected distance.

A middle 90% interval estimate for the expected standardized distance is below. one_expected <- function(x){ expected_y <- post[ , "beta0"] + x * post[ , "beta1"] data.frame(Expected_distances_STD = expected_y) } expected_distances_STD_1 <- one_expected(1) names(expected_distances_STD_1) <- c("value") quantile(expected_distances_STD_1$value, c(0.05, 0.95)) ## 5% 95% ## 0.7685179 1.1180680 (e) Obtain a simulated sample from the predicted standardized distance when the standardized speed is equal to one. Construct a 90% prediction interval and compare this interval with the interval constructed in part (c).

A middle 90% prediction estimate for the standardized distance is below. This interval is wider than that in part (c), due to the fact that when predictions are made, variability from the normal sampling model, specifically, the standard deviation of σ is introduced. one_predicted <- function(x){ expected_y <- post[ , "beta0"] +

x * post[ , "beta1"] 254

predicted_y <- rnorm(5000, expected_y, post[, "sigma"]) data.frame(Predicted_distances_STD = predicted_y)

} predicted_distances_STD_1 <- one_predicted(1) names(predicted_distances_STD_1) <- c("value") quantile(predicted_distances_STD_1$value, c(0.1, 0.9)) ## 10% 90% ## 0.4682184 1.4338868

Exercise 13. Distance of Batted Balls (continued) Consider the use of the regression model on the standardized data Yi∗ ∼ Normal(µ∗i , σ) where µ∗i = β0 + β1 x∗i , where yi∗ and x∗i are respectively the rescaled distance traveled and rescaled launch speed of the i-th batted ball. (a) Suppose one believes that if Trout hits a ball with average speed, his expected distance is Normal with mean 350 feet and standard deviation 10 feet. In addition, one believe the correlation between launch speed and distance is Normal with mean 0.9 and a standard deviation of 0.02. Construct a prior distribution on the vector of parameters (β0 , β1 ) that reflects this information.

On the standardized data, the following prior distributions reflect the information:

β0

∼ Normal(350, 10)

(36)

β1

∼ Normal(0.9, 0.02)

(37)

(b) Use JAGS with this informative prior to obtain a simulated sample from the posterior distribution of all parameters. ## write the model modelString <-" model { ## sampling for (i in 1:N){ y[i] ~ dnorm(beta0 + beta1*x[i], invsigma2) } ## priors beta0 ~ dnorm(mu0, g0) beta1 ~ dnorm(mu1, g1) invsigma2 ~ dgamma(a, b) sigma <- sqrt(pow(invsigma2, -1)) } " y <- as.vector(BattedballsData$hit_distance_sc_STD) x <- as.vector(BattedballsData$launch_speed_STD) N <- length(y) the_data <- list("y" = y, "x" = x, "N" = N, "mu0" = 350, "g0" = 0.01, 255

"mu1" = 0.9, "g1" = 2500, "a" = 0.1, "b" = 0.1) initsfunction <- function(chain){ .RNG.seed <- c(1,2)[chain] .RNG.name <- c("base::Super-Duper", "base::Wichmann-Hill")[chain] return(list(.RNG.seed=.RNG.seed, .RNG.name=.RNG.name)) } posterior <- run.jags(modelString, n.chains = 1, data = the_data, monitor = c("beta0", "beta1", "sigma"), adapt = 1000, burnin = 5000, sample = 5000, inits = initsfunction) ## Compiling rjags model... ## Calling the simulation using the rjags method... ## Note: the model did not require adaptation ## Burning in the model for 5000 iterations... ## Running the model for 5000 iterations... ## Simulation complete ## Calculating summary statistics... ## Finished running the simulation (c) Construct a 90% interval estimate for the slope parameter β1 and compare your answer with the interval estimate constructed in part (c) of Exercise 12.

A 90% interval estimate for β1 under this prior is tighter, i.e. narrower, than that in part (c) of Exercise 12.

This suggests a stronger and tighter prior has a stronger influence on the posterior estimate. post <- as.mcmc(posterior) quantile(post[, "beta1"], c(0.05, 0.95)) ## 5% 95% ## 0.8715713 0.9351830

Exercise 14. Gas Bill The dataset gas2017.csv contains the average temperature and gas bill in dollars for twelve months for a particular homeowner. This chapter described the use of a conditional means prior to construct a prior distribution on the regression vector (β0 , β1 ) in Section 11.9. (a) Using this conditional means prior and a weakly informative Gamma(1, 1) prior on φ = 1/σ 2 , use JAGS to simulate 5000 draws from the posterior distribution of (β0 , β1 , σ). Construct a 90% interval estimate for the regression slope β1 .

Using this conditional means prior, a middle 90% interval estimate for β1 is below.

GasData <- gas2017 GasData$Temp_STD <- scale(GasData$Temp) GasData$Bill_STD <- scale(GasData$Bill)

256

## write the model modelString <-" model { ## sampling for (i in 1:N){ y[i] ~ dnorm(beta0 + beta1*x[i], invsigma2) } ## priors beta1 <- (mu2 - mu1) / (x2 - x1) beta0 <- mu1 - x1 * (mu2 - mu1) / (x2 - x1) mu1 ~ dnorm(m1, s1) mu2 ~ dnorm(m2, s2) invsigma2 ~ dgamma(a, b) sigma <- sqrt(pow(invsigma2, -1)) } " y <- as.vector(GasData$Bill_STD) x <- as.vector(GasData$Temp_STD) N <- length(y) the_data <- list("y" = y, "x" = x, "N" = N, "m1" = 100, "s1" = 0.0025, "m2" = 50, "s2" = 0.0044, "x1" = 40, "x2" = 60, "a" = 1, "b" = 1) initsfunction <- function(chain){ .RNG.seed <- c(1,2)[chain] .RNG.name <- c("base::Super-Duper", "base::Wichmann-Hill")[chain] return(list(.RNG.seed=.RNG.seed, .RNG.name=.RNG.name)) } posterior_CM <- run.jags(modelString, n.chains = 1, data = the_data, monitor = c("beta0", "beta1", "sigma"), adapt = 1000, burnin = 5000, sample = 5000, inits = initsfunction) ## Compiling rjags model... ## Calling the simulation using the rjags method... ## Note: the model did not require adaptation ## Burning in the model for 5000 iterations... ## Running the model for 5000 iterations... ## Simulation complete ## Calculating summary statistics... ## Finished running the simulation

257

post_CM <- as.mcmc(posterior_CM) quantile(post_CM[, "beta1"], c(0.05, 0.95)) ## 5% 95% ## 0.650035 1.051728 (b) Rerun this analysis using a suitable weakly informative prior on all the parameters. Simulate 5000 draws and construct a 90% interval estimate for the slope β1 . ## write the model modelString <-" model { ## sampling for (i in 1:N){ y[i] ~ dnorm(beta0 + beta1*x[i], invsigma2) } ## priors beta0 ~ dnorm(mu0, g0) beta1 ~ dnorm(mu1, g1) invsigma2 ~ dgamma(a, b) sigma <- sqrt(pow(invsigma2, -1)) } " y <- as.vector(GasData$Bill_STD) x <- as.vector(GasData$Temp_STD) N <- length(y) the_data <- list("y" = y, "x" = x, "N" = N, "mu0" = 0, "g0" = 0.01, "mu1" = 0, "g1" = 0.01, "a" = 1, "b" = 1) initsfunction <- function(chain){ .RNG.seed <- c(1,2)[chain] .RNG.name <- c("base::Super-Duper", "base::Wichmann-Hill")[chain] return(list(.RNG.seed=.RNG.seed, .RNG.name=.RNG.name)) } posterior <- run.jags(modelString, n.chains = 1, data = the_data, monitor = c("beta0", "beta1", "sigma"), adapt = 1000, burnin = 5000, sample = 5000, inits = initsfunction) ## Compiling rjags model... ## Calling the simulation using the rjags method... ## Note: the model did not require adaptation ## Burning in the model for 5000 iterations...

258

## Running the model for 5000 iterations... ## Simulation complete ## Calculating summary statistics... ## Finished running the simulation post <- as.mcmc(posterior) quantile(post[, "beta1"], c(0.05, 0.95)) ## 5% 95% ## -1.1933526 -0.4548097 (c) Compare your interval estimates for parts (a) and (b).

The two interval estimates are very different in parts (a) and (b). This suggests that prior choices affect the posterior inferences in this study. It is not surprising because the sample size is small, only 12 observaitons.

Exercise 15. Gas Bill (continued) A traditional “maximum likelihood” fit of the model Yi | β0 , β1 , σ ∼ Normal(β0 + β1 x, σ) can be found by use of the lm() function. fit <- lm(Bill ~ Temp, data = gas2017) summary(fit) The output provides estimates of the regression coefficients β0 and β1 and the ’Residual standard error" provides an estimate at the standard deviation σ. Compare these estimates with the posterior means of β0 , β1 , σ found in Exercise 14. fit <- lm(Bill ~ Temp, data = gas2017) summary(fit) ## ## Call: ## lm(formula = Bill ~ Temp, data = gas2017) ## ## Residuals: ## Min 1Q Median 3Q Max ## -40.922 -11.878 0.211 15.036 28.665 ## ## Coefficients: ## Estimate Std. Error t value Pr(>|t|) ## (Intercept) 173.0518 23.1177 7.486 2.1e-05 *** ## Temp -1.9182 0.4139 -4.635 0.000929 *** ## --## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1 ## ## Residual standard error: 22.14 on 10 degrees of freedom ## Multiple R-squared: 0.6823, Adjusted R-squared: 0.6506 ## F-statistic: 21.48 on 1 and 10 DF, p-value: 0.0009294 These estimates are not comparable to those in Exercise 14 because the lm() sample script is fit on data on the original scale. fit <- lm(Bill_STD ~ Temp_STD, data = GasData) summary(fit) ## ## Call: 259

## lm(formula = Bill_STD ~ Temp_STD, data = GasData) ## ## Residuals: ## Min 1Q Median 3Q Max ## -1.09270 -0.31717 0.00564 0.40150 0.76540 ## ## Coefficients: ## Estimate Std. Error t value Pr(>|t|) ## (Intercept) 1.282e-16 1.706e-01 0.000 1.000000 ## Temp_STD -8.260e-01 1.782e-01 -4.635 0.000929 *** ## --## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1 ## ## Residual standard error: 0.5911 on 10 degrees of freedom ## Multiple R-squared: 0.6823, Adjusted R-squared: 0.6506 ## F-statistic: 21.48 on 1 and 10 DF, p-value: 0.0009294 colMeans(post) ## ##

beta0 beta1 0.002002336 -0.826568998

sigma 0.725378724

colMeans(post_CM) ## beta0 beta1 sigma ## 0.04287136 0.87364265 1.80661501 The lm() output seems closer to the output from the weakly informative prior in Exericse 14. This suggests if we have strong prior information, the posterior inference could be influenced a lot. If the prior is good, we should certainly leverage the information it carries.

Exercise 16. Conditional Means Prior Suppose you are interested in predicting the cost of purchasing groceries based on the number of items purchased. You consider the regression model Yi | β0 , β1 , σ ∼ Normal(β0 + β1 xi , σ), where xi and yi are respectively the number of grocery items and the total cost (in dollars) of the i-th purchase. Use a conditional mean prior using the following information.

Let µ∗1 = β0 + β1 (10) and µ∗2 = β0 + β1 (30) denote the expected cost of purchasing 10 and 30 grocery items, respectively. You assume that your beliefs about µ∗1 and µ∗2 are independent where µ∗1 ∼ Normal(20, 5) and µ∗2 ∼ Normal(70, 5). (a) Use JAGS to simulate 1000 draws from the prior distribution of (β0 , β1 ). modelString = " model{ beta1 <- (mu2 - mu1) / (x2 - x1) beta0 <- mu1 - x1 * (mu2 - mu1) / (x2 - x1) mu1 ~ dnorm(m1, s1) mu2 ~ dnorm(m2, s2) }" the_data <- list("x1" = 10, "x2" = 30, "m1" = 20, "s1" = 0.04, "m2" = 70, "s2" = 0.04) initsfunction <- function(chain){ 260

}

.RNG.seed <- c(1,2)[chain] .RNG.name <- c("base::Super-Duper", "base::Wichmann-Hill")[chain] return(list(.RNG.seed=.RNG.seed, .RNG.name=.RNG.name))

prior <- run.jags(modelString, n.chains = 1, data = the_data, monitor = c("beta0", "beta1"), adapt = 1000, burnin = 5000, sample = 1000, inits = initsfunction) ## Compiling rjags model... ## Calling the simulation using the rjags method... ## Note: the model did not require adaptation ## Burning in the model for 5000 iterations... ## Running the model for 1000 iterations... ## Simulation complete ## Calculating summary statistics... ## Finished running the simulation (b) Construct a scatterplot of the values of β0 and β1 and describe the relationship that you see in the scatterplot.

From the constructed scatterplot, β0 and β1 is negatively related. prior <- as.mcmc(prior) df <- as.data.frame(cbind(prior[, "beta0"], prior[, "beta1"])) names(df) <- c("beta0", "beta1") ggplot(as.data.frame(df), aes(beta0, beta1)) + geom_point(color = "blue", alpha = 0.2) + increasefont()

261

3.5

beta1

3.0

2.5

2.0

1.5 −20

0

20

beta0 (c) Suppose you believe that your prior beliefs about the regression parameters are too strong. Choose a new conditional means prior that reflects this belief.

We can increase the standard deviation from 5 to bigger values to weaken prior beliefs. Various answers.

Exercise 17. Olympic Swimming Times The dataset olympic_butterfly.csv contains the winning time in seconds for the men’s and women’s 100m butterfly race for the Olympics from 1964 through 2016. Suppose we focus on the women’s times. If yi and xi denote respectively the winning time for the women’s 100m butterfly and year for the i-th Olympics, consider the use of the regression model Yi | β0 , β1 , σ ∼ Normal(β0 + β1 (xi − 1964), σ). (a) Give interpretations for the regression parameters β0 and β1 . β0 represents the mean time for the women’s 100m butterfly race in 1964 and β1 represents the increase in the mean time each year. (b) Assuming weakly informative priors for all parameters, use JAGS to simulate 5000 values from the joint posterior distribution. ## write the model modelString <-" model { ## sampling for (i in 1:N){ y[i] ~ dnorm(beta0 + beta1*x[i], invsigma2) }

262

## priors beta0 ~ dnorm(mu0, g0) beta1 ~ dnorm(mu1, g1) invsigma2 ~ dgamma(a, b) sigma <- sqrt(pow(invsigma2, -1)) } " womens_data <- filter(olympic_butterfly, Gender == "Women") the_data <- list(y = womens_data$Time, x = womens_data$Year - 1964, N = 14, mu0 = 0, g0 = 0.01, mu1 = 1, g1 = 0.01, a = 0.01, b = 0.01) require(runjags) posterior <- run.jags(modelString, n.chains = 1, data = the_data, monitor = c("beta0", "beta1", "sigma"), adapt = 1000, burnin = 5000, sample = 5000) ## Compiling rjags model... ## Calling the simulation using the rjags method... ## Note: the model did not require adaptation ## Burning in the model for 5000 iterations... ## Running the model for 5000 iterations... ## Simulation complete ## Calculating summary statistics... ## Finished running the simulation summary(posterior) ## Lower95 Median Upper95 Mean SD Mode MCerr ## beta0 62.6531114 63.9162719 65.1262756 63.9201100 0.62226410 NA 0.0225992911 ## beta1 -0.2137238 -0.1719438 -0.1322424 -0.1719319 0.02045695 NA 0.0007361326 ## sigma 0.7616030 1.1692957 1.7481687 1.2142396 0.27611373 NA 0.0051823827 ## MC%ofSD SSeff AC.10 psrf ## beta0 3.6 758 0.02715208 NA ## beta1 3.6 772 0.04954819 NA ## sigma 1.9 2839 -0.01847920 NA (c) Suppose one is interested in predicting the winning time for the women’s 100m butterfly the 2020 Olympics. Simulate 5000 draws from the posterior predictive distribution and construct a 90% prediction interval for this winning time. posterior %>% as.mcmc() %>% as.data.frame() -> post post %>% mutate(lp = beta0 + beta1 * (2020 - 1964), ys = rnorm(5000, lp, sigma)) %>% summarize(LO = quantile(ys, 0.05),

263

HI = quantile(ys, 0.95)) ## LO HI ## 1 51.95979 56.62828

Exercise 18. Olympic Swimming Times (continued) One way to judge the suitability of the linear model µi = β0 + β1 (xi − 1964) for the Olympics race data is to look for a pattern in the predictive residuals ri = yi − ŷi . (a) Using the draws from the posterior distribution of (β0 , β1 , σ), simulate a sample from the posterior predictive distribution of the future observation ỹi for all i using the algorithm described in Section ??. (b) Compute the sample of predictive residuals ri and find 90% interval estimates for each i. one_p_residual <- function(x, y){ lp <- post[ , "beta0"] + x * post[ , "beta1"] yp <- rnorm(5000, lp, post[, "sigma"]) residual <- y - yp s_residual <- quantile(residual, c(.05, .95)) data.frame(Value = x, LO = s_residual[1], HI = s_residual[2]) } require(purrr) df <- map2_df(the_data$x, the_data$y, one_p_residual) head(df) ## Value LO HI ## 1 52 -1.713550 2.740393 ## 2 48 -1.894316 2.532822 ## 3 44 -1.775011 2.565311 ## 4 40 -1.528798 2.804413 ## 5 36 -3.246070 1.026192 ## 6 32 -1.425292 2.778335 (c) Construct a graph of the 90% intervals for ri against xi . ggplot(df, aes(Value, LO)) + geom_linerange(aes(ymin = LO, ymax = HI), color = "blue", size = 1.3) + xlab("Year - 1964") + ylab("Predictive Residual") + geom_hline(yintercept = 0, color = "black", size = 1.5) + theme_grey(base_size = 18, base_family = "")

264

Predictive Residual

2.5

0.0

−2.5

0

10

20

30

40

50

Year − 1964 (d) Comment on any lack of fit of the linear model from looking at the residual graph.

The residuals look large when Year - 1964 is 4 and 8, and residual looks small when Year - 1964 is 12.

Exercise 19. Priors for Two-Group Model Returning to a tennis example described in Section 8.3, suppose one is interested in comparing the timeto-serve for two tennis servers Roger Federer and Rafael Nadal. One collects the time to serve yi for both players for many serves. Once assumes that Yi is distributed Normal(µi , σ) where µi = β0 + β1 xi , where xi is an indicator of the server where xi = 0 if Federer is serving and xi = 1 if Nadal is serving. In this setting, β0 is the mean time to serve for Federer and β1 is the increase in mean serving time for Nadal. (a) Construct a reasonable prior for the intercept parameter β0 .

Here β0 represents the mean time for Federer to serve. I believe Federer is a relatively fast server, so I assign β0 a normal prior with mean 15 sec and standard deviation 2 sec. (b) If you believe that Nadal is significantly slower than Federer in his time-to-serve, construct a prior for β1 that reflects this belief.

Here β1 represents the difference in the mean serving times for Nadal and Federer. I believe Nadal is, on average, 8 seconds slower than Federer, so I assign β1 a normal prior with mean 8 and standard deviation 2. (c) Suppose that the range of serving times is about 3 seconds. Construct a prior for σ that reflects this knowledge.

The range for a normal distribution is approximately 4 times the standard deviation, so I will assign σ a normal distribution with mean 0.75 and standard devation 0.20.

265

(d) Construct a joint prior for (β0 , β1 , σ) using the priors constructed in parts (a), (b), and (c).

By independence, my joint prior will be the product of the priors of β0 , β1 and σ.

Exercise 20. Two-Group Model (continued) In Exercise 19, you constructed an informative prior for (β0 , β1 , σ) for the regression model for the time-to-serve measurements for two tennis servers. The data file “two_players_time_to_serve.csv” contain measurements for 100 serves for the players Roger Federer and Rafael Nadal. Use JAGS to obtain a simulated sample of the posterior distribution using your prior and this data. Construct a 90% interval estimate for the regression slope β1 that measures the differences in the mean time to serve for the two players. model_script <- "model { ## sampling for (i in 1:N){ y[i] ~ dnorm(beta0 + beta1*x[i], invsigma2) } ## priors beta0 ~ dnorm(mu0, g0) beta1 ~ dnorm(mu1, g1) sigma ~ dnorm(mu2, g2) invsigma2 <- 1 / pow(sigma, 2) }" the_data <- list(y = two_players_time_to_serve$time, x = ifelse(two_players_time_to_serve$Player == "Federer", 0, 1), N = 100, mu0 = 15, g0 = 1 / 2 ^ 2, mu1 = 8, g1 = 1 / 2 ^ 2, mu2 = 0.75, g2 = 1 / 0.20 ^ 2) require(runjags) posterior <- run.jags(model_script, n.chains = 1, data = the_data, monitor = c("beta0", "beta1", "sigma"), adapt = 1000, burnin = 5000, sample = 5000) ## Compiling rjags model... ## Calling the simulation using the rjags method... ## Adapting the model for 1000 iterations... ## Burning in the model for 5000 iterations... ## Running the model for 5000 iterations... ## Simulation complete ## Calculating summary statistics... ## Finished running the simulation posterior %>% as.mcmc() %>% summary() ## ## Iterations = 6001:11000 ## Thinning interval = 1 ## Number of chains = 1

266

## Sample size per chain = 5000 ## ## 1. Empirical mean and standard deviation for each variable, ## plus standard error of the mean: ## ## Mean SD Naive SE Time-series SE ## beta0 16.443 0.4376 0.006188 0.01041 ## beta1 9.899 0.6128 0.008666 0.01419 ## sigma 3.210 0.1012 0.001431 0.00174 ## ## 2. Quantiles for each variable: ## ## 2.5% 25% 50% 75% 97.5% ## beta0 15.571 16.150 16.447 16.744 17.287 ## beta1 8.720 9.471 9.893 10.309 11.106 ## sigma 3.015 3.141 3.208 3.279 3.409

Chapter 12 Bayesian Multiple Regression and Logistic Models Exercise 1. Olympic Swimming Times The dataset olympic_butterfly.csv contains the winning time in seconds for the men’s and women’s 100m butterfly race for the Olympics from 1964 through 2016. Let yi and xi denote the winning time and year for the i-th Olympics. In addition, let wi denote an indicator variable that is 1 for the women’s race and 0 for the men’s race. Consider the regression model Yi ∼ Normal(µi , σ), where the mean is given by µi = β0 + β1 (xi − 1964) + β2 wi . (a) Interpret the parameter β0 in terms of the winning time in the race. β0 is the mean winning time in 1964 for men. (b) Interpret the parameter β0 + β2 . β0 + β2 is the mean winning time in 1964 for women. (c) Interpret the parameter β0 + 8β1 . β0 + 8β1 is the mean winning time in 1972 for men. (d) Interpret the parameter β0 + 8β1 + β2 . β0 + 8β1 + β2 is the mean winning time in 1972 for women.

Exercise 2. Olympic Swimming Times (continued) Consider the regression model for the 100m Olympic butterfly race times described in Exercise 1. Suppose the regression parameters β0 , β1 , β2 and the precision parameter φ = 1/σ 2 are assigned weakly informative priors. (a) Using JAGS, sample 5000 draws from the joint posterior distribution of all parameters. modelString_reg2 <-" model { for (i in 1:N){ y[i] ~ dnorm(beta0 + beta1 * x1[i] +

267

beta2 * x2[i], invsigma2) } beta0 ~ dnorm(mu0, g0) beta1 ~ dnorm(mu1, g1) beta2 ~ dnorm(mu2, g2) invsigma2 ~ dgamma(a, b) sigma <- sqrt(pow(invsigma2, -1)) } " library(ProbBayes) the_data <- list("y" = olympic_butterfly$Time, "x1" = olympic_butterfly$Year - 1964, "x2" = ifelse(olympic_butterfly$Gender == "Women", 1, 0), "N" = dim(olympic_butterfly)[1], "mu0" = 0, "g0" = 0.0025, "mu1" = 0, "g1" = 0.0025, "mu2" = 0, "g2" = 0.0025, "a" = 0.001, "b" = 0.001) library(runjags) posterior <- run.jags(modelString_reg2, n.chains = 1, data = the_data, monitor = c("beta0", "beta1", "beta2", "sigma"), adapt = 1000, burnin = 2000, sample = 5000) ## Compiling rjags model... ## Calling the simulation using the rjags method... ## Note: the model did not require adaptation ## Burning in the model for 2000 iterations... ## Running the model for 5000 iterations... ## Simulation complete ## Calculating summary statistics... ## Finished running the simulation (b) Construct 90% interval estimates for each of the regression coefficients. posterior %>% as.mcmc() %>% as.data.frame() -> post apply(post[, 1:3], 2, quantile, c(0.05, 0.50, 0.95)) ## beta0 beta1 beta2 ## 5% 56.07122 -0.1679180 5.704949 ## 50% 56.88815 -0.1456990 6.409022 ## 95% 57.65677 -0.1233962 7.129363 (c) Based on your work, describe how the mean winning time in the butterfly race has changed over time.

In addition, describe how the men times differ from the women times.

On the average, the winning time has decreased by 0.145 seconds per year. On average, the time for the women’s race is 6.4 seconds slower than the men’s times. (d) Construct 90% interval estimates for the predictive residuals ri = yi − ỹi where ỹi is simulated from the posterior predictive distribution. Plot these interval estimates and comment on any interval that does 268

not include zero.

The one_p_residual() function is a modification of the function used in Chapter 11. Since there are two predictors, the function is a function of x1, x2 and y. Also we will be plotting the predictive residual interval as a function of the mean linear predictor E(beta0 + beta1 * x1 + beta2 * x2) library(tidyverse) one_p_residual <- function(x1, x2, y){ lp <- post[ , "beta0"] + x1 * post[ , "beta1"] + x2 * post[, "beta2"] yp <- rnorm(5000, lp, post[, "sigma"]) residual <- y - yp s_residual <- quantile(residual, c(.05, .95)) data.frame(LP = mean(lp), LO = s_residual[1], HI = s_residual[2]) } x1 <- olympic_butterfly$Year - 1964 x2 <- ifelse(olympic_butterfly$Gender == "Women", 1, 0) y <- olympic_butterfly$Time x1 <- x1[-28] x2 <- x2[-28] y <- y[-28] df <- pmap_df(list(x1, x2, y), one_p_residual) ggplot(df, aes(LP, LO)) + geom_linerange(aes(ymin = LO, ymax = HI), color = "blue", size = 0.5) + xlab("Linear Predictor") + ylab("Predictive Residual") + geom_hline(yintercept = 0, color = "black", size = 1.5) + theme_grey(base_size = 18, base_family = "")

269

Predictive Residual

4

2

0

−2

52

56

60

Linear Predictor

We

note that one of the predictive residual intervals does not contain 0.

Exercise 3. Olympic Swimming Times (continued) For the 100m Olympic butterfly race times described in Exercise 1 consider the regression model where the mean race time has the form µi = β0 + β1 (xi − 1964) + β2 wi + β3 (xi − 1964)wi , where xi denotes the year for the i-th Olympics and wi denote an indicator variable that is 1 for the women’s race and 0 for the men’s race. (a) Write the expression for the mean time for the men’s race, and for the mean time for the women’s race.

Using this expressions, interpret the parameters β2 and β3 . mean time for men’s race is β0 + β1 (xi − 1964) mean time for women’s race is β0 + β2 + (β1 + β3 )(xi − 1964) β2 is increase in women’s mean time for 1964 β3 is additional increase in women’s mean time for each additional year (b) Using weakly informative priors for all parameters, use JAGS to draw a sample of 5000 draws from the joint posterior distribution.

270

modelString_reg3 <-" model { for (i in 1:N){ y[i] ~ dnorm(beta0 + beta1 * x1[i] + beta2 * x2[i] + beta3 * x3[i], invsigma2) } beta0 ~ dnorm(mu0, g0) beta1 ~ dnorm(mu1, g1) beta2 ~ dnorm(mu2, g2) beta3 ~ dnorm(mu3, g3) invsigma2 ~ dgamma(a, b) sigma <- sqrt(pow(invsigma2, -1)) } " library(ProbBayes) x1 <- olympic_butterfly$Year - 1964 x2 <- ifelse(olympic_butterfly$Gender == "Women", 1, 0) the_data <- list("y" = olympic_butterfly$Time, "x1" = x1, "x2" = x2, "x3" = x1 * x2, "N" = dim(olympic_butterfly)[1], "mu0" = 0, "g0" = 0.0025, "mu1" = 0, "g1" = 0.0025, "mu2" = 0, "g2" = 0.0025, "mu3" = 0, "g3" = 0.0025, "a" = 0.001, "b" = 0.001) library(runjags) posterior <- run.jags(modelString_reg3, n.chains = 1, data = the_data, monitor = c("beta0", "beta1", "beta2", "beta3", "sigma"), adapt = 1000, burnin = 2000, sample = 5000) ## Compiling rjags model... ## Calling the simulation using the rjags method... ## Note: the model did not require adaptation ## Burning in the model for 2000 iterations... ## Running the model for 5000 iterations... ## Simulation complete ## Calculating summary statistics... ## Finished running the simulation (c) Based on your work, is there evidence that the regression model between year and mean race time differs between men and women? posterior %>% as.mcmc() %>% as.data.frame() -> post apply(post[, 1:4], 2, quantile, c(0.05, 0.50, 0.95))

271

## beta0 beta1 beta2 beta3 ## 5% 54.81569 -0.13456020 7.189514 -0.11112668 ## 50% 55.72683 -0.10467102 8.420977 -0.07363726 ## 95% 56.68285 -0.07622451 9.619463 -0.03342713 We see that there is evidence that both β1 and β3 are nonzero – the intercept and slope of the women’s regression is different from that of the men.

Exercise 4. Prices of Personal Computers What factors determine the price of a personal computer in the early days? A sample of 500 personal computer sales was collected from 1993 to 1995 in the United States. In addition to the sale price (price in US dollars of 486 PCs), information on clock speed in MHz, size of hard drive in MB, size of Ram in MB, and premium status of the manufacturer (e.g. IBM, COMPAQ) was collected. The dataset is in ComputerPriceSample.csv.

Suppose one considers the regression model Yi ∼ Normal(µi , σ) where µi = β0 + β1 xi1 + β2 xi2 , yi is the sale price, xi1 is the clock speed, and x2i is the logarithm of the hard drive size. (a) Using a weakly informative prior on β = (β0 , β1 , β2 ) and σ, use JAGS to produce a simulated sample of size 5000 from the posterior distribution on (β, σ).

Use the same JAGs model script as before.

Note that the Price variable has been standardized to have mean 0 and standard deviation 1. the_data <- list("y" = with(ComputerPriceSample, (Price - mean(Price)) / sd(Price)), "x1" = ComputerPriceSample$Speed, "x2" = log(ComputerPriceSample$HardDrive), "N" = dim(ComputerPriceSample)[1], "mu0" = 0, "g0" = 0.0025, "mu1" = 0, "g1" = 0.0025, "mu2" = 0, "g2" = 0.0025, "a" = 0.001, "b" = 0.001) library(runjags) posterior <- run.jags(modelString_reg2, n.chains = 1, data = the_data, monitor = c("beta0", "beta1", "beta2", "sigma"), adapt = 1000, burnin = 2000, sample = 5000) ## Compiling rjags model... ## Calling the simulation using the rjags method... ## Note: the model did not require adaptation ## Burning in the model for 2000 iterations... ## Running the model for 5000 iterations... ## Simulation complete ## Calculating summary statistics... ## Finished running the simulation

272

(b) Obtain 95% interval estimates for β1 and β2 . posterior %>% as.mcmc() %>% as.data.frame() -> post apply(post[, 2:3], 2, quantile, c(0.025, 0.975)) ## beta1 beta2 ## 2.5% 0.003085136 0.5017633 ## 97.5% 0.011599366 0.7718132 (c) On the basis of your work, are both clock speed and hard drive size useful predictors of the sale price?

Both interval estimates do not include 0, so both clock speed and hard drive size are useful predictors.

Exercise 5. Prices of Personal Computers (continued) (a) Suppose a consumer is interested in a computer with a clock speed of 33 MHz and a 540 MB hard drive (so log 450 = 6.1). Simulate 5000 draws from the expected selling price β0 + β1 x1 + β2 for computer with this clock speed and hard drive size. Construct a 90% interval estimate for the expected sale price.

Note that we have rescaled price to have mean 0 and standard deviation 1, so these expected sale prices are on this standardized scale. lp <- post[ , "beta0"] + 33 * post[ , "beta1"] + log(450) * post[, "beta2"] quantile(lp, c(0.05, 0.95)) ## 5% ## -0.09667989

95% 0.10932180

(b) Instead suppose the consumer wishes to predict the selling price of a computer with this clock speed and hard drive size. Simulate 5000 draws from the posterior predictive distribution and use these simulated draws to find a 90% prediction interval.

Again note that the price is on the standardized scale. lp <- post[ , "beta0"] + 33 * post[ , "beta1"] + log(450) * post[, "beta2"] y <- rnorm(5000, lp, post[, "sigma"]) quantile(y, c(0.05, 0.95)) ## 5% ## -1.441968

95% 1.467227

Exercise 6. Salaries for Professors A sample contains the 2008-09 nine-month academic salary for Assistant Professors, Associate Professors and Professors in a college in the U.S. The data were collected as part of the on-going effort of the college’s administration to monitor salary differences between male and female faculty members. In addition to the nine-month salary (in US dollars), information on gender, rank (Assistant Professor, Associate Professor, Professor), discipline (A is “theoretical” and B is “applied”), years since PhD, and years of service were collected. The dataset is in ProfessorSalary.csv. Suppose that the salary of the i-th professor, yi , is distributed Normal with mean µi and standard deviation σ, where the mean is given by µi = β0 + β1 xi1 + β2 xi2 , where xi1 is the years of service and xi2 is the gender (where 1 corresponds to male and 0 to female).

273

(a) Assuming a weakly informative prior on β and σ, use JAGS to simulate a sample of 5000 draws from the posterior distribution on (β, σ).

Note that we have rescaled salary to have mean 0 and standard deviation 1, so these expected salaries are on this standardized scale. the_data <- list("y" = with(ProfessorSalary, (salary - mean(salary)) / sd(salary)), "x1" = ProfessorSalary$yrs.service, "x2" = ifelse(ProfessorSalary$sex == "Male", 1, 0), "N" = dim(ProfessorSalary)[1], "mu0" = 0, "g0" = 0.0025, "mu1" = 0, "g1" = 0.0025, "mu2" = 0, "g2" = 0.0025, "a" = 0.001, "b" = 0.001) posterior <- run.jags(modelString_reg2, n.chains = 1, data = the_data, monitor = c("beta0", "beta1", "beta2", "sigma"), adapt = 1000, burnin = 1000, sample = 5000) ## Compiling rjags model... ## Calling the simulation using the rjags method... ## Note: the model did not require adaptation ## Burning in the model for 1000 iterations... ## Running the model for 5000 iterations... ## Simulation complete ## Calculating summary statistics... ## Finished running the simulation (b) Simulate 1000 draws from the posterior of β0 + 10β1 , the mean salary among all female professors with 10 years of service.

Remember that we have standardized the salary variable, so these mean salary values are on the standardized scale. posterior %>% as.mcmc() %>% as.data.frame() -> post lp1 <- post$beta0 + 10 * post$beta1 (s_salary <- quantile(lp1, c(0.05, 0.95))) ## 5% 95% ## -0.7176871 -0.2257825 Put these values back to the salary scale. (salary <- mean(ProfessorSalary$salary) + s_salary * sd(ProfessorSalary$salary)) ## ##

5% 95% 91968.41 106867.72

(c) Simulate 1000 draws from the posterior of the mean salary of male professors with 10 years of service β0 + 10β1 + β2 .

274

lp1 <- post$beta0 + 10 * post$beta1 + post$beta2 (s_salary <- quantile(lp1, c(0.05, 0.95))) ## 5% 95% ## -0.25670365 -0.06281325 Put in salary scale: (salary <- mean(ProfessorSalary$salary) + s_salary * sd(ProfessorSalary$salary)) ## 5% 95% ## 105931.2 111803.9 (d) By comparing the intervals computed in parts (b) and (c), is there a substantial difference in the mean salaries of male and female professors with 10 years of service?

Exercise 7. Salaries for Professors (continued) (a) Suppose the college is interested in predicting the salary of a female professor with 10 years of service.

By simulating 5000 draws from the posterior predictive distribution, construct a 90% prediction interval for this salary. lp1 <- post$beta0 + 10 * post$beta1 y <- rnorm(5000, lp1, post$sigma) s_salary <- quantile(y, c(0.05, 0.95)) mean(ProfessorSalary$salary) + sd(ProfessorSalary$salary) * s_salary ## ##

5% 95% 52703.47 148011.84

(b) Use a similar method to obtain a 90% prediction interval for the salary of a male professor with 10 years of service. lp1 <- post$beta0 + 10 * post$beta1 + post$beta2 y <- rnorm(5000, lp1, post$sigma) s_salary <- quantile(y, c(0.05, 0.95)) mean(ProfessorSalary$salary) + sd(ProfessorSalary$salary) * s_salary ## ##

5% 95% 61949.64 154626.13

Exercise 8. Graduate School Admission What factors determine admission to graduate school? In a study, data on 400 graduate school admission cases was collected. Admission is a binary response, with 0 indicating not admitted, and 1 indicating admitted.

Moreover, the applicant’s GRE score, and undergraduate grade point average (GPA) are available. The dataset is in GradSchoolAdmission.csv (GRE score is out of 800). Let pi denote the probability that the i-th student is admitted. Consider the logistic model

pi log = β0 + β1 xi1 + β2 xi2 , 1 − pi where x1i and x2i are respectively the GRE score and the GPA for the i-th student.

275

(a) Assuming weakly informative priors on β0 , β1 , and β2 , write a JAGS script defining the Bayesian model. model_logistic <-" model { for (i in 1:N){ y[i] ~ dbern(p[i]) logit(p[i]) <- beta0 + beta1 * x1[i] + beta2 * x2[i] } beta0 ~ dnorm(0, 0.001) beta1 ~ dnorm(0, 0.001) beta2 ~ dnorm(0, 0.001) } " (b) Take a sample of 5000 draws from the posterior distribution of β = (β0 , β1 , β2 ). the_data <- list("y" = GradSchoolAdmission$Admission, "x1" = GradSchoolAdmission$GRE, "x2" = GradSchoolAdmission$GPA, "N" = dim(GradSchoolAdmission)[1]) posterior <- run.jags(model_logistic, n.chains = 1, data = the_data, monitor = c("beta0", "beta1", "beta2"), adapt = 1000, burnin = 5000, sample = 5000) ## Compiling rjags model... ## Calling the simulation using the rjags method... ## Adapting the model for 1000 iterations... ## Burning in the model for 5000 iterations... ## Running the model for 5000 iterations... ## Simulation complete ## Calculating summary statistics... ## Finished running the simulation (c) Consider a student with a 550 GRE score and a GPA of 3.50. Construct a 90% interval estimate for the probability that this student is admitted to graduate school. posterior %>% as.mcmc() %>% as.data.frame() -> post lp <- post$beta0 + 550 * post$beta1 + 3.50 * post$beta2 quantile(exp(lp) / (1 + exp(lp)), c(.05, .50, .95)) ## 5% 50% 95% ## 0.2583288 0.3005756 0.3470858 (d) Construct a 90% interval estimate for the probability a student with a 500 GRE score and a 3.2 GPA is admitted to graduate school. lp <- post$beta0 + 500 * post$beta1 + 3.20 * post$beta2 quantile(exp(lp) / (1 + exp(lp)), c(.05, .50, .95))

276

## 5% 50% 95% ## 0.1872427 0.2300535 0.2792892

Exercise 9. Graduate School Admission (continued) Consider the logistic model described in Exercise 8 where the logit probability of being admitted to graduate school is a linear function of his GRE score and GPA. It is assumed that JAGS is used to obtain a simulated sample from the posterior distribution of the regression vector. (a) Consider a student with a 580 GRE score. Construct 90% posterior interval estimates for the probability that this student achieves admission for GPA values equally spaced from 3.0 to 3.8. Graph these posterior interval estimates as a function of the GPA. prob_interval <- function(x2, x1, post){ lp <- post$beta0 + x1 * post$beta1 + x2 * post$beta2 quantile(exp(lp) / (1 + exp(lp)), c(.05, .95)) } out <- sapply(seq(3.0, 3.8, by = 0.02), prob_interval, 580, post) df_out <- data.frame(GPA = seq(3.0, 3.8, by = 0.02), Low = out[1, ], Hi = out[2, ]) ggplot(df_out) + geom_segment(aes(x = GPA, y = Low, xend = GPA, yend = Hi), size = 2) + ylab("Prob(Admission)") + ylim(0, 1) + ggtitle("Student with 580 GRE score")

277

Student with 580 GRE score 1.00

Prob(Admission)

0.75

0.50

0.25

0.00 3.0

3.2

3.4

3.6

3.8

GPA (b) Consider a student with a 3.4 GPA. Find 90% interval estimates for the probability this student is admitted for GRE score values equally spaced from 520 to 700. Graph these interval estimates as a function of the GRE score. prob_interval <- function(x1, x2, post){ lp <- post$beta0 + x1 * post$beta1 + x2 * post$beta2 quantile(exp(lp) / (1 + exp(lp)), c(.05, .95)) } out <- sapply(seq(520, 700, by = 10), prob_interval, 3.4, post) df_out <- data.frame(GRE = seq(520, 700, by = 10), Low = out[1, ], Hi = out[2, ]) ggplot(df_out) + geom_segment(aes(x = GRE, y = Low, xend = GRE, yend = Hi), size = 2) + ylab("Prob(Admission)") + ylim(0, 1) + ggtitle("Student with 3.4 GPA")

278

Student with 3.4 GPA 1.00

Prob(Admission)

0.75

0.50

0.25

0.00 550

600

650

700

GRE

Exercise 10. Personality Determinants of Volunteering In a study of the personality determinants of volunteering for psychological research, a subject’s neuroticism (scale from Eysenck personality inventory), extraversion (scale from Eysenck personality inventory), gender, and volunteering status were collected. One intends to find out what personality determinants affect a person’s volunteering choice. The dataset is in Cowles.csv. Let pi denote the probability that the i-th subject elects to volunteer. Consider the logistic model

pi log = β0 + β1 xi1 + β2 xi2 , 1 − pi where x1i and x2i are respectively the neuroticism and extraversion measures for the i-th subject. (a) Assuming weakly informative priors on β0 , β1 , and β2 , write a JAGS script defining the model and draw a sample of 5000 draws from the posterior distribution of β = (β0 , β1 , β2 ). the_data <- list("y" = ifelse(Cowles$volunteer == "yes", 1, 0), "x1" = Cowles$neuroticism, "x2" = Cowles$extraversion, "N" = dim(Cowles)[1]) posterior <- run.jags(model_logistic, n.chains = 1, data = the_data, monitor = c("beta0", "beta1", "beta2"), adapt = 1000, burnin = 5000, sample = 5000) 279

## Compiling rjags model... ## Calling the simulation using the rjags method... ## Adapting the model for 1000 iterations... ## Burning in the model for 5000 iterations... ## Running the model for 5000 iterations... ## Simulation complete ## Calculating summary statistics... ## Finished running the simulation (b) By inspecting the locations of the posterior distributions of β1 and β2 , which personality characteristic is most important in determining a person’s volunteering choice? posterior %>% as.mcmc() %>% as.data.frame() -> post apply(post, 2, quantile, c(0.05, 0.95)) ## beta0 beta1 beta2 ## 5% -1.6796422 -0.007832642 0.04274592 ## 95% -0.8450771 0.027785757 0.09285571 (c) Let O = p/(1 − p) denote the odds of volunteering.

Construct a 90% interval estimate for the odds a student with a neuroticism score of 12 and an extraversion score of 13 will elect to volunteer. lp <- post$beta0 + post$beta1 * 12 + post$beta2 * 13 p <- exp(lp) / (1 + exp(lp)) odds <- p / (1 - p) quantile(odds, c(0.05, 0.95)) ## 5% 95% ## 0.6908575 0.8267231

Exercise 11. The Divide by Four Rule Suppose one considers the logistic model log

p=

p 1−p

= β0 + β1 x. This model is rewritten as

exp(β0 + β1 x) .

1 + exp(β0 + β1 x)

(a) Show that the derivative of p with respect to x is written as dp = p(1 − p)β1 . dx (b) Suppose the probability is close to the value 0.5. Using part (a), what is the approximate derivative of p with respect to x in this region?

The approximate derivative of p with respect to x when p is close to 0.5 will be 0.5 (1 - 0.5) β1 . (c) Fill in the blank in the following sentence. In this logistic model, the quantity β1 /4 can be interpreted as the change in the probability when x increases by one unit. p (d) Suppose one is interested in fitting the logistic model log 1−p = β0 + β1 x where x is the number of study hours and p is the probability of passing an exam. One obtains the fitted model

log

p̂ = −1 + 0.2x.

1 − p̂

Using your work in parts (b) and (c), what is the (approximate) change in the fitted pass probability if a student studies an additional hour for the exam?

280

Since β1 = 0.2, the approximate change in the fitted probability would be 0.2 / 4 = 0.05.

Exercise 12. Football Field Goal Kicking The data file football_field_goal.csv contains data on field goal attempts for professional football kickers.

Focus on the kickers who played during the 2015 season. Let yi denote the response (success or failure) of a field goal attempt from xi yards. One is interested in fitting the logistic model log

pi = β0 + β1 x i , 1 − pi

where pi is the probability of a successful attempt. (a) Using weakly informative priors on β0 and β1 , use JAGS to take a simulated sample from the posterior distribution of (β0 , β1 ). modelString <-" model { ## sampling for (i in 1:N){ y[i] ~ dbern(p[i]) logit(p[i]) <- beta0 + beta1*x[i] } ## priors beta1 ~ dnorm(0, 0.0001) beta0 ~ dnorm(0, 0.0001) } " fg_2015 <- filter(football_field_goals, Year == 2015) the_data <- list("y" = fg_2015$Success, "x" = fg_2015$Distance, "N" = dim(fg_2015)[1]) posterior <- run.jags(modelString, n.chains = 1, data = the_data, monitor = c("beta0", "beta1"), adapt = 1000, burnin = 5000, sample = 5000) ## Compiling rjags model... ## Calling the simulation using the rjags method... ## Adapting the model for 1000 iterations... ## Burning in the model for 5000 iterations... ## Running the model for 5000 iterations... ## Simulation complete ## Calculating summary statistics... ## Finished running the simulation (b) Suppose a kicker is attempting a field goal from 40 yards. Construct a 90% interval estimate for the probability of a success. posterior %>% as.mcmc() %>% as.data.frame() -> post lp <- post$beta0 + 40 * post$beta1 p_success <- exp(lp) / (1 + exp(lp)) quantile(p_success, c(0.05, 0.95))

281

## 5% 95% ## 0.8542304 0.8934913 (c) Suppose instead that one is interested in estimating the yardage x∗ where the probability of a success is equal to 0.8. First express the yardage x∗ as a function of β0 and β1 , and then find a 90% interval estimate for x∗ .

Solve logit(0.8) = β0 + β1 x∗ for x∗ – get x∗ =

logit(0.8) − β0 β1

x <- (log(0.8/0.2) - post$beta0) / post$beta1 quantile(x, c(0.05, 0.95)) ## 5% 95% ## 43.56254 46.24555 (d) Suppose 50 field goals are attempted at a distance of 40 yards Simulate from the posterior predictive distribution to construct a 90% interval estimate for the number of successful attempts. lp <- post$beta0 + 40 * post$beta1 p_success <- exp(lp) / (1 + exp(lp)) ys <- rbinom(5000, size = 50, prob = p_success) quantile(ys, c(0.05, 0.95)) ## ##

5% 95% 39 47

Exercise 13. Predicting Baseball Batting Averages The data file batting_2018.csv contains batting data for every player in the 2018 Major League Baseball season. The variables AB.x and H.x in the dataset contain the number of at-bats (opportunities) and number of hits of each player in the first month of the baseball season. The variables AB.y and H.y in the dataset contain the at-bats and hits of each player for the remainder of the season.

Take a random sample of size 50 from batting_2018.csv. Suppose one is interested in predicting the players’ batting averages H.y/AB.y for the remainder of the season. Consider the following three estimates: • Individual Estimate: Use the player’s first month batting average H.x/AB.x.

P P • Pooled Estimate: Use the pooled estimate H.x/ AB.x.

• Compromise Estimate: Use the shrinkage estimate P AB.x H.x 135 H.x P + .

AB.x + 135 AB.x AB.x + 135 AB.x For your sample, compute values of the individual, pooled, and compromise estimates. For each set of estimates, compute the sum of squared prediction errors, where the prediction error is defined to be the difference between the estimate and the batting average in the remainder of the season. Which estimate do you prefer? Why? individual_estimates <- batting_2018$H.x / batting_2018$AB.x pooled_estimate <- sum(batting_2018$H.x) / sum(batting_2018$AB.x) frac <- batting_2018$AB.x / (batting_2018$AB.x + 135) compromise_estimates <- frac * individual_estimates + (1 - frac) * pooled_estimate

282

future_BA <- batting_2018$H.y / batting_2018$AB.y (pe_individual <- sum((individual_estimates - future_BA) ^ 2)) ## [1] 11.02634 (pe_combined <- sum((pooled_estimate - future_BA) ^ 2)) ## [1] 5.489044 (pe_compromise <- sum((compromise_estimates - future_BA) ^ 2)) ## [1] 5.258355 Here we would prefer the compromise estimate which has the smallest prediction error.

Exercise 14. Predicting Baseball Batting Averages (continued) In Exercise 13, for the i-th player in the sample of 50 one observes the number of hits yi (variable H.x) distributed binomial with sample size ni (variable AB.x) and probability of success pi . Consider the logistic model

pi log = γi .

1 − pi Use JAGS to simulate from the following three models: • Individual Model: Assume the γi values are distinct and assign each parameter a weakly informative normal distribution.

• Pooled Model: Assume that γ1 = ... = γ50 = γ and assign the single γ parameter a weakly informative normal distribution.

• Partially Pooled Hierarchical Model: Assume that γi ∼ N ormal(µ, τ ) where µ and the precision P = 1/τ 2 are assigned weakly informative distributions.

Focus on a particular player corresponding to the index k. Contrast 90% interval for estimates for pk using the individual, pooled, and partially pooled hierarchical models fit in parts (a), (b), and (c).

Fit of individual model: model_individual <-" model { ## sampling for (i in 1:N){ y[i] ~ dbin(p[i], n[i]) logit(p[i]) <- gam[i] } for (i in 1:N){ gam[i] ~ dnorm(0, 0.001) } } " the_data <- list("y" = batting_2018$H.x, "n" = batting_2018$AB.x, "N" = dim(batting_2018)[1]) posterior_ind <- run.jags(model_individual, n.chains = 1, data = the_data, monitor = c("gam"), adapt = 1000, 283

burnin = 5000, sample = 5000) ## Compiling rjags model... ## Calling the simulation using the rjags method... ## Adapting the model for 1000 iterations... ## Burning in the model for 5000 iterations... ## Running the model for 5000 iterations... ## Simulation complete ## Note: Summary statistics were not produced as there are >50 monitored ## variables ## [To override this behaviour see ?add.summary and ?runjags.options] ## FALSEFinished running the simulation Fit of pooled model: model_pooled <-" model { ## sampling for (i in 1:N){ y[i] ~ dbin(p[i], n[i]) logit(p[i]) <- gam } gam ~ dnorm(0, 0.001) } " the_data <- list("y" = batting_2018$H.x, "n" = batting_2018$AB.x, "N" = dim(batting_2018)[1]) posterior_pooled <- run.jags(model_pooled, n.chains = 1, data = the_data, monitor = c("gam"), adapt = 1000, burnin = 5000, sample = 5000) ## Compiling rjags model... ## Calling the simulation using the rjags method... ## Adapting the model for 1000 iterations... ## Burning in the model for 5000 iterations... ## Running the model for 5000 iterations... ## Simulation complete ## Calculating summary statistics... ## Finished running the simulation Fit of partially pooled model: model_pp <-" model { ## sampling for (i in 1:N){ y[i] ~ dbin(p[i], n[i]) logit(p[i]) <- gam[i] } for (i in 1:N){

284

gam[i] ~ dnorm(mu, P) } mu ~ dnorm(0, 0.0001) P ~ dgamma(1, 1) } " the_data <- list("y" = batting_2018$H.x, "n" = batting_2018$AB.x, "N" = dim(batting_2018)[1]) posterior_pp <- run.jags(model_pp, n.chains = 1, data = the_data, monitor = c("gam"), adapt = 1000, burnin = 5000, sample = 5000) ## Compiling rjags model... ## Calling the simulation using the rjags method... ## Adapting the model for 1000 iterations... ## Burning in the model for 5000 iterations... ## Running the model for 5000 iterations... ## Simulation complete ## Note: Summary statistics were not produced as there are >50 monitored ## variables ## [To override this behaviour see ?add.summary and ?runjags.options] ## FALSEFinished running the simulation Illustrate the three model fits for Jesus Aguilar (index is 6) who had AVG of 0.378 = 17 / 45 in first month of the season. posterior_ind %>% as.mcmc() %>% as.data.frame() -> post_ind posterior_pooled %>% as.mcmc() %>% as.data.frame() -> post_pooled posterior_pp %>% as.mcmc() %>% as.data.frame() -> post_pp Find posterior means of p6 under three models. invlogit <- function(x) {exp(x) / (1 + exp(x))} mean(invlogit(post_ind$`gam[6]`)) ## [1] 0.3803839 mean(invlogit(post_pooled$gam)) ## [1] 0.2448515 mean(invlogit(post_pp$`gam[6]`)) ## [1] 0.281047

285

Exercise 15. Comparing Career Trajectory Models In Section 12.3, the Deviance Information Criteria (DIC) was used to compare four regression models for Mike Schmidt’s career trajectory of home run rates. By fitting the model using JAGS and using the extract.runjags() function, find the DIC values for fitting the linear, cubic, and quartic models and compare your answers with the values in Table 12.2. For each model, assume that the regression parameters and the precision parameter have weakly informative priors.

Scripts for linear, cubic, quartic models: model_linear = " model { for (i in 1:N){ y[i] ~ dnorm(mu[i], phi) mu[i] <- beta0 + beta1 * (x[i] - 30) } beta0 ~ dnorm(0, 0.001) beta1 ~ dnorm(0, 0.001) phi ~ dgamma(0.001, 0.001) } " model_cubic = " model { for (i in 1:N){ y[i] ~ dnorm(mu[i], phi) mu[i] <- beta0 + beta1 * (x[i] - 30) + beta2 * pow(x[i] - 30, 2) + beta3 * pow(x[i] - 30, 3) } beta0 ~ dnorm(0, 0.001) beta1 ~ dnorm(0, 0.001) beta2 ~ dnorm(0, 0.001) beta3 ~ dnorm(0, 0.001) phi ~ dgamma(0.001, 0.001) } " model_quartic = " model { for (i in 1:N){ y[i] ~ dnorm(mu[i], phi) mu[i] <- beta0 + beta1 * (x[i] - 30) + beta2 * pow(x[i] - 30, 2) + beta3 * pow(x[i] - 30, 3) + beta4 * pow(x[i] - 30, 4) } beta0 ~ dnorm(0, 0.001) beta1 ~ dnorm(0, 0.001) beta2 ~ dnorm(0, 0.001) beta3 ~ dnorm(0, 0.001) beta4 ~ dnorm(0, 0.001) phi ~ dgamma(0.001, 0.001) } "

286

d <- filter(sluggerdata, Player == "Schmidt", AB >= 200) the_data <- list(y = d$HR / d$AB, x = d$Age, N = 16) Fit and extract DIC for each model. post_1 <- run.jags(model_linear, n.chains = 2, monitor = c("beta0", "beta1", "phi"), data = the_data) ## Compiling rjags model... ## Calling the simulation using the rjags method... ## Note: the model did not require adaptation ## Burning in the model for 4000 iterations... ## Running the model for 10000 iterations... ## Simulation complete ## Calculating summary statistics... ## Calculating the Gelman-Rubin statistic for 3 variables.... ## Finished running the simulation extract.runjags(post_1, "dic") ## Obtaining DIC samples... ## Mean deviance: -83.97 ## penalty 3.508 ## Penalized deviance: -80.46 post_3 <- run.jags(model_cubic, n.chains = 2, monitor = c("beta0", "beta1", "beta2", "beta3", "phi"), data = the_data) ## Compiling rjags model... ## Calling the simulation using the rjags method... ## Note: the model did not require adaptation ## Burning in the model for 4000 iterations... ## Running the model for 10000 iterations... ## Simulation complete ## Calculating summary statistics... ## Calculating the Gelman-Rubin statistic for 5 variables.... ## Finished running the simulation extract.runjags(post_3, "dic") ## Obtaining DIC samples... ## Mean deviance: -88.11 ## penalty 5.904 ## Penalized deviance: -82.21

287

post_4 <- run.jags(model_quartic, n.chains = 2, monitor = c("beta0", "beta1", "beta2", "beta3", "beta4", "phi"), data = the_data) ## Compiling rjags model... ## Calling the simulation using the rjags method... ## Note: the model did not require adaptation ## Burning in the model for 4000 iterations... ## Running the model for 10000 iterations... ## Simulation complete ## Calculating summary statistics... ## Calculating the Gelman-Rubin statistic for 6 variables.... ## Finished running the simulation extract.runjags(post_4, "dic") ## Obtaining DIC samples... ## Mean deviance: -86.16 ## penalty 7.29 ## Penalized deviance: -78.87

Exercise 16. Comparing Models for the CE Sample Example For the Consumer Expenditure Survey (CE) example, the objective was to learn about a CU’s expenditure based on the person’s income and his/her urban/rural status. There are four possible regression models depending on the inclusion or exclusion of each predictor. Use JAGS to fit each of the possible models and compute the value of DIC. For each model, assume that the regression parameters and the precision parameter have weakly informative priors. By comparing the DIC values, decide on the most appropriate model and compare your results with the discussion in Section 12.2.

Write scripts for four possible models that have inclusion or exclusion of each covariate. model_11 <-" model { ## sampling for (i in 1:N){ y[i] ~ dnorm(beta0 + beta1*x_income[i] + beta2*x_rural[i], invsigma2) } ## priors beta0 ~ dnorm(0, 0.001) beta1 ~ dnorm(0, 0.001) beta2 ~ dnorm(0, 0.001) invsigma2 ~ dgamma(0.001, 0.001) sigma <- sqrt(pow(invsigma2, -1)) } " model_10 <-" model { ## sampling for (i in 1:N){ 288

y[i] ~ dnorm(beta0 + beta1*x_income[i], invsigma2) } ## priors beta0 ~ dnorm(0, 0.001) beta1 ~ dnorm(0, 0.001) invsigma2 ~ dgamma(0.001, 0.001) sigma <- sqrt(pow(invsigma2, -1)) } " model_01 <-" model { ## sampling for (i in 1:N){ y[i] ~ dnorm(beta0 + beta2*x_rural[i], invsigma2) } ## priors beta0 ~ dnorm(0, 0.001) beta2 ~ dnorm(0, 0.001) invsigma2 ~ dgamma(0.001, 0.001) sigma <- sqrt(pow(invsigma2, -1)) } " model_00 <-" model { ## sampling for (i in 1:N){ y[i] ~ dnorm(beta0, invsigma2) } ## priors beta0 ~ dnorm(0, 0.001) invsigma2 ~ dgamma(1, 1) sigma <- sqrt(pow(invsigma2, -1)) } " J <- nrow(CEsample) CEsample %>% mutate(log_TotalIncome = log(TotalIncomeLastYear), log_TotalExp = log(TotalExpLastQ), Rural = ifelse(UrbanRural == 2, 1, 0)) %>% filter() -> CEsample y <- as.vector(CEsample$log_TotalExp) x_income <- as.vector(CEsample$log_TotalIncome) x_rural <- as.vector(CEsample$Rural) N <- length(y) Fit all models and compute the DIC measure. the_data <- list("y" = y, "N" = N) post_00 <- run.jags(model_00, n.chains = 2, data = the_data,

289

monitor = c("beta0", "sigma")) ## Compiling rjags model... ## Calling the simulation using the rjags method... ## Note: the model did not require adaptation ## Burning in the model for 4000 iterations... ## Running the model for 10000 iterations... ## Simulation complete ## Calculating summary statistics... ## Calculating the Gelman-Rubin statistic for 2 variables.... ## Finished running the simulation extract.runjags(post_00, "dic") ## Obtaining DIC samples... ## Mean deviance: 2632 ## penalty 1.976 ## Penalized deviance: 2634 the_data <- list("y" = y, "x_income" = x_income, "x_rural" = x_rural, "N" = N) post_11 <- run.jags(model_11, n.chains = 2, data = the_data, monitor = c("beta0", "sigma", "beta1", "beta2")) ## Compiling rjags model... ## Calling the simulation using the rjags method... ## Note: the model did not require adaptation ## Burning in the model for 4000 iterations... ## Running the model for 10000 iterations... ## Simulation complete ## Calculating summary statistics... ## Calculating the Gelman-Rubin statistic for 4 variables.... ## Finished running the simulation extract.runjags(post_11, "dic") ## Obtaining DIC samples... ## Mean deviance: 2313 ## penalty 3.978 ## Penalized deviance: 2317 the_data <- list("y" = y, "x_income" = x_income, "N" = N) post_10 <- run.jags(model_10, n.chains = 2, data = the_data, monitor = c("beta0", "sigma", "beta1")) ## Compiling rjags model... ## Calling the simulation using the rjags method... ## Note: the model did not require adaptation ## Burning in the model for 4000 iterations...

290

## Running the model for 10000 iterations... ## Simulation complete ## Calculating summary statistics... ## Calculating the Gelman-Rubin statistic for 3 variables.... ## Finished running the simulation extract.runjags(post_10, "dic") ## Obtaining DIC samples... ## Mean deviance: 2318 ## penalty 3.078 ## Penalized deviance: 2321 the_data <- list("y" = y, "x_rural" = x_rural, "N" = N) post_01 <- run.jags(model_01, n.chains = 2, data = the_data, monitor = c("beta0", "sigma", "beta2")) ## Compiling rjags model... ## Calling the simulation using the rjags method... ## Note: the model did not require adaptation ## Burning in the model for 4000 iterations... ## Running the model for 10000 iterations... ## Simulation complete ## Calculating summary statistics... ## Calculating the Gelman-Rubin statistic for 3 variables.... ## Finished running the simulation extract.runjags(post_01, "dic") ## Obtaining DIC samples... ## Mean deviance: 2624 ## penalty 2.994 ## Penalized deviance: 2627 Summarize . . . here are the four models and the corresponding DIC values. (a) null model, DIC = 2634 (b) only income, DIC = 2320 (c) only rural, DIC = 2637 (d) both income and rural, DIC = 2316 This indicate that both variables should be in the model.

Exercise 17. Grades in a Calculus Class Suppose one is interested in how the grade in a calculus class depends on the grade in the prerequisite math course. One is interested in fitting the logistic model

pi log = β0 + β1 x i , 1 − pi where pi is the probability of an A of the ith student and xi represents the grade of the ith student in the previous math class (1 if an A was received, and 0 otherwise).

291

(a) Suppose one believes a Beta(12, 8) prior reflects the belief about the probability of an A for a student who has received an A in the previous math, and a Beta(5, 15) prior reflects the belief about the probability of an A for a student who has not received an A in the previous course. Use JAGS to simulate 1000 draws from the prior of (β0 , β1 ).

Prior_Model <-" model { ## priors beta1 <- (logit(p1) - logit(p2)) / (x1 - x2) beta0 <- logit(p1) - beta1 * x1 p1 ~ dbeta(a1, b1) p2 ~ dbeta(a2, b2) } " the_data <- list("x1" = 1, "a1" = 12, "b1" = 8, "x2" = 0, "a2" = 5, "b2" = 15) prior <- run.jags(Prior_Model, n.chains = 1, data = the_data, monitor = c("beta0", "beta1"), adapt = 1000, burnin = 1000, sample = 1000) ## Compiling rjags model... ## Calling the simulation using the rjags method... ## Note: the model did not require adaptation ## Burning in the model for 1000 iterations... ## Running the model for 1000 iterations... ## Simulation complete ## Calculating summary statistics... ## Finished running the simulation prior %>% as.mcmc() %>% as.data.frame() %>% ggplot(aes(beta0, beta1)) + geom_point(alpha = 0.5)

292

4

3

beta1

2

1

0

−3

−2

−1

0

beta0 (b) Data for 100 students is contained in the data file “‘calculus.grades.csv}. Using JAGS to simulate 5000 draws from the posterior of (β0 , β1 ).

Posterior_Model <-" model { ## sampling for (i in 1:N){ y[i] ~ dbern(p[i]) logit(p[i]) <- beta0 + beta1*x[i] } ## priors beta1 <- (logit(p1) - logit(p2)) / (x1 - x2) beta0 <- logit(p1) - beta1 * x1 p1 ~ dbeta(a1, b1) p2 ~ dbeta(a2, b2) } " the_data <- list("y" = calculus.grades$grade, "x" = calculus.grades$prev.grade, "N" = dim(calculus.grades)[1], "x1" = 1, "a1" = 12, "b1" = 8, "x2" = 0, "a2" = 5, "b2" = 15)

293

posterior <- run.jags(Posterior_Model, n.chains = 1, data = the_data, monitor = c("beta0", "beta1"), adapt = 1000, burnin = 1000, sample = 1000) ## Compiling rjags model... ## Calling the simulation using the rjags method... ## Adapting the model for 1000 iterations... ## Burning in the model for 1000 iterations... ## Running the model for 1000 iterations... ## Simulation complete ## Calculating summary statistics... ## Finished running the simulation (c) Construct a 90% interval estimate for β1 . Is there evidence that the grade in the prerequisite math course is helpful in explaining the grade in the calculus class? posterior %>% as.mcmc() %>% as.data.frame() %>% summarize(LO = quantile(beta1, 0.05), HI = quantile(beta1, 0.95)) ## LO HI ## 1 1.246522 2.591747 Since this interval for β1 does not include 0, there is significant evidence that the previous grade is helpful in explaining the grade in the present course.

Exercise 18. Grades in a Calculus Class (continued) The traditional way of fitting the logistic model in Exercise 17 is by maximum likelihood. The variables grade and prev.grade contain the relevant variables in the data frame calculus.grades. The maximum likelihood is achieved by the function glm() with the family = binomial option. fit <- glm(grade ~ prev.grade, data = calculus.grades, family = binomial) summary(fit) Look at the estimates and associated standard errors of the regression coefficients and contrast these values with the posterior means and standard deviations from the informative prior Bayesian analysis in Exercise 17. fit <- glm(grade ~ prev.grade, data = calculus.grades, family = binomial) summary(fit) ## ## Call: ## glm(formula = grade ~ prev.grade, family = binomial, data = calculus.grades) ## ## Deviance Residuals: ## Min 1Q Median 3Q Max ## -1.7610 -0.8167 -0.8167 0.6905 1.5876 ## ## Coefficients:

294

## Estimate Std. Error z value Pr(>|z|) ## (Intercept) -0.9268 0.2710 -3.419 0.000628 *** ## prev.grade 2.2389 0.5048 4.436 9.18e-06 *** ## --## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1 ## ## (Dispersion parameter for binomial family taken to be 1) ## ## Null deviance: 137.63 on 99 degrees of freedom ## Residual deviance: 114.01 on 98 degrees of freedom ## AIC: 118.01 ## ## Number of Fisher Scoring iterations: 4 To compare, one can check if the estimate and standard error for the previous grade are similar to the posterior mean and standard deviation of β1 .

There are differences due to the subjective prior placed on (β0 , β1 ). posterior %>% as.mcmc() %>% as.data.frame() %>% summarize(Mean_Beta1 = mean(beta1), SD_Beta1 = sd(beta1)) ## Mean_Beta1 SD_Beta1 ## 1 1.912562 0.3983502

Exercise 19. Logistic Model to Compare Proportions In Exercise 19 of Chapter 7, one was comparing proportions of science majors for two years at some liberal arts colleges. One can formulation this problem in terms of logistic regression. Let yi denote the number of science majors out of a sample of ni for the ith year. One assumes that yi is distributed Binomial(ni , pi ) where pi satisfies the logistic model

pi log = β0 + β1 x i , 1 − pi where xi = 0 for year 2005 and x = 1 for year 2015. (a) Assuming that β0 and β1 are independent with weakly informative priors, use JAGS to simulate a sample of 5000 from the posterior distribution. (In the JAGS script, the dbin(p, n) denotes the Binomial distribution with probability p and sample size n.

Logistic_Model <-" model { ## sampling for (i in 1:2){ y[i] ~ dbin(p[i], n[i]) logit(p[i]) <- beta0 + beta1*x[i] } ## priors beta0 ~ dnorm(0, 0.001) beta1 ~ dnorm(0, 0.001) } " the_data <- list("y" = c(264, 437), "x" = c(0, 1), "n" = c(1496, 1495))

295

posterior <- run.jags(Logistic_Model, n.chains = 1, data = the_data, monitor = c("beta0", "beta1"), adapt = 1000, burnin = 5000, sample = 5000) ## Compiling rjags model... ## Calling the simulation using the rjags method... ## Adapting the model for 1000 iterations... ## Burning in the model for 5000 iterations... ## Running the model for 5000 iterations... ## Simulation complete ## Calculating summary statistics... ## Finished running the simulation (b) Find a 90% interval estimate for β1 . posterior %>% as.mcmc() %>% as.data.frame() -> post quantile(post$beta1, c(0.05, 0.50, 0.95)) ## 5% 50% 95% ## 0.5119185 0.6564891 0.8052840 (c) Use the result in (b) to describe how the proportion of science majors has changed (on the logit scale) from 2005 to 2015, Since the estimate of β1 is 0.66, the logit of the proportion of science majors has increased by 0.656 between 2005 and 2015.

Exercise 20. Separation in Logistic Regression Consider data in Table 12.4 that gives the number of class absences and the grade (1 for passing and 0 for failure) for ten students. If pi denotes the probability the ith student passes the class, then consider the logistic model

pi log = β0 + β1 x i , 1 − pi where xi is the number of absences.

Table 12.4. Number of absences and grades for ten students.

Student

Absences

Grade

Student

Absences

Grade

1 2 3 4 5

0 0 0 1 1

1 1 1 1 1

6 7 8 9 10

2 2 5 8 10

1 1 0 0 0

(a) Using the glm() function as shown in Exercise 18, find maximum likelihood estimates of β0 and β1 . grade <- c(1, 1, 1, 1, 1, 1, 1, 0, 0, 0) absences <- c(0, 0, 0, 1, 1, 2, 2, 5, 8, 10) fit <- glm(grade ~ absences, family = binomial) summary(fit)

296

## ## Call: ## glm(formula = grade ~ absences, family = binomial) ## ## Deviance Residuals: ## Min 1Q Median 3Q Max ## -1.555e-05 -1.050e-08 2.110e-08 2.110e-08 1.253e-05 ## ## Coefficients: ## Estimate Std. Error z value Pr(>|z|) ## (Intercept) 54.00 88663.27 0.001 1.000 ## absences -15.37 24466.58 -0.001 0.999 ## ## (Dispersion parameter for binomial family taken to be 1) ## ## Null deviance: 1.2217e+01 on 9 degrees of freedom ## Residual deviance: 5.5570e-10 on 8 degrees of freedom ## AIC: 4 ## ## Number of Fisher Scoring iterations: 24 (b) Comment on the output of implementing the glm() function. (The strange behavior is related to the problem of separation in logistic research.) Do some research on this topic and describe why one is observing this unusual behavior. (c) By use of a weakly informative prior, use JAGS to simulate a sample of 5000 from the posterior distribution.

Logistic_Model <-" model { ## sampling for (i in 1:10){ y[i] ~ dbern(p[i]) logit(p[i]) <- beta0 + beta1*x[i] } ## priors beta0 ~ dnorm(0, 0.001) beta1 ~ dnorm(0, 0.001) } " the_data <- list(y = grade, x = absences) posterior <- run.jags(Logistic_Model, n.chains = 1, data = the_data, monitor = c("beta0", "beta1"), adapt = 1000, burnin = 5000, sample = 5000) ## Compiling rjags model... ## Calling the simulation using the rjags method... ## Adapting the model for 1000 iterations... ## Burning in the model for 5000 iterations...

297

## Running the model for 5000 iterations... ## Simulation complete ## Calculating summary statistics... ## Finished running the simulation (d) Compute posterior means and standard deviations of β0 and β1 and compare your results with the traditional fit in part (a). posterior %>% as.mcmc() %>% as.data.frame() -> post apply(post, 2, mean) ## ##

beta0 beta1 37.87446 -12.85177

apply(post, 2, sd) ## beta0 ## 20.044868

beta1 7.608174

Chapter 13 Case Studies Exercise 1. Federalist Papers Word Study The frequencies of word use of Madison and Hamilton are stored in the data file fed_word_data.csv. Consider the counts of the word “there” in the 50 Federalist Papers authored by Hamilton. Suppose the count yi in the i-th paper is distributed Poisson with mean ni λ/1000 where ni is the number of words in the paper and λ is the rate of the word “there” per 1000 words. library(ProbBayes) federalist_word_study %>% filter(Authorship == "Hamilton", word == "there") -> df head(df) ## Name Total word N Rate Authorship Disputed ## 1 Federalist No. 1 1622 there 2 0.001233046 Hamilton no ## 2 Federalist No. 11 2511 there 8 0.003185982 Hamilton no ## 3 Federalist No. 12 2171 there 9 0.004145555 Hamilton no ## 4 Federalist No. 13 970 there 9 0.009278351 Hamilton no ## 5 Federalist No. 15 3095 there 18 0.005815832 Hamilton no ## 6 Federalist No. 16 2047 there 4 0.001954079 Hamilton no (a) Assuming a weakly informative prior for λ, use JAGS to fit this Poisson sampling model. library(runjags) modelString = " model{ for (i in 1:N) { y[i] ~ dpois(n[i] * lambda / 1000) } lambda ~ dgamma(0.001, 0.001) } " the_data <- list("y" = df$N, "n" = df$Total, N = length(df$N)) posterior <- run.jags(modelString,

298

data = the_data, monitor = c("lambda"), n.chains = 1, burnin = 2000, sample = 5000) ## Compiling rjags model... ## Calling the simulation using the rjags method... ## Note: the model did not require adaptation ## Burning in the model for 2000 iterations... ## Running the model for 5000 iterations... ## Simulation complete ## Calculating summary statistics... ## Finished running the simulation post <- as.mcmc(posterior) (b) Compute a 90% probability interval for the rate λ. quantile(post, c(.05, .95)) ## 5% 95% ## 3.121960 3.689618 (c) Consider a new essay with 1000 words. By simulating 1000 draws from the posterior predictive distribution, construct a 90% prediction interval for the number of occurrences of the word “there” in this essay. lambda <- post y_new <- rpois(5000, 1000 * lambda / 1000) quantile(y_new, c(.05, .95)) ## ##

5% 95% 1 7

Exercise 2. Federalist Papers Word Study (continued) Instead of Poisson sampling, suppose the count of the word yi “there” in the i-th Federalist paper is distributed Negative Binomial with parameters pi and α, where pi = β/(β + ni /1000) where ni is the number of words in the paper and α/β is the rate of the word “there” per 1000 words. (a) Using a suitable weakly informative prior for α and β, use JAGS to simulate 1000 draws from the posterior distribution. library(ProbBayes) federalist_word_study %>% filter(Authorship == "Hamilton", word == "there") -> df modelString = " model{ for(i in 1:N){ p[i] <- beta / (beta + n[i] / 1000) y[i] ~ dnegbin(p[i], alpha) } mu <- alpha / beta alpha ~ dgamma(.001, .001) 299

beta ~ dgamma(.001, .001) } " the_data <- list("y" = df$N, "n" = df$Total, N = length(df$N)) posterior <- run.jags(modelString, data = the_data, monitor = c("alpha", "beta"), n.chains = 1, burnin = 2000, sample = 1000) ## Compiling rjags model... ## Calling the simulation using the rjags method... ## Adapting the model for 1000 iterations... ## Burning in the model for 2000 iterations... ## Running the model for 1000 iterations... ## Simulation complete ## Calculating summary statistics... ## Finished running the simulation (b) Construct a 90% interval estimate for the rate parameter α/β. post <- as.data.frame(as.mcmc(posterior)) quantile(post$alpha / post$beta, c(0.05, 0.95)) ## 5% 95% ## 3.087336 3.688437 (c) By simulating from the posterior predictive distribution, construct a 90% prediction interval for the number of there’s in a new essay of 1000 words. p <- post$beta / (post$beta + 1000 / 1000) y <- rnbinom(1000, size = post$alpha, prob = p) quantile(y, c(.05, .95)) ## ##

5% 95% 1 7

(d) Compare your answers with the answers in Exercise assuming Poisson sampling.

The limits of the prediction intervals are the same in this example.

Exercise 3. Comparing Word Use Using Negative Binomial sampling models, compare the average word use of Hamilton and Madison for the words “this”, “on”, “his”, and “by”. Suppose the mean rate per 1000 words is measured by α1 /β1 and α2 /β2 for Hamilton and Madison, respectively. For each word, construct a 90% interval estimate for the difference in use rates D = α1 /β1 − α2 /β2 . By looking at the locations of these interval estimates, which words were more often used by Hamilton and which ones were more likely to be used by Madison? The data file is “‘fed_word_data.csv}. library(ProbBayes) library(runjags) modelString = " model{ for(i in 1:N1){ 300

p1[i] <- beta1 / (beta1 + n1[i] / 1000) y1[i] ~ dnegbin(p1[i], alpha1) } for(i in 1:N2){ p2[i] <- beta2 / (beta2 + n2[i] / 1000) y2[i] ~ dnegbin(p2[i], alpha2) } alpha1 ~ dgamma(.001, .001) beta1 ~ dgamma(.001, .001) alpha2 ~ dgamma(.001, .001) beta2 ~ dgamma(.001, .001) difference <- (alpha2 / beta2) - (alpha1 / beta1) } " bayes_one_word <- function(theword){ d1 <- filter(federalist_word_study, Authorship == "Hamilton", word == theword) d2 <- filter(federalist_word_study, Authorship == "Madison", word == theword) the_data <- list("y1" = d1$N, "n1" = d1$Total, "N1" = length(d1$N), "y2" = d2$N, "n2" = d2$Total, "N2" = length(d2$N)) posterior <- run.jags(modelString, data = the_data, monitor = c("difference"), n.chains = 1, burnin = 2000, sample = 5000) quantile(as.mcmc(posterior$mcmc), c(.05, .5, .95)) } bayes_one_word("this") ## Compiling rjags model... ## Calling the simulation using the rjags method... ## Adapting the model for 1000 iterations... ## Burning in the model for 2000 iterations... ## Running the model for 5000 iterations... ## Simulation complete ## Calculating summary statistics... ## Finished running the simulation ## 5% 50% 95% ## -2.2988859 -1.6234872 -0.9126625 bayes_one_word("on") ## Compiling rjags model... ## Calling the simulation using the rjags method... ## Adapting the model for 1000 iterations... ## Burning in the model for 2000 iterations...

301

## Running the model for 5000 iterations... ## Simulation complete ## Calculating summary statistics... ## Finished running the simulation ## 5% 50% 95% ## 3.472893 4.248638 5.070904 bayes_one_word("his") ## Compiling rjags model... ## Calling the simulation using the rjags method... ## Adapting the model for 1000 iterations... ## Burning in the model for 2000 iterations... ## Running the model for 5000 iterations... ## Simulation complete ## Calculating summary statistics... ## Finished running the simulation ## 5% 50% ## -2.2255453 -0.8528996

95% 0.4657973

bayes_one_word("by") ## Compiling rjags model... ## Calling the simulation using the rjags method... ## Adapting the model for 1000 iterations... ## Burning in the model for 2000 iterations... ## Running the model for 5000 iterations... ## Simulation complete ## Calculating summary statistics... ## Finished running the simulation ## 5% 50% 95% ## 3.106865 4.246584 5.407454

Exercise 4. Comparing Word Use (continued) As in Exercise , using Negative Binomial sampling models, compare the average word use of Hamilton and Madison for the words “this”, “on”, “his”, and “by”. If the mean rate per 1000 words is measured by α1 /β1 and α2 /β2 for Hamilton and Madison, respectively, suppose one is interested in comparing the rates using the ratio α1 /β1 R= . α2 /β2 Construct and graph 90% interval estimates for R for each word in the study. library(ProbBayes) library(runjags) modelString = " model{ for(i in 1:N1){ p1[i] <- beta1 / (beta1 + n1[i] / 1000) y1[i] ~ dnegbin(p1[i], alpha1) } for(i in 1:N2){ p2[i] <- beta2 / (beta2 + n2[i] / 1000)

302

y2[i] ~ dnegbin(p2[i], alpha2) } alpha1 ~ dgamma(.001, .001) beta1 ~ dgamma(.001, .001) alpha2 ~ dgamma(.001, .001) beta2 ~ dgamma(.001, .001) ratio <- (alpha2 / beta2) / (alpha1 / beta1) } " bayes_one_word <- function(theword){ d1 <- filter(federalist_word_study, Authorship == "Hamilton", word == theword) d2 <- filter(federalist_word_study, Authorship == "Madison", word == theword) the_data <- list("y1" = d1$N, "n1" = d1$Total, "N1" = length(d1$N), "y2" = d2$N, "n2" = d2$Total, "N2" = length(d2$N)) posterior <- run.jags(modelString, data = the_data, monitor = c("ratio"), n.chains = 1, burnin = 2000, sample = 5000) quantile(as.mcmc(posterior$mcmc), c(.05, .5, .95)) } bayes_one_word("this") ## Compiling rjags model... ## Calling the simulation using the rjags method... ## Adapting the model for 1000 iterations... ## Burning in the model for 2000 iterations... ## Running the model for 5000 iterations... ## Simulation complete ## Calculating summary statistics... ## Finished running the simulation ## 5% 50% 95% ## 0.7300378 0.8045837 0.8923351 bayes_one_word("on") ## Compiling rjags model... ## Calling the simulation using the rjags method... ## Adapting the model for 1000 iterations... ## Burning in the model for 2000 iterations... ## Running the model for 5000 iterations... ## Simulation complete ## Calculating summary statistics... ## Finished running the simulation

303

## 5% 50% 95% ## 1.959621 2.272658 2.635862 bayes_one_word("his") ## Compiling rjags model... ## Calling the simulation using the rjags method... ## Adapting the model for 1000 iterations... ## Burning in the model for 2000 iterations... ## Running the model for 5000 iterations... ## Simulation complete ## Calculating summary statistics... ## Finished running the simulation ## 5% 50% 95% ## 0.4837625 0.7390309 1.1323114 bayes_one_word("by") ## Compiling rjags model... ## Calling the simulation using the rjags method... ## Adapting the model for 1000 iterations... ## Burning in the model for 2000 iterations... ## Running the model for 5000 iterations... ## Simulation complete ## Calculating summary statistics... ## Finished running the simulation ## 5% 50% 95% ## 1.400894 1.575146 1.772971

Exercise 5. Basketball Shooting Data Table 13.3 displays the number of free throw attempts FTA and the number of successful free throws FT for all the seasons of Isiah Thomas, a great basketball point guard who played in the National Basketball Association from 1982 to 1994. This data is contained in the file nba\_guards.csv where the Player variable is equal to “THOMAS.” Let pj denote the probability of a successful free throw for the j-th season. Consider the quadratic logistic model

pj log = β0 + β1 (xj − 30) + β2 (xj − 30)2 , 1 − pj where xj is Thomas’ age during the j-th season.

Table 13.3. Free throw shooting data for the basketball player Isiah Thomas.

Age

FTA

FT

Age

FTA

FT

20 21 22 23 24 25 26

429 518 529 493 462 521 394

302 368 388 399 365 400 305

27 28 29 30 31 32

351 377 229 378 377 258

287 292 179 292 278 181

(a) By using JAGS with a reasonable choice of weakly informative prior on the regression parameters, 304

collect 5000 draws from the posterior distribution on β = (β0 , β1 , β2 ). require(runjags) require(tidyverse) require(coda) require(ProbBayes) script <- JAGS_script("trajectory") nba_guards %>% filter(Player == "THOMAS") -> thomas data <- list(y = thomas$FG, n = thomas$FGA, x = thomas$Age, N = length(thomas$Age)) posterior <- run.jags(script, n.chains = 1, data = data, monitor = c("beta0", "beta1", "beta2"), adapt = 1000, burnin = 5000, sample = 20000) ## Compiling rjags model... ## Calling the simulation using the rjags method... ## Adapting the model for 1000 iterations... ## Burning in the model for 5000 iterations... ## Running the model for 20000 iterations... ## Simulation complete ## Calculating summary statistics... ## Finished running the simulation (b) Construct a density estimate and a 90% interval estimate for the age h1 (β) where Thomas attained peak performance. (c) Construct a density estimate and a 90% interval estimate for the probability p that Thomas makes a free throw at age 28. posterior %>% as.mcmc() %>% as.data.frame() %>% mutate(peak.age = 30 - beta1 / 2 / beta2) -> dp ggplot(dp, aes(peak.age)) + geom_density() + xlim(17, 30)

305

0.6

density

0.4

0.2

0.0 20

24

28

peak.age quantile(dp$peak.age, c(0.05, 0.95)) ## 5% 95% ## 22.49122 25.35243

Exercise 6. Basketball Shooting Data (continued)} The dataset nba_guards.csv contains the number of free throw attempts FTA and the number of successful free throws FT for all of the seasons played by fifteen great point guards in the National Basketball Association.

Let pij denote the probability of a successful free throw of the i-th player during the j-th season. Suppose the probabilities {pij } for the i-th player satisfy the quadratic model

pij log = βi0 + βi1 (xij − 30) + βi2 (xij − 30)2 , 1 − pij where xij is the age of the i-th player during the j-th season and βi = (βi0 , βi1 , βi2 ) denotes the vector of regression coefficients for the i-th player. (a) Construct a hierarchical prior for the regression vectors β1 , ..., β15 analogous to the one used for baseball hitters in the chapter. (b) Use JAGS to simulate a sample of 5000 draws from the posterior distribution of the βj and also of the second stage prior µβ . require(runjags) require(tidyverse) require(coda) require(ProbBayes)

306

modelString <- JAGS_script("hier_trajectory") nba_guards$Player_Number <as.numeric(as.factor(nba_guards$Player)) b.data <- list(y = nba_guards$FG, x = nba_guards$Age, player = nba_guards$Player_Number, n = nba_guards$FGA, N = dim(nba_guards)[1], J = max(nba_guards$Player_Number), mean = c(0, 0, 0), Omega = diag(c(.1,.1,.1)), prec = diag(c(1.0E-6,1.0E-6,1.0E-6))) posterior <- run.jags(modelString, n.chains = 1, data = b.data, monitor = c("B", "mu.beta"), adapt = 1000, burnin = 5000, sample = 10000) ## Compiling rjags model... ## Calling the simulation using the rjags method... ## Adapting the model for 1000 iterations... ## Burning in the model for 5000 iterations... ## Running the model for 10000 iterations... ## Simulation complete ## Calculating summary statistics... ## Finished running the simulation (c) For one player, consider the age h1 (β) where he attained peak performance in free-throw shooting.

Compare the posterior distributions of h1 (β) using an individual logistic model and using the hierarchical model.

Illustrate for Thomas: posterior %>% as.mcmc() %>% as.data.frame() -> post peak_age <- 30 - post$`B[14,2]` / 2 / post$`B[14,3]` quantile(peak_age, c(0.05, 0.95)) ## 5% 95% ## 22.61726 25.17581

Exercise 7. Football Field Goal Kicking The data file football_field_goal.csv contains data on field goal attempts for professional football kickers.

Let yj denote the response (success or failure) of a field goal attempt from xj yards. One is interested in fitting the logistic model

pj log = β0 + β1 x j , 1 − pj where pj is the probability of a successful attempt. Figure 13.18 displays individual logistic fits for ten kickers in the 2005 season. These fits were found using weakly informative priors on the regression parameters β0 307

and β1 on individual fits for each player. (a) Looking at Figure 13.18, do you believe these individual fits of success probability against distance are suitable for all players? Explain. (b) For the player Brown, assuming this logistic model and a weakly informative prior on the regression coefficients, use JAGS to simulate from the posterior distribution. From the output to construct a 90 percent interval estimate for the probability of success at 30 yards.

Note: I’m just using Brown’s kicking data for the 2005 season. require(runjags) require(tidyverse) require(coda) require(ProbBayes) script <"model { for (j in 1:N){ y[j] ~ dbern(p[j]) logit(p[j]) <- beta0 + beta1 * x[j] } beta0 ~ dnorm(0, 0.0001) beta1 ~ dnorm(0, 0.0001) }" football_field_goals %>% filter(Year == 2005, Kicker == "Brown") -> brown data <- list(y = brown$Success, x = brown$Distance, N = length(brown$Success)) posterior <- run.jags(script, n.chains = 1, data = data, monitor = c("beta0", "beta1"), adapt = 1000, burnin = 5000, sample = 10000) ## Compiling rjags model... ## Calling the simulation using the rjags method... ## Adapting the model for 1000 iterations... ## Burning in the model for 5000 iterations... ## Running the model for 10000 iterations... ## Simulation complete ## Calculating summary statistics... ## Finished running the simulation posterior %>% as.mcmc() %>% as.data.frame() -> post x <- 30 lp <- post$beta0 + post$beta1 * 30 prob <- exp(lp) / (1 + exp(lp)) quantile(prob, c(.05, .95)) ##

5%

95% 308

## 0.7975737 0.9586797 (c) Pool the data for all 10 players, and use JAGS to fit from the logistic model where the probability of success is a function of the distance. Use JAGS to simulate from the posterior and from the output construct a 90 percent interval estimate for the probability of success at 30 yards.

Again, I am just using the data for the 2005 season. d_2005 <- filter(football_field_goals, Year == 2005) data <- list(y = d_2005$Success, x = d_2005$Distance, N = length(d_2005$Success)) posterior <- run.jags(script, n.chains = 1, data = data, monitor = c("beta0", "beta1"), adapt = 1000, burnin = 5000, sample = 10000) ## Compiling rjags model... ## Calling the simulation using the rjags method... ## Adapting the model for 1000 iterations... ## Burning in the model for 5000 iterations... ## Running the model for 10000 iterations... ## Simulation complete ## Calculating summary statistics... ## Finished running the simulation posterior %>% as.mcmc() %>% as.data.frame() -> post x <- 30 lp <- post$beta0 + post$beta1 * 30 prob <- exp(lp) / (1 + exp(lp)) quantile(prob, c(.05, .95)) ## 5% 95% ## 0.9004023 0.9364413 (d) Compare your answers to parts (b) and (c).

Exercise 8. Football Field Goal Kicking (continued) In the logistic model predicting success on a field goal attempt based on the distance in feet, suppose βi = (βi0 , βi1 ) denotes the regression vector for the logistic model on the i-th player. (a) Write down a hierarchical prior for the ten regression vectors β1 , ..., β10 similar to what was used for the baseball hitters in the chapter. (b) Using JAGS, simulate a sample of 5000 from the posterior distribution of β1 , ..., β10 .

I am using data from the 2005 season. library(dplyr) library(runjags) modelString = " model {

309

for (i in 1:N){ y[i] ~ dbern(p[i]) logit(p[i]) <- a[player[i]] + b[player[i]] * x[i] } for (j in 1:J){ a[j] <- B[j,1] b[j] <- B[j,2] B[j,1:2] ~ dmnorm (mu.beta[], Tau.B[,]) } mu.beta[1:2] ~ dmnorm(mean[1:2],prec[1:2 ,1:2 ]) Tau.B[1:2 , 1:2] ~ dwish(Omega[1:2 ,1:2], 2) }" d_2005 <- filter(football_field_goals, Year == 2005) d_2005$Player <- as.numeric(factor(d_2005$Kicker, labels = 1:39)) b.data <- list(y = d_2005$Success, x = d_2005$Distance, player = d_2005$Player, N = dim(d_2005)[1], J = max(d_2005$Player), mean = c(0, 0), Omega = diag(c(.1,.1)), prec = diag(c(1.0E-6,1.0E-6))) posterior <- run.jags(modelString, n.chains = 1, data = b.data, monitor = c("B"), adapt = 1000, burnin = 5000, sample = 10000) ## Compiling rjags model... ## Calling the simulation using the rjags method... ## Adapting the model for 1000 iterations... ## Burning in the model for 5000 iterations... ## Running the model for 10000 iterations... ## Simulation complete ## Note: Summary statistics were not produced as there are >50 monitored ## variables ## [To override this behaviour see ?add.summary and ?runjags.options] ## FALSEFinished running the simulation (c) Display the posterior means of the probabilities of success for all kickers as a function of distance similar to Figure ??. (d) For the player Brown, construct a 90 percent interval estimate for the probability of success at 30 yards.

Compare your answer to the individual fit (part (b) of Exercise ) and the pooled data fit (part (c) of Exercise ).

Exercise 9. Checking for Overdispersion In the hitter’s career trajectory example in Section ??, it was assumed that the number of on-base events in season yj was distributed Binomial with a specific probability of success pj where the {pj } satisfy a logistic 310

quadratic model. If one views a scatterplot of the observed rates OB/P A against age for Chase Utley (Figure ??), one notices some variation about the fitted curve. It is natural to ask if the variability about this line is greater than one would predict from the Binomial model. (a) Following the example in Section ??, obtain a posterior sample from the posterior distribution of β using Utley’s data and a weakly informative choice of prior distribution. library(runjags) library(ProbBayes) script <- JAGS_script("trajectory") source("get_onbase_data.R") utley <- get_onbase_data(1978, 1000) %>% filter(nameLast == "Utley") head(utley) ## nameLast Age AgeD PA OB Player ## 1 Utley 24 -6 152 49 33 ## 2 Utley 25 -5 286 88 33 ## 3 Utley 26 -4 628 236 33 ## 4 Utley 27 -3 739 280 33 ## 5 Utley 28 -2 612 251 33 ## 6 Utley 29 -1 706 268 33 the_data <- list(y = utley$OB, n = utley$PA, N = dim(utley)[1], x = utley$Age) posterior <- run.jags(script, n.chains = 1, data = the_data, monitor = c("beta0", "beta1", "beta2"), adapt = 1000, burnin = 2000, sample = 5000) ## Compiling rjags model... ## Calling the simulation using the rjags method... ## Adapting the model for 1000 iterations... ## Burning in the model for 2000 iterations... ## Running the model for 5000 iterations... ## Simulation complete ## Calculating summary statistics... ## Finished running the simulation (b) Compute the posterior mean of β and obtain an estimate for the on-base probability for all of Utley’s ages. posterior %>% as.mcmc() %>% as.data.frame() -> post mean_beta <- apply(post, 2, mean) lp <- mean_beta[1] + mean_beta[2] * (utley$Age - 30) + mean_beta[3] * (utley$Age - 30) ^ 2 prob <- exp(lp) / (1 + exp(lp)) data.frame(Age = utley$Age, On_base_prob = prob) ## ## 1 ## 2 ## 3

Age On_base_prob 24 0.3457852 25 0.3576772 26 0.3672933

311

## 4 ## 5 ## 6 ## 7 ## 8 ## 9 ## 10 ## 11 ## 12 ## 13 ## 14 ## 15 ## 16

27 28 29 30 31 32 33 34 35 36 37 38 39

0.3745429 0.3793635 0.3817164 0.3815837 0.3789662 0.3738843 0.3663788 0.3565150 0.3443869 0.3301223 0.3138884 0.2958960 0.2764025

(c) Write a function to simulate one replicated sample of yjR from the posterior predictive distribution.

Compute the sum of squares of the rates yjR /nj about the fitted on-base probabilities. one_rep <- function(j){ beta <- unlist(post[j, ]) lp <- beta[1] + beta[2] * (utley$Age - 30) + beta[3] * (utley$Age - 30) ^ 2 p <- exp(lp) / (1 + exp(lp)) N <- length(p) yp <- rbinom(N, size = utley$PA, prob = p) sum((yp / utley$PA - p) ^ 2) } (d) Using the function written in part (c), repeat this simulation 1000 times, obtaining 1000 sum of squares values from the posterior predictive distribution. many_sim <- sapply(1:1000, one_rep) (e) By comparing the posterior predictive sample in part (d) with the observed sum of squares about the fitted curve, comment about the suitability of the Binomial sampling model. observed <- sum((utley$OB / utley$PA - prob)^ 2) hist(many_sim) abline(v = observed, col = "red", lwd = 3)

312

150 100 0

50

Frequency

200

250

Histogram of many_sim

0.005

0.010

0.015

0.020

0.025

0.030

many_sim Exercise 10. Moby Dick Word Study Project Gutenberg offers a large number of free electronic books and the gutenbergr R package facilitates reading in these books into R for textual analysis. Use this package to download the famous novel Moby Dick by Herman Melville and collect all of the distinct words in this novel in a data frame. library(gutenbergr) library(tidyverse) library(tidytext) library(stringr) moby <- gutenberg_download(gutenberg_id = 2701) Extract words from rows 11 through 23104 of data frame. Using tidytext package to extract individual words. moby[11:23104, ] %>% unnest_tokens(word, text, to_lower = TRUE) -> tidy_text Remove numbers: tidy_text %>% filter(str_detect(word, "[0-9]") == FALSE) -> tidy_text (a) Divide the words of the novel into 1000-word groups and count the number of occurrences of a particular filler word in each group. tidy_text %>% mutate(Group = round(row_number() / 1000) + 1) %>% group_by(Group) %>%

313

summarize(Total = n(), N_by = sum(word == "by"), N_with = sum(word == "with")) -> S (b) Use a Negative Binomial distribution to model the counts of the filler word across groups. library(runjags) modelString = " model{ for(i in 1:N){ p[i] <- beta / (beta + n[i] / 1000) y[i] ~ dnegbin(p[i], alpha) } mu <- alpha / beta alpha ~ dgamma(.001, .001) beta ~ dgamma(.001, .001) } " the_data <- list("y" = S$N_by, "n" = S$Total, N = length(S$Total)) posterior <- run.jags(modelString, data = the_data, monitor = c("mu"), n.chains = 1, burnin = 2000, sample = 1000) ## Compiling rjags model... ## Calling the simulation using the rjags method... ## Adapting the model for 1000 iterations... ## Burning in the model for 2000 iterations... ## Running the model for 1000 iterations... ## Simulation complete ## Calculating summary statistics... ## Finished running the simulation summary(posterior) ## Lower95 Median Upper95 Mean SD Mode MCerr MC%ofSD SSeff ## mu 5.15997 5.542189 5.932453 5.548195 0.1994506 NA 0.006625331 3.3 906 ## AC.10 psrf ## mu -0.0389207 NA (c) Consider the use of a different filler word, and use a Negative Binomial distribution to model the counts of this new word across 1000-word groups. the_data <- list("y" = S$N_with, "n" = S$Total, N = length(S$Total)) posterior2 <- run.jags(modelString, data = the_data, monitor = c("mu"), n.chains = 1, burnin = 2000, sample = 1000) ## Compiling rjags model...

314

## Calling the simulation using the rjags method... ## Adapting the model for 1000 iterations... ## Burning in the model for 2000 iterations... ## Running the model for 1000 iterations... ## Simulation complete ## Calculating summary statistics... ## Finished running the simulation summary(posterior2) ## Lower95 Median Upper95 Mean SD Mode MCerr MC%ofSD SSeff ## mu 7.618405 7.979706 8.395431 7.988636 0.2031061 NA 0.008187036 4 615 ## AC.10 psrf ## mu -0.01923357 NA (d) By use of a suitable model, compare the rates (per 1000 words) of the two types of filler words. Construct a 90% interval estimate for the ratio of rates of use of the two words. library(coda) posterior %>% as.mcmc() %>% as.data.frame() -> post1 posterior2 %>% as.mcmc() %>% as.data.frame() -> post2 ratio <- post2$mu / post1$mu quantile(ratio, c(0.05, 0.95)) ## 5% 95% ## 1.337332 1.549920

Exercise 11. An Outlier Model Suppose one observes a sample measurement data where there is a small possibility of an outlier. One observes yi which is either Normal(µ, σ) with probability p or Normal(µ, 3σ) with probability 1 − p. Assume a weakly informative prior on µ and p is Beta(2, 20). Introduce latent class assignments where zi = 0 or zi = 1 depending if the observation is Normal(µ, σ) or Normal(µ, 3σ). Use JAGS with a script similar to used to Section ?? to fit this model. The dataset darwin.csv in the ProbBayes R package contains measurements on the differences of the heights of cross and self fertilized plants quoted by Fisher (1960). Compute the posterior probability that each observation is an outlier. Plot the measurements against these posterior outlier probabilities and comment on the pattern in the scatterplot.

Here we let the latent variable z be normal with precision φ/9 or normal with precision φ. library(ProbBayes) library(runjags) modelString<-" model { ## sampling for (i in 1:N){ theta[i] <- phi * (equals(z[i], 1) / 9 + equals(z[i], 0) * 1) y[i] ~ dnorm(mu, theta[i]) } for (i in 1:N){ z[i] ~ dbern(p) } ## priors p ~ dbeta(2, 20) mu ~ dnorm(0, 0.001)

315

phi ~ dgamma(0.01, 0.01) } " the_data <- list(y = darwin$difference, N = dim(darwin)[1]) posterior <- posterior <- run.jags(modelString, n.chains = 1, data = the_data, monitor = c("z"), adapt = 1000, burnin = 1000, sample = 5000) ## Compiling rjags model... ## Calling the simulation using the rjags method... ## Note: the model did not require adaptation ## Burning in the model for 1000 iterations... ## Running the model for 5000 iterations... ## Simulation complete ## Calculating summary statistics... ## Finished running the simulation posterior %>% as.mcmc() %>% as.data.frame() -> post darwin %>% mutate(Prob_Out = apply(post, 2, mean)) -> darwin ggplot(darwin, aes(difference, Prob_Out)) + geom_point()

0.5

Prob_Out

0.4

0.3

0.2

0.1

−40

0

difference

316

40

80

Exercise 12. Another Latent Data Model Suppose n students are taking a multiple-choice exam of m questions. As in Section ??, suppose there are two types of students, the “guessing” students and the “knowledgeable” students who answer each question correct with respective probabilities of p0 and p1 . The following R code will simulate some data in this scenario where n = 50, m = 20, p0 = 0.4, p1 = 0.7 and the probability that a student is a guesser is π = 0.2 (p in the JAGS script). set.seed(123) p0 <- 0.40; p1 <- 0.70 m <- 20; n <- 50; p <- 0.2 z <- sample(0:1, size = n, prob = c(p, 1-p), replace = TRUE) prob <- (z == 0) * p0 + (z == 1) * p1 y <- rbinom(n, size = m, prob) By use of a latent class model similar to what was used in Section ??, simulate from the joint posterior distribution of all parameters. Estimate values of p0 , p1 , and π from the posterior and compare these estimates with the “true” values of the parameters used in the simulation. modelString<-" model { ## sampling for (i in 1:n){ theta[i] <- equals(z[i], 1) * p1 + equals(z[i], 0) * p0 y[i] ~ dbin(theta[i], m) } for (i in 1:n){ z[i] ~ dbern(q) } ## priors p1 ~ dbeta(1, 1) T(0, 1) p0 ~ dbeta(1,1) T(0, p1) q ~ dbeta(1, 1) }" the_data <- list("y" = y, "n" = 20, "m" = 50) posterior <- run.jags(modelString, n.chains = 1, data = the_data, monitor = c("z", "p1", "p0", "q"), adapt = 1000, burnin = 5000, sample = 5000) ## Compiling rjags model... ## Calling the simulation using the rjags method... ## Adapting the model for 1000 iterations... ## Burning in the model for 5000 iterations... ## Running the model for 5000 iterations... ## Simulation complete ## Calculating summary statistics... ## Finished running the simulation S <- summary(posterior) S[, "Mean"]

317

## z[1] z[2] z[3] z[4] z[5] z[6] z[7] z[8] ## 0.7664000 0.6766000 0.5974000 0.2346000 0.3976000 0.7380000 0.7330000 0.4454000 ## z[9] z[10] z[11] z[12] z[13] z[14] z[15] z[16] ## 0.5488000 0.7104000 0.4402000 0.7614000 0.7104000 0.7086000 0.5874000 0.3844000 ## z[17] z[18] z[19] z[20] p1 p0 q ## 0.5950000 0.5928000 0.5900000 0.3876000 0.2792929 0.1943556 0.5755040

Exercise 13. Determining Authorship From a Single Word In the dataset federalist_word_study.csv}, the variableAuthorship} indicates the authorship of the Federalist Papers and the variable “‘Disputed} indicates the papers where the authorship is disputed.

Following the work in Section ?? and the JAGS script, fit a latent variable model using the filler word “from”.

Using this particular word, examine the posterior probabilities of authorship for the ten papers of disputed authorship. Is this single word helpful for determining authorship for any of the papers? Repeat this analysis using the filler word “this”.

In the JAGS model script below, there are N1 papers attributed to Hamilton, N2 papers attributed to Madison, and N3 papers of disputed authorship. For the ith Hamilton paper, y1[i] denotes the frequency of the particular word and n1[i] denotes the total word count. For the ith Madison paper, y2[i] denotes the frequency of the word and n2[i] denotes the total word count. For the ith paper of disputed authorship, y3[i] denotes the frequency of the word and n3[i] denotes the total word count.

Here 49 papers are clearly Hamilton, 15 papers are clearly Madison, and 10 are disputed authorship. modelString = " model{ for(i in 1:N1){ p1[i] <- beta1 / (beta1 + n1[i] / 1000) y1[i] ~ dnegbin(p1[i], alpha1) } for(i in 1:N2){ p2[i] <- beta2 / (beta2 + n2[i] / 1000) y2[i] ~ dnegbin(p2[i], alpha2) } for(i in 1:N3){ theta[i] <- equals(z[i], 0) * alpha1 + equals(z[i], 1) * alpha2 gamma[i] <- equals(z[i], 0) * beta1 + equals(z[i], 1) * beta2 p[i] <- gamma[i] / (gamma[i] + n[i] / 1000) y[i] ~ dnegbin(p[i], theta[i]) z[i] ~ dbern(0.5) } alpha1 ~ dgamma(.001, .001) beta1 ~ dgamma(.001, .001) alpha2 ~ dgamma(.001, .001) beta2 ~ dgamma(.001, .001) } " federalist_word_study %>% filter(word == "can") -> fed_can N1 <- 49 y1 <- filter(fed_can, Authorship == "Hamilton") %>% 318

select(N) %>% pull() n1 <- filter(fed_can, Authorship == "Hamilton") %>% select(Total) %>% pull() N2 <- 15 y2 <- filter(fed_can, Authorship == "Madison", Disputed == "no") %>% select(N) %>% pull() n2 <- filter(fed_can, Authorship == "Madison", Disputed == "no") %>% select(Total) %>% pull() N3 <- 10 y <- filter(fed_can, Disputed == "yes") %>% select(N) %>% pull() n <- filter(fed_can, Disputed == "yes") %>% select(Total) %>% pull() the_data <- list("y1" = y1, "n1" = n1, "N1" = N1, "y2" = y2 , "n2" = n2, "N2" = N2, "y" = y, "n" = n, "N3" = N3) jags <- run.jags(modelString, data = the_data, monitor = c("z"), n.chains = 1, burnin = 2000, sample = 5000) ## Compiling rjags model... ## Calling the simulation using the rjags method... ## Adapting the model for 1000 iterations... ## Burning in the model for 2000 iterations... ## Running the model for 5000 iterations... ## Simulation complete ## Calculating summary statistics... ## Finished running the simulation post <- as.mcmc(jags) apply(post, 2, mean) -> pmeans ggplot(data.frame(Paper = 1:10, Mean = pmeans), aes(Paper, Mean)) + geom_point(color = "blue", size = 3) + increasefont() + ylim(0, 1) + scale_x_continuous(breaks=seq(1, 10))

319

1.00

Mean

0.75

0.50

0.25

0.00 1

2

3

4

5

6

7

8

9

10

Paper Exercise 14. Determining Authorship From Multiple Words Suppose one wishes to use all of the filler words “by”, “from”, “to”, “an”, “any”, “may”, “his”, “upon”, “also”, “can”, “of”, “on”, “there”, and“this” to determine authorship of the ten disputed papers. Using the approach described at the end of Section ??, write a JAGS script to fit the latent variable model. Collect a posterior sample from the posterior distribution of the classification variables. Use the posterior means of the classification variables to determine authorship for each of the ten variables.

In the JAGS model script below, there are N1 papers attributed to Hamilton, N2 papers attributed to Madison, and N3 papers of disputed authorship. For the ith Hamilton paper, y1[i, j] denotes the frequency of word j and n1[i] denotes the total word count. For the ith Madison paper, y2[i, j] denotes the frequency of word j and n2[i] denotes the total word count. For the ith paper of disputed authorship, y3[i, j] denotes the frequency of word j and n3[i] denotes the total word count. modelString = " model{ for(i in 1:N1){ for(j in 1:K) { p1[i, j] <- beta1[j] / (beta1[j] + n1[i] / 1000) y1[i, j] ~ dnegbin(p1[i, j], alpha1[j]) }} for(i in 1:N2){ for(j in 1:K) { p2[i, j] <- beta2[j] / (beta2[j] + n2[i] / 1000) y2[i, j] ~ dnegbin(p2[i, j], alpha2[j]) }} for(i in 1:N3){ 320

for(j in 1:K){ theta[i, j] <- equals(z[i], 0) * alpha1[j] + equals(z[i], 1) * alpha2[j] gamma[i, j] <- equals(z[i], 0) * beta1[j] + equals(z[i], 1) * beta2[j] p[i, j] <- gamma[i, j] / (gamma[i, j] + n[i] / 1000) y[i, j] ~ dnegbin(p[i, j], theta[i, j]) }} for(i in 1:N3){ z[i] ~ dbern(0.5) } for(j in 1:K){ alpha1[j] ~ dgamma(.001, .001) beta1[j] ~ dgamma(.001, .001) alpha2[j] ~ dgamma(.001, .001) beta2[j] ~ dgamma(.001, .001) } }" Below we illustrate working with four words (K = 4). The below code sets up the matrices y1, y2, y3 and the vectors n1, n2, n3. word_list <- c("by", "from", "to", "an", "any", "may", "his", "upon", "also", "can", "of", "on", "there", "this") many_words <- federalist_word_study %>% filter(word %in% word_list[1:4]) Hamilton <- filter(many_words, Authorship == "Hamilton") Madison <- filter(many_words, Authorship == "Madison") Disputed <- filter(many_words, Disputed == "yes") n1 <- Hamilton$Total select(Hamilton, Name, word, N) %>% pivot_wider(names_from = word, values_from = N) %>% select(-Name) %>% as.matrix() -> d1 n2 <- Madison$Total select(Madison, Name, word, N) %>% pivot_wider(names_from = word, values_from = N) %>% select(-Name) %>% as.matrix() -> d2 n3 <- Disputed$Total select(Disputed, Name, word, N) %>% pivot_wider(names_from = word, values_from = N) %>% select(-Name) %>% as.matrix() -> d3 Here is the code for setting up the data to be input into JAGS and the run.jags() implementation. the_data <- list("y1" = d1, "n1" = n1, "N1" = 51,

321

"y2" = d2, "n2" = n2, "N2" = 29, "y" = d3, "n" = n3, "N3" = 12, "K" = 4) jags <- run.jags(modelString, data = the_data, monitor = c("z"), n.chains = 1, burnin = 2000, sample = 5000) ## Compiling rjags model... ## Calling the simulation using the rjags method... ## Adapting the model for 1000 iterations... ## Burning in the model for 2000 iterations... ## Running the model for 5000 iterations... ## Simulation complete ## Calculating summary statistics... ## Finished running the simulation S <- summary(jags) S[, "Mean"] ## z[1] z[2] z[3] z[4] z[5] z[6] z[7] z[8] z[9] z[10] z[11] ## 0.5150 0.8538 0.8430 0.7920 0.1966 0.1906 0.1988 0.6930 0.7796 0.7424 0.2974 ## z[12] ## 0.2318

322