Knowledge Game Title

Knowledge Game: Laying the foundations for (more) adequate simulations in social epistemology Alette Holmberg-Nielsen – Søren Lind Kristiansen March 14, 2012 – Version 680.

Abstract Traditionel epistemologi beskæftiger sig med viden i et subjekt. Målet er blandt andet at afdække betingelserne for, at sætninger som ‘S ved p’, ‘S tror p’ og ‘S ved, at S ved p’ er sande. Indenfor traditionel epistemologi er man ikke nødvendigvis optaget af det sociale aspekt af viden. Det er man imidlertid indenfor social epistemologi. Udveksling af viden er et socialt fænomen, som er under indflydelse af en lang række af sociale faktorer som for eksempel tillid til sig selv, tillid mellem agenter, løgne og gruppepres.

Én ide indenfor social epistemologi er at bruge computere til at simulere disse komplekse sociale processer, og dermed få mulighed for at lave eksperimenter, som ville være svære at gennemføre med virkelige mennesker. Et eksempel på sådan en simulator er Laputa, som er udviklet af Erik Olsson og Staffan Angere fra Lunds Universitet.

Der er imidlertid flere udfordringer, når man vil simulere sociale processer generelt og i særdeleshed udveksling af viden. For det første er der langt fra opnået konsensus om, hvordan den menneskelige bevidsthed fungerer, når vi taler om at forme og opdatere formodninger. For det andet er de sociale interaktioner mellem mennesker ekstremt komplicerede og betinget af en hel del forskellige ting, såsom humør, følelser, intelligens og en hel masse mere.

Disse udfordringer betyder, at det langt fra er simpelt at lave en adækvat model af agenter og interaktioner mellem dem. Og selv hvis det skulle lykkes én at lave en adækvat model, så er det også en udfordring at afgøre hvilke sociale processer, der så skal simuleres ved hjælp af denne model. Hverken modellen eller processen, som skal bruges i simulatoren er altså enkel at konstruere, så den er tilstrækkelig adækvat – og dette er da heller ikke lykkedes i Laputa.

I dette speciale vil vi lægge grundlaget for mere adækvate simulationer i social epistemologi. Vi vil give et alternativt bud på, hvordan agenter og deres interaktioner kan modelleres, og vi vil imødegå de problemer, som findes i Laputa. Vi vil teste denne model ved at lade agenterne indgå i en simpel, kontrolleret proces, nemlig et spil. Dette spil kalder vi Knowledge Game. Alle disse ting har vi bygget ind i en simulator, som er skrevet i C# og i sidste del af dette speciale vil vi se på en håndfuld interessante eksperimenter kørt i denne simulator.

Parts I

Motivation

14

II

Designing the Simulator

46

III

Running the Simulator

119

IV

Further work

142

V

Appendix

177

1

Contents Abstract

1

List of Figures

9

Responsibility

11

I

14

Motivation

1 Introduction 1.1 Simulations in social epistemology 1.2 Our aim . . . . . . . . . . . . . .

1.2.1 Faithfulness and adequacy 1.2.2 Two steps . . . . . . . . .

1.3 An overview . . . . . . . . . . . .

. . . . .

. . . . .

. . . . .

2 Laputa 2.1 Simulations in social epistemology . . .

2.2 Framework . . . . . . . . . . . . . . . .

2.3 The model . . . . . . . . . . . . . . . .

2.4 Challenges . . . . . . . . . . . . . . . .

2.4.1 Liars . . . . . . . . . . . . . . .

2.4.1.1 Example . . . . . . . .

2.4.2 Divine Inspiration vs. reasoning 2.4.3 Brute force . . . . . . . . . . .

2.5 Conclusions . . . . . . . . . . . . . . .

. . . . .

. . . . . . . . .

. . . . .

. . . . . . . . .

. . . . .

. . . . . . . . .

. . . . .

. . . . . . . . .

. . . . .

. . . . . . . . .

. . . . .

. . . . . . . . .

. . . . .

. . . . . . . . .

. . . . .

. . . . . . . . .

. . . . .

. . . . . . . . .

. . . . .

. . . . . . . . .

. . . . .

. . . . . . . . .

. . . . .

. . . . . . . . .

. . . . .

15 15 16 17 19 19

. . . . . . . . .

22 22 23 24 26 26 27 28 29 30

3 What to expect from a simulator 31 3.1 What is a simulation? . . . . . . . . . . . . . . . . . . . . . . 31 2

3.2

3.3

3.4

3.5

3.1.1 Our raytracer . . . . . . . . . . . . . . .

What to expect from a computer . . . . . . . .

3.2.1 Input determines output . . . . . . . . .

3.2.2 Our raytracer . . . . . . . . . . . . . . .

Unpredictability . . . . . . . . . . . . . . . . . .

3.3.1 Surprise . . . . . . . . . . . . . . . . . .

3.3.2 Simulation results vs the world . . . . .

3.3.2.1 Simple simulator error . . . . .

3.3.2.2 Missing known laws or forces .

3.3.2.2.1 Our raytracer . . . . .

3.3.2.3 Missing unknown laws or forces 3.3.3 No difference . . . . . . . . . . . . . . .

What can we learn? . . . . . . . . . . . . . . . .

3.4.1 Supervenience . . . . . . . . . . . . . . .

3.4.2 Comparing to the real world . . . . . . .

Making it explicit . . . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . .

32 34 34 35 36 37 38 38 38 38 41 41 42 42 43 43

4 What did we just learn?

45

II

46

Designing the Simulator

5 Introduction 5.1 An adequate and relevant model .

5.1.1 Actual or optimal? . . . .

5.2 Building a game . . . . . . . . . .

5.3 Introducing Knowledge Game . .

5.3.1 Entities . . . . . . . . . .

5.3.2 Game statement . . . . .

5.3.3 Acquiring new knowledge 5.3.4 The Representations . . .

5.3.5 Common sense realism . .

5.3.6 Definition of Knowledge .

5.3.6.1 Gettier examples 5.3.6.2 Justification . . .

5.3.7 Known Unknowns . . . . .

5.3.8 Hesperus is Phosphorus .

3

. . . . . . . . . . . . . .

. . . . . . . . . . . . . .

. . . . . . . . . . . . . .

. . . . . . . . . . . . . .

. . . . . . . . . . . . . .

. . . . . . . . . . . . . .

. . . . . . . . . . . . . .

. . . . . . . . . . . . . .

. . . . . . . . . . . . . .

. . . . . . . . . . . . . .

. . . . . . . . . . . . . .

. . . . . . . . . . . . . .

. . . . . . . . . . . . . .

. . . . . . . . . . . . . .

. . . . . . . . . . . . . .

. . . . . . . . . . . . . .

47 47 48 49 51 52 52 53 54 54 56 57 59 61 63

6 Agents 6.1 What is an agent? . . . . . . . . . . . . . . . .

6.1.1 Agent diversity . . . . . . . . . . . . . .

6.1.2 Are agents like humans? . . . . . . . . .

6.2 Actions . . . . . . . . . . . . . . . . . . . . . .

6.2.1 Inference . . . . . . . . . . . . . . . . . .

6.2.2 Investigation vs inquiries . . . . . . . . .

6.2.2.1 Specific entity . . . . . . . . . .

6.2.2.2 Go Fish . . . . . . . . . . . . .

6.2.2.3 Quality of investigations . . . .

6.2.3 Inquiries . . . . . . . . . . . . . . . . . .

6.3 Trust . . . . . . . . . . . . . . . . . . . . . . . .

6.4 Trust in Knowledge Game . . . . . . . . . . . .

6.4.1 Assertion . . . . . . . . . . . . . . . . .

6.4.1.1 Calculating assertion . . . . . .

6.4.1.2 Bias . . . . . . . . . . . . . . .

6.5 Liars . . . . . . . . . . . . . . . . . . . . . . . .

6.5.1 TruthProbability . . . . . . . . . . . . .

6.6 Epistemic profiles . . . . . . . . . . . . . . . . .

6.6.1 Agents . . . . . . . . . . . . . . . . . . .

6.6.1.1 AssertionProbabilityThreshold 6.6.1.2 EvidenceBias . . . . . . . . . .

6.6.1.3 SelfTrust . . . . . . . . . . . .

6.6.1.4 InvestigationQuality . . . . . .

6.6.1.5 InvestigationProbability . . . .

6.6.2 Communication lines . . . . . . . . . . .

6.6.2.1 Trust . . . . . . . . . . . . . .

6.6.2.2 TruthProbability . . . . . . . .

7 Designing a logic 7.1 Introduction . . . . . . . . . . . . .

7.2 Decidability . . . . . . . . . . . . .

7.3 Expression trees . . . . . . . . . . .

7.3.1 2 + 3 * 4 . . . . . . . . . .

7.4 Context-free grammars . . . . . . .

7.4.1 The language of ab. . . . . .

7.4.1.1 Applying the rules 4

. . . . . . .

. . . . . . .

. . . . . . .

. . . . . . .

. . . . . . .

. . . . . . .

. . . . . . .

. . . . . . . . . . . . . . . . . . . . . . . . . . .

. . . . . . .

. . . . . . . . . . . . . . . . . . . . . . . . . . .

. . . . . . .

. . . . . . . . . . . . . . . . . . . . . . . . . . .

. . . . . . .

. . . . . . . . . . . . . . . . . . . . . . . . . . .

. . . . . . .

. . . . . . . . . . . . . . . . . . . . . . . . . . .

. . . . . . .

. . . . . . . . . . . . . . . . . . . . . . . . . . .

. . . . . . .

. . . . . . . . . . . . . . . . . . . . . . . . . . .

. . . . . . .

. . . . . . . . . . . . . . . . . . . . . . . . . . .

65 65 65 66 68 69 70 70 72 73 73 75 77 79 81 82 84 85 86 86 87 87 87 88 88 88 89 89

. . . . . . .

90 90 92 93 93 95 96 97

7.5 7.6 7.7

7.4.1.2 Example .

7.4.2 Adding parentheses Building a tree . . . . . .

Getting the priorities right Operator associativity . .

8 Our monadic predicate 8.1 Syntax . . . . . . . .

8.1.1 Example . . .

8.1.2 Syntax . . . .

. . . . .

. . . . .

. . . . .

. . . . .

. . . . .

. . . . .

. . . . .

. . . . .

. . . . .

. . . . .

. . . . .

. . . . .

. . . . .

. . . . .

. . . . .

. . . . .

. . . . .

. . . . .

. . . . .

. . . . .

99 99 99 102 105

logic . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

107 . 107 . 107 . 109

9 Model checker 9.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . .

9.2 The task at hand . . . . . . . . . . . . . . . . . . . . . . . .

9.2.1 Known unknowns . . . . . . . . . . . . . . . . . . . .

9.2.2 Tautologies, valid expressions and contradictions . . .

9.3 Top-down . . . . . . . . . . . . . . . . . . . . . . . . . . . .

9.3.1 Example . . . . . . . . . . . . . . . . . . . . . . . . .

9.4 Semi-trivalent system . . . . . . . . . . . . . . . . . . . . . .

9.4.1 Propagating unknowns . . . . . . . . . . . . . . . . .

9.4.1.1 Both p and p are known . . . . . . . . . . .

9.4.1.2 Either p is known and q is unknown or vice versa . . . . . . . . . . . . . . . . . . . . . .

9.4.1.3 Both p and q are unknown . . . . . . . . . .

111 . 111 . 111 . 111 . 112 . 113 . 113 . 115 . 116 . 116 . 117 . 117

III

119

Running the Simulator

10 Experiments in Knowledge Game 120 10.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . 120 11 How the smart beat the stupid 11.1 Description . . . . . . . . . .

11.2 Experiment definition . . . . .

11.3 Agents . . . . . . . . . . . . .

11.4 Results . . . . . . . . . . . . .

11.5 Analysis . . . . . . . . . . . .

5

. . . . .

. . . . .

. . . . .

. . . . .

. . . . .

. . . . .

. . . . .

. . . . .

. . . . .

. . . . .

. . . . .

. . . . .

. . . . .

. . . . .

. . . . .

. . . . .

. . . . .

122 . 122 . 123 . 123 . 123 . 124

12 How the outspoken beat the careful 12.1 Description . . . . . . . . . . . . .

12.2 Experiment definition . . . . . . . .

12.3 Agents . . . . . . . . . . . . . . . .

12.4 Results . . . . . . . . . . . . . . . .

12.5 Analysis . . . . . . . . . . . . . . .

. . . . .

13 How the stupid and outspoken beat 13.1 Description . . . . . . . . . . . . .

13.2 Experiment definition . . . . . . . .

13.3 Agents . . . . . . . . . . . . . . . .

13.4 Results . . . . . . . . . . . . . . . .

13.5 Analysis . . . . . . . . . . . . . . .

the smart and careful 128 . . . . . . . . . . . . . . . 128 . . . . . . . . . . . . . . . 129 . . . . . . . . . . . . . . . 129 . . . . . . . . . . . . . . . 130 . . . . . . . . . . . . . . . 130

14 How the stupid and outspoken beat revisited 14.1 Description . . . . . . . . . . . . .

14.2 Experiment definition . . . . . . . .

14.3 Agents . . . . . . . . . . . . . . . .

14.4 Results . . . . . . . . . . . . . . . .

14.5 Analysis . . . . . . . . . . . . . . .

the smart and careful – 132 . . . . . . . . . . . . . . . 132 . . . . . . . . . . . . . . . 132 . . . . . . . . . . . . . . . 133 . . . . . . . . . . . . . . . 133 . . . . . . . . . . . . . . . 134

15 Saints and sinners 15.1 Description . . . . .

15.2 Experiment definition 15.3 Agents . . . . . . . .

15.4 Network . . . . . . .

15.5 Results . . . . . . . .

15.6 Analysis . . . . . . .

. . . . . .

. . . . . .

. . . . . .

. . . . . .

. . . . . .

. . . . . .

. . . . . .

. . . . . .

. . . . . .

. . . . .

. . . . . .

. . . . .

. . . . . .

. . . . .

. . . . . .

. . . . .

. . . . . .

. . . . .

. . . . . .

. . . . .

. . . . . .

. . . . .

. . . . . .

. . . . .

. . . . . .

. . . . .

. . . . . .

. . . . .

. . . . . .

. . . . .

. . . . . .

. . . . .

. . . . . .

. . . . .

. . . . . .

125 . 125 . 125 . 126 . 126 . 127

135 . 135 . 136 . 136 . 136 . 137 . 137

16 Conclusions 139 16.1 Conclusions . . . . . . . . . . . . . . . . . . . . . . . . . . . . 139

IV

Further work

142

17 Introduction 143 17.1 First version of our simulation software . . . . . . . . . . . . . 143 17.2 What to do next? . . . . . . . . . . . . . . . . . . . . . . . . . 143 6

18 Dynamic trust 18.1 Introduction . . . . . . . . . . . . .

18.2 Keeping track of statements . . . .

18.3 Rough ideas for an implementation 18.3.1 Memory quality . . . . . . .

18.3.2 Other adjustments needed .

. . . . .

145 . 145 . 145 . 146 . 149 . 149

19 Patience 19.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . .

19.2 The Patience Feature . . . . . . . . . . . . . . . . . . . . . .

19.3 Saints and sinners revisited . . . . . . . . . . . . . . . . . . .

151 . 151 . 151 . 154

20 Diversifying agents’ logical abilities 20.1 Imperfect logicians . . . . . . . . . . . . . . .

20.2 Perfect logicians . . . . . . . . . . . . . . . . .

20.3 Satisfiability solver . . . . . . . . . . . . . . .

20.4 Wason selection task . . . . . . . . . . . . . .

20.4.1 Solution . . . . . . . . . . . . . . . . .

20.4.2 Wason’s results and Knowledge Game

. . . . . .

157 . 157 . 157 . 158 . 159 . 159 . 161

. . . . . . . . .

162 . 162 . 164 . 164 . 165 . 167 . 167 . 167 . 170 . 171

22 Independence 22.1 What is independence? . . . . . . . . . . . . . . . . . . . . .

22.2 Why does independence matter? . . . . . . . . . . . . . . . .

22.3 Implementing independence . . . . . . . . . . . . . . . . . .

172 . 172 . 173 . 173

. . . . .

. . . . .

. . . . .

. . . . .

. . . . .

. . . . .

. . . . .

. . . . . .

. . . . .

. . . . . .

. . . . .

. . . . . .

. . . . .

. . . . . .

. . . . .

. . . . . .

. . . . .

. . . . . .

21 Peer disagreement 21.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . .

21.2 Implementing update strategies in Knowledge Game . . .

21.2.1 Disagreement . . . . . . . . . . . . . . . . . . . .

21.2.1.1 Disagreement defined by differing ratios 21.2.1.2 Disagreement defined by communication 21.2.2 Epistemic peers . . . . . . . . . . . . . . . . . . .

21.2.3 Is the existing implementation conformism? . . .

21.2.4 Non-conformism . . . . . . . . . . . . . . . . . . .

21.2.5 Updating when not disagreeing . . . . . . . . . .

23 Conclusion

. . . . .

. . . . . .

. . . . . . . . .

174

7

V

Appendix

177

A Scandal of deduction 178 A.1 The asylum . . . . . . . . . . . . . . . . . . . . . . . . . . . . 179 B Muddy Children Problem B.1 Introduction . . . . . . . . . . . . . . . . . . .

B.2 The puzzle . . . . . . . . . . . . . . . . . . . .

B.2.1 Solution . . . . . . . . . . . . . . . . .

B.2.1.1 Assuming one dirty child . . .

B.2.1.2 Assuming two dirty children .

B.2.1.3 Assuming three dirty children B.2.1.4 The social part . . . . . . . .

B.2.2 General problem and solution . . . . .

B.3 Further reading . . . . . . . . . . . . . . . . .

Bibliography

. . . . . . . . .

. . . . . . . . .

. . . . . . . . .

. . . . . . . . .

. . . . . . . . .

. . . . . . . . .

. . . . . . . . .

. . . . . . . . .

182 . 182 . 182 . 183 . 184 . 184 . 185 . 185 . 186 . 186 187

8

List of Figures 2.1

An example of a network in Laputa. Note that the edges are directed. That is, Sherlock Holmes can talk to Mrs. Hudson, but Mrs. Hudson cannot talk to Sherlock Holmes. . . . . . . . 25

3.1

The results of a raytracer implemented using an inadequate model of light. . . . . . . . . . . . . . . . . . . . . . . . . . . . 35 The results of a raytracer implemented using a better model of light. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 39 The results of the raytracer using ambient light. . . . . . . . . 40

3.2 3.3 5.1 5.2 5.3 5.4 5.5 5.6

It is a small world. . . . . . . . . . . . . . . . . . . . . . . .

An agent with no knowledge of the world. . . . . . . . . . .

An agent with a little knowledge of the world. . . . . . . . .

Illustration of the relationship between the world and the representations of it. . . . . . . . . . . . . . . . . . . . . . . . .

A Gettier example as it can be constructed in our system. .

An agent. . . . . . . . . . . . . . . . . . . . . . . . . . . . .

6.1 6.2

A world with eight objects. . . . . . . . . . . . . . . . . . . . . 71 The agent as an onion. . . . . . . . . . . . . . . . . . . . . . . 81

7.1 7.2

Correct tree for 2 + 3 ∗ 4. . . . . . . . . . . . . . . . . . . . . .

The remaining tree after carrying out the first step of calculating 2 + 3 ∗ 4. . . . . . . . . . . . . . . . . . . . . . . . . . .

Grammar for the language of ab. . . . . . . . . . . . . . . . .

Grammar for the language of ab and parentheses. . . . . . . .

Grammar for the language for multiplying numbers. . . . . . .

The partial tree after performing the first step. . . . . . . . . .

The complete tree for 2 * 3 after performing the second step.

7.3 7.4 7.5 7.6 7.7

9

. 52 . 54 . 54 . 56 . 58 . 62

94 94 97 100 100 101 101

Two possible trees for 2 + 3 ∗ 4. . . . . . . . . . . . . . . . .

Correct grammar for the language for adding and multiplying numbers: multiplication has priority over addition. . . . . .

7.10 Parse tree for 2 + 3 * 4. . . . . . . . . . . . . . . . . . . .

7.11 Incorrect grammar for the language for adding and multiplying numbers: addition has priority over multiplication. . . . . .

7.12 Two possible trees for 1 − 2 + 3. . . . . . . . . . . . . . . . .

7.8 7.9

. 102 . 103 . 103 . 104 . 105

8.1

Grammar defining our predicate logic used in Knowledge Game.108

9.1 9.2

A world. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 113 The expression tree for (Ax)Red(x) | (Ex)Small(x). . . . . 114

19.1 An example of the patience feature. . . . . . . . . . . . . . . . 153

10

Responsibility Shared We share the responsibility for the following chapters: • Chapter 2: Laputa, pages 22 – 30, totalling 9 pages.

• Chapter 3: What to expect from a simulator, pages 31 – 44, totalling 14 pages.

• Chapter 5: Introduction, pages 47 – 64, totalling 18 pages.

• Chapter 6: Agents, pages 65 – 89, totalling 25 pages.

• Chapter 21: Peer disagreement, pages 162 – 171, totalling 10 pages.

• Chapter 12: How the outspoken beat the careful, pages 125 – 127, totalling 3 pages.

Shared responsibility in total: 79 pages.

Alette Holmberg-Nielsen • Chapter 1: Introduction, pages 15 – 21, totalling 7 pages.

• Chapter 8: Our monadic predicate logic, pages 107 – 110, totalling 4 pages.

• Chapter 9: Model checker, pages 111 – 118, totalling 8 pages.

• Chapter 10: Experiments in Knowledge Game, pages 120 – 121, totalling 2 pages.

11

• Chapter 13: How the stupid and outspoken beat the smart and careful, pages 128 – 131, totalling 4 pages.

• Chapter 15: Saints and sinners, pages 135 – 138, totalling 4 pages.

• Chapter 16: Conclusions, pages 139 – 141, totalling 3 pages.

• Chapter 17: Introduction, pages 143 – 144, totalling 2 pages.

• Chapter 19: Patience, pages 151 – 156, totalling 6 pages.

Alette’s responsibility in total: 40 printed pages.

Søren Lind Kristiansen • Abstract, pages 1 – 1, totalling 1 page.

• Chapter 4: What did we just learn?, pages 45 – 45, totalling 1 page.

• Chapter 7: Designing a logic, pages 90 – 106, totalling 17 pages.

• Chapter 11: How the smart beat the stupid, pages 122 – 124, totalling 3 pages.

• Chapter 14: How the stupid and outspoken beat the smart and careful – revisited, pages 132 – 134, totalling 3 pages.

• Chapter 18: Dynamic trust, pages 145 – 150, totalling 6 pages.

• Chapter 20: Diversifying agents’ logical abilities, pages 157 – 161, totalling 5 pages.

• Chapter 22: Independence, pages 172 – 173, totalling 2 pages.

• Chapter 23: Conclusion, pages 174 – 176, totalling 3 pages.

Søren’s responsibility in total: 41 printed pages.

12

Percentages The shared part amounts to 50 percent, Alette’s part amounts to 25 percent and Søren’s part amounts to 25 percent. The total size of the document excluding table of contents, index etc. is 160 pages, roughly equalling 120 normal pages.

13

Part I Motivation

14

Chapter 1 Introduction 1.1

Simulations in social epistemology

In traditional epistemology one is concerned with knowledge in the subject.

This is an individualistic approach to knowledge. Epistemology is about the conditions under which you can say that ‘S knows p’, ‘S believes p’ or ‘S is justified in believing p’. Also the subject can reason about himself, for example ‘S knows that S knows p’. But this traditional approach to epistemology does not concern itself with the social aspects of knowledge exchange and belief update. Social epistemology does exactly that. We get a lot of our beliefs from social interaction with other epistemic agents. This social exchange of knowledge is influenced by some social concepts such as trust in others, trust in oneself, group pressure, lying, etc. Examples of social practice involving knowledge exchange could be the processes in the scientific community, for example peer reviewing of papers or the work of a jury assigned to determine the guilt of a person charged with a crime. These are examples of social practice in society of interest in social epistemology.

But interesting examples of social practice are also found in smaller groups.

For example belief polarization where people who agree on a given subject talk to each other and get even stronger belief in their original belief. Another example is belief division where people who agree on a given subject talk to each other and end up disagreeing on the matter. These are all social phenomena happening in the real world.1 They are examples of real world social practice which all involve knowledge exchange, but they are not studied 1

[Sun09].

15

in traditional epistemology. They are, on the other hand, studied in social epistemology.

Social epistemology is ‘the study of the social dimensions of knowledge or information’2 . This is a very broad description and the terms ‘social’ and ‘knowledge’ are used in several different ways. This has led to a lot of different approaches to social epistemology.

The different approaches are in themselves not unusual for philosophy in general nor for epistemology in particular, but one approach is a bit unusual: the use of computer simulations.

A simulation is some process imitating a real object or process. An example of a simulation could be a flight simulator where different planes and whether conditions can be simulated to enable the pilot to test his skills under the given conditions.

Since social epistemology is – no matter how broadly we understand the term – concerned with some kind of agents exchanging some kind of information, it seems obvious that a computer could simulate the agents and the information exchange. Indeed, several computer simulators for use in social epistemology have already been written and used to study different aspects of information exchange.

1.2

Our aim

Our aim is to add to the existing work by creating a simulator with a very specific purpose: to study a variety of social practice when playing a game we call Knowledge Game. The game can be described briefly as a game of a number of agents who compete against each other to achieve enough information to answer a specific question about the world (or more specific: determine the truth value of a statement about the world). The game is turnbased and in each turn, the agent can choose if he wants to seek information by making his own investigations or by asking one of the other agents a question. Alternatively, if the agent thinks he already has enough information to answer the question, he may choose do so. The first agent who is able to correctly answer the question wins the game. If an agent answers falsely, then he is out of the game, but the remaining agents will continue.

The rules of the game are quite simple and one may be tempted to object 2

[Gol01].

16

that the game is so simple it has nothing to do with the real world. However, we do see this as a pro, not a con. The simplicity of the rules makes it easier to see what the simulator actually is, what it simulates and what it does not simulate and thereby making it more clear what conclusions one could draw from the results of a simulation. As we shall see, the question of what information you actually get from a simulator is a pressing one and one that must be addressed before writing the simulator. We will do this in the following chapters.

1.2.1

Faithfulness and adequacy

Two central concepts in this thesis are faithfulness and adequacy. Faithfulness is a measure of how well a model captures the target phenomenon.

Faithfulness is an absolute measure in the sense that it is not relative to the purpose of the model. We can imagine determining the faithfulness of a model by examining each and every aspect of the phenomenon and then deciding whether it is modeled correctly. This introduces the concept of correctness: Aspect A of a phenomenon P is modeled correctly in model M just in case A works exactly the same way in M and P .

Note that our notion of correctness does not in anyway mention the purpose of the model. Now we can easily talk of both partial and full faithfulness: Model M of phenomenon P is fully faithful (of P) just in case M models every aspect of P correctly.

Model M of phenomenon P is partially faithful (of P) just in case M models some aspects of P correctly.

Note that faithfulness does not mention the purpose of the model: a model is fully faithful just in case every aspect of the phenomenon is modeled correctly no matter if only a few of the aspects are relevant for the purpose of the model. This is where adequacy comes in: Model M of phenomenon P is adequate to degree X just in case M serves its purposes to degree X.

17

For example, imagine a model of a plane in a flight simulator. If the model of the plane contains every aspect of that plane from the color of the seats to its weight and engine, then the model is fully faithful. If the model of the plane contains the relevant aspects of the plane, not the color of the seats, but surely its weight and engine, then the model is only partially faithful but it may very well be adequate depending on the purpose: If the flight simulator is only meant to be a simulation from the pilot’s perspective it may be adequate without the color of the seats but if it will be used to simulate the experience of the flight for the passengers the lack of simulated colors may render the model inadequate. We will see later on, how this distinction is relevant for our work.

Our goal is to build an adequate model which contains all the relevant aspects of the domain we want to simulate. Since we want a simulator for use in social epistemology and the process we want to simulate is the above described game, we need to build an adequate model of agents, interaction between the agents and the game.

The purpose of such a model is to test hypotheses and give us predictive power. When we have an adequate model, we can simulate situations we have certain hypotheses about and see whether or not our hypotheses are confirmed. For example, one might have the hypothesis that very skilled agents always win the game and not so skilled agents do not. This can be tested with our simulator. In fact, we will test it in Chapter 11 and 13.

An adequate model can give us predictive power, so that we can predict the outcome of a complex situation. Something which we would not necessarily be able to do in the real world. Predictive power is useful in situations where it’s not feasible to run an experiment. For example a certain experiment may not be feasible because it would be too expensive or simply not practically possible because of the size of the group of people required. In those cases we could run the experiments in the simulator instead. Furthermore, if the simulator incorporates an adequate model it might be useful to run the experiment in the simulator even if we could have run a real-life experiment – instead of gathering a group of people for the experiment we could just use the simulator. In Part III we will have a look at some actual experiments involving different types of agents. Our simulator allows us to run such experiments in a very controlled environment where it is possible to isolate the influencing factors – something which would be hard to do in real life.

18

1.2.2

Two steps

Our work can be seen as a two-step process. First step is to create the building blocks of the system. The blocks of the model are agents and the interaction between the agents. These two blocks are as such independent of each other. If we find out that the agents need other properties, then it will not necessarily mean that the interaction between the agents needs to be changed. And likewise, if we find out that the interaction between the agents needs to be defined in another way, then it will not necessarily change the agents themselves.

The second step is finding a social process to simulate. The process we want to simulate is the game mentioned above, the Knowledge Game. This is because a game represents a simple social process which is easy to control both in real life and in the computer simulation.

Simulating a game is also an example of a method where we start with a simple case. The simple case will reveal a lot of assumptions and decisions to us. When we fully understand the simple case, we can move on to more complex cases of simulation. Of course, one needs to make sure that the simple case is not too simple. It needs to be complex enough that we can learn something from it and complex enough that it enables us to move forward to more complex cases. We need to navigate between simplicity in danger of not simulating anything real at all and complexity in danger of simulating too much at the same time and thereby lose control and overview of the influencing factors.

Both with regard to the first step (the building blocks) and the second step (the process) we can ask whether this step is fully adequate or partially adequate (or not adequate at all). If the first step is not fully adequate, it seems that the second step can never be fully adequate too. If the building blocks are not fully adequate, it seems implausible that the social process could be fully adequate. On the other hand, it seems perfectly plausible that we can have fully adequate building blocks, but not a fully adequate social process.

1.3

An overview

In this thesis we will lay the foundations for more adequate simulations in social epistemology. We will do that by obtaining a higher degree of adequacy 19

with respect to the first step: the building blocks of our model. As the second step, we will go on to study a fairly simple social process: the Knowledge Game. The study of a simple social process will help us getting the building blocks of the system right, something which is not obtained by some of the other similar systems, Laputa being the case in hand (see the discussion in Chapter 2). The study of a simple social process is also enabling us to test our building blocks in a simple, controlled environment, which is crucial in this setting, where the possible fallacies of making false conclusions based on the results are so many. When the building blocks are as close to fully adequate as possible, it will be possible to move on to more complex social processes.

In the chapters that follow, we will explore the theoretical foundations of simulations in social epistemology. We will try to answer the questions: What can we learn from the existing work in the field? and What can we learn from simulations in general?.

In Chapter 2, we will first have a look at an existing simulator for use in social epistemology, namely Laputa by Erik Olsson and Staffan Angere from Lund University. This should give us an overview of the existing work in the field and point us to where we can add something. In Chapter 3, we will address the theoretic question of what conclusions it is possible for one to draw from the results of a simulation in general. Throughout this chapter we will use an example of a simulator, a raytracer, which simulates light. This has not a lot to do with simulations in social epistemology, but it will allow us to make some general points about what to expect from simulators. This will also make it clear why we have tried to keep our simulator somewhat simple and make it a game with simple rules.

In Part II, we will describe and discuss our actual simulator. This part gets a bit technical to make sure that all relevant parts of the inner workings are well-understood, as this is necessary to avoid misunderstandings and false conclusions based on the simulator. This part also includes some wellknown philosophical discussions, for example Gettier examples and ‘Hesperus is Phosphorus’, and the possibilities of modeling these in the simulator.

Part III contains some of the actual experiments we have made with our simulator. These experiments involve comparisons of different types of agents, among these are smart agents, stupid agents, outspoken agents, careful agents, lying agents and truth telling agents. Our system allows us to compare these different types of agents in a very controlled environment 20

where we have full control over the influencing factors. Something which would be very hard to do in real life.

21

Chapter 2 Laputa 2.1

Simulations in social epistemology

As mentioned, we are not the first wanting to simulate knowledge in a social environment. Actually there are quite a number of different simulation systems for use in social epistemology. Rainer Hegselmann and Ulrich Krause have developed what is known as the Hegselmann-Krause model.

The Hegselmann-Krause model is used to study opinion dynamics within interacting groups. The model allows us to investigate opinion formation in different types of groups, defined among other things by spread in opinions and confidence.1 Another example of a simulation approach in social epistemology is the one of Stephan Hartmann, Carlo Martini and Jan Sprenger. Even though they do not mention anything about implementing their model in a computer in [HMS09], they use the Lehrer-Wagner model2 to investigate consensual decision-making among epistemic peers.

In this thesis, we are mainly concerned with one simulation software, Laputa. Laputa is developed by Erik Olsson and Staffan Angere from Lund University and it is described in [Ols11] and [Ang10].

The main reasons why we have chosen Laputa as a starting point for this work is because of the detailed information on the actual software and partially because of the accessibility of the source code of Laputa. In case a simulation in Laputa yields a result which we find odd, having the source 1 2

[HK02].

[LW81].

22

code ready will put is in a position to identify exactly why that result is generated. We have also had the chance to hear Olsson and Angere speak about Laputa on several occasions. Besides that, we disagree on a number of fundamental points regarding the model of Laputa. First, the model of trust in the system is in our opinion not realistic and second, the model of beliefs and interaction is too simplistic. Let us look at the challenges for Laputa in detail below.

2.2

Framework

The theoretical framework used in Laputa is based on the work on veritistic social epistemology done by Alvin Goldman. In veritistic social epistemology, the goal is to measure social practices by their ability to promote truth.

Goldman has an algorithm for measuring this ability, he calls it veritistic value. degrees of belief (DB) have veritistic value relative to a question Q, so that any DB in the true answer to Q has the same amount of V-value as the strength of the DB. In Goldman’s terminology, V-value of DBX(true) = X. Suppose, for example that Mary is interested in the question whether it will rain tomorrow. If the strength of Mary’s belief that it will rain tomorrow is .8, and it will in fact rain tomorrow, then the V-value of Mary’s state of belief vis-a-vis the rain issue is .8.3 The veritistic value of a network is then the average of the veritistic value of the beliefs of the agents in the network. Goldman acknowledges that the more complicated the networks are, the longer it will take to compute the veritistic value of the network. And the interesting social practice will often be the more complicated practice, which requires some computational power.

Erik Olsson’s and Staffan Angere’s contribution is to use computers to evaluate the veritistic value of social networks. Computers have the computational powers we do not and therefore Goldman’s thoughts on the veritistic value of social networks come to life in Laputa. In Laputa, the networks can be as complicated as you have patience to build them, which means that Laputa is 3

[Ols11], p. 3.

23

able to calculate the veritistic value of some more interesting and complicated networks than we can do by hand.

To understand what it is one can simulate in Laputa, let us have a closer look at what you can call the model of Laputa.

2.3

The model

Goldman calls his model a question-answering model. And that holds for Laputa too.

In a question-answering model, agent S’s belief states (of either the trichotomous or the DB kind) have value or disvalue when they are responses to a question that interests S (or, more generally, when other agents are interested in S’s knowing the answer).4 Laputa lets us specify a network of agents. All these agents are concerned about is one question: whether p is true. The network of agents can be represented by a directed graph where the edges are the communication lines and the nodes are agents, as shown in Figure 2.1.

Laputa is also a Bayesian model.5 This means that updates of the agents’ beliefs happen via a calculation of probabilities. Every agent has the following probabilities: • Initial degree of belief - The degree of belief in p or ¬p before any communication has taken place. For example, agent A beliefs p with an initial degree of belief of 0.5.

• Inquiry accuracy - This is the probability that the agent is correct in his inquiries. For example, agent A has an inquiry accuracy of 0.8.

• Inquiry chance - This is the probability that the agent does inquiries.

For example, agent A has a 0.5 chance of doing inquiries.

• Inquiry trust - This is the agents trust in his own inquiries. For example, agent A has a 0.9 trust in his own inquiries.

4

[Gol99], p. 88.

We will just describe the parameters of Laputa in the same way Erik Olsson does it in [Ols11] and not go into computational and implementational details.

5

24

Figure 2.1: An example of a network in Laputa. Note that the edges are directed. That is, Sherlock Holmes can talk to Mrs. Hudson, but Mrs. Hudson cannot talk to Sherlock Holmes.

Every communication line between the agents has the following probabilities: • Listen chance - This is the probability that the receiver will listen to the message from the sender. For example, there is a 0.8 chance that agent A will listen to agent B. • Listen trust - This is the receivers trust in the sender. For example, agent A trusts agent B to a degree of 0.4.

• Threshold of assertion - This is the degree of belief in p necessary for the sender to say p to his receivers. For example, agent A has a threshold of assertion at 0.4. Then agent A’s degree of belief in p needs to be above 0.4 for A to communicate p to other agents.

This is the model of Laputa in short. More details on the calculations done by Laputa can be found in [Ang10].

25

There are always some questions you can ask about such a model. Is it realistic? Is it meant to be realistic? What role do these parameters described above play? In the following we will have a look at a few challenges for Laputa.

2.4

Challenges

There are three main challenges in Laputa. The first one is about the model of trust. The second one is about the very limited model of beliefs. Both challenges raise doubts about the adequacy of the system. One can argue that the assumptions about trust and the simplicity of beliefs are too far away from what we know about human behavior and reasoning. The third challenge is about the way Laputa finds the optimal veritistic networks. Let us look at them one by one.

2.4.1

Liars

An interesting thing in Laputa is the role trust plays. Inquiry trust and Listen trust are the two parameters determining whether an agent will update his belief.

The interesting thing, and perhaps a bit counter-intuitive, is that if one’s trust in another agent is so low that you consider the agent to be a liar, then you will, without exception, negate all communication coming from that agent. This is a somewhat dramatic way to deal with persons you have no trust in: Let us say, I have no trust at all in my employer who says that I have a meeting tomorrow morning. If I am an agent in Laputa, I will now know that I do not have a meeting tomorrow morning. In real life I would probably just ignore my employer’s statement and check my schedule again to find out whether I have a meeting or not.

In Laputa, this assumption about low or no trust is used to show examples of belief division. Belief division is the situation where a group of agents first agrees upon a subject, then discusses it and ends up disagreeing on the subject. Erik Olsson presented the following example of belief division in Laputa at 2nd Copenhagen Conference in Epistemology in August 2010.

26

2.4.1.1

Example

The example is constructed around two agents: • Both agents claim to know p since their degree of belief is above their threshold of assertion.

• None of the agents trust each other and therefore they will both take each others statements as evidence for ¬p and therefore downgrade their degree of belief in p.

• One of the agents, let us call him A, still has a degree of belief in p higher than his threshold of assertion, so he still claims that p is true even though his degree of belief in p is downgraded.

• The other agent, B, listens to A claiming p and thus gets even more confident that ¬p is the case. (This is because of the implementation of low trust in Laputa) • A listens to B claiming ¬p and thus gets even more confident that p is the case.

• The longer they carry on communicating with each other, the more confident they become in respectively p and ¬p.

• In Laputa, trust is also dynamic, so while they disagree, they also downgrade their trust in each other.

This is Erik Olsson’s example of belief division in Laputa.6 It is obvious that the example relies heavy on the assumption about negating statements from non-trusted sources. But it is not obvious that this is the best or most realistic way to implement trust: It seems that Laputa’s implementation of trust is an adequate model of how to treat non-trusted sources only when the agent has positive reason to believe that the other agent always get things wrong about the relevant kind of issue. However, it is quite obvious that in the vast majority of cases in which we do not trust someone we do not have that kind of reason. As mentioned above, a better and more adequate strategy would be to just ignore statements from non-trusted sources. Further more this would protect the agents in situations where the non-trusted 6

[Ols10].

27

source is telling the truth and the cases in which he lies but is mistaken. For example, imagine that I completely distrust my employer. He believes there is no meeting tomorrow, but lies about it to me and tells me there is in fact a meeting. But his belief is wrong and there actually is a meeting tomorrow.

In Laputa, since I do not trust my employer, I would negate his statement and thus miss the meeting. If I just ignored my employers statement that there is a meeting tomorrow and checked my schedule myself, my chances of knowing the truth would be improved. In that situation, the assumption in Laputa about liars is not truth promoting.

The agents’ attitude towards non-trusted sources in Laputa is an example of an important assumption one can question. It plays a crucial role in the example of belief division and in determining the results generated in Laputa.

It is crucial that in this thesis we will not model trust the way Laputa does.

This is one of the points where our system is fundamentally different to Laputa. It will be clear in the following parts how we will model trust in our system.

2.4.2

Divine Inspiration vs. reasoning

Another challenge in Laputa is the assumption about only a single proposition. As seen above, the agents are only concerned with one single proposition, namely the truth value of p. The only question, the agents are asked, is if p or ¬p is the case7 . The idea of letting the agents answer a closed question is Goldman’s.8 Laputa implements this idea. The challenge is that the agents in Laputa does not only answer a simple question: p or ¬p. They have no beliefs about other propositions, neither atom propositions like q nor complex propositions like p → q. The doxastic comprehension of the agents is very limited.

It is hard to imagine a realistic situation where nothing else than just p is in the head of the agents. Even if the goal is not to model a complete human mind, then this is a serious simplification. If an agent has a belief about q, it seems intuitively fitting that he must be capable of giving some kind of reasons for q. If q is the proposition: the street is wet, then the reasons for 7

Note that when we mention asking whether p or ¬p is the case we mean the question of which one of p and ¬p is the case – not the tautological interpretation according to which ’p or ¬p’ is always true.

8 [Gol99], p. 87-100.

28

q could be p: it rains and p → q: if it rains, then the street is wet. This kind of reasoning cannot be represented in Laputa. In general, it is hard to see how the agents can justify their beliefs in Laputa. Justification seems to be an important epistemic practice, when we want to simulate knowledge in a social context and therefore one might wish for a way of representing justification.

This is related to another challenge for Laputa. Erik Olsson and Staffan Angere calls it inquiries, but basically what it is, is some kind of divine inspiration. The agents cannot perform any real inquiries: The agents do not do anything besides saying what they believe and update their beliefs – and specifically they cannot reason. In Laputa there is no world to make inquiries about. What is called inquiries in the above parameters is just the chance that the agent gets a divine inspiration and suddenly believes something about p. Since the agents cannot perform investigations or inquiries, but only get this divine inspiration, this cannot be given as justification by the agents.

One way to go about this is to introduce the possibility of agents entertaining more than one proposition. Combined with complex propositions in a fitting logic and a system of inference, it will be possible for the agents to have reasons for their propositions. We will develop this approach in details below.

2.4.3

Brute force

In Chapter 3 we will see how important it is to know what to model when building a simulator. In Laputa, the model consist of a set of assumptions about beliefs and trust, as described above. But how can we know what exact situation to simulate? What network of agents and what distribution of trust and inquiry chance should we build? The answer is: we cannot know.

In Laputa, the solution is to use brute force. Laputa simply tries every possible network and distribution of trust and knowledge to find the optimal structure – that is, the one with the highest veritistic value – within the limits we as users of the simulator specify. For example, we can compare networks with a lot of communication lines to network with few communication lines.

The challenge is that this is not enough. Laputa can only use brute force on a limited set of possibilities, namely the assumptions about trust and knowledge described above. An example of something Laputa cannot use 29

brute force on is different strategies of belief revision. Laputa uses a Bayesian framework and therefore Laputa cannot compare the veritistic value of other strategies. This applies to other strategies for communication, the order in which the agents communicate and types of agents as well. None of these other possibilities can be brute forced by Laputa.

Therefore brute force is a solution to the problem that we do not know which network to explore, but it is important to be aware of the fact that even brute force is limited by the assumptions made in the framework.

2.5

Conclusions

As pointed out above, there are some challenges for Laputa. First, the modeling of trust in Laputa is very different from what one would believe happens between real agents and it implies some very strong reactions in situations with low trust. Second, the doxastic comprehension of the agents is very limited. The agents can only contain one belief, namely the belief about whether p is true. They get new information about this single belief through some kind of divine inspiration. And third, these and more assumptions in Laputa weaken the power of brute force and thereby the possible results of the simulations made in Laputa.

This means that Laputa faces some severe challenges regarding the model of the agents and the interaction between agents. Indeed, this gives one reasons to be skeptical about the fruitfulness of simulations in social epistemology, at least in its current form.

In this thesis, we would like to experiment with a simulator in which we are able to run more complicated experiments with respect to the agents’ reasoning and interaction, and as we shall see in Part II, we also have quite another view on the way to evaluate the epistemic value of an agent. Hopefully, it will then be clear what we can learn from simulations in social epistemology.

30

Chapter 3 What to expect from a simulator 3.1

What is a simulation?

A simulation is a process or action, a machine or computer software, imitating a real object or process. The reason for having a simulation is often that the real object or process is in some way or another inaccessible. An example of a simulation is a wind tunnel used by engineers to investigate how a bridge reacts to different weather conditions. In the wind tunnel, a model of the real bridge is used under the assumption that the behavior of the model gives us information about the behavior of the real bridge. This helps the engineers know if the real bridge will stand or fall before it is actually built.

Other examples of simulations are flight simulators used to train air pilots.

The flight simulator can help answer questions like ‘Is the pilot good enough to control the airplane x under conditions y?’. In this case, there is no physical model of the plane and flight, only a mathematical one. By carrying out some calculations using the mathematical model, one will be able to acquire knowledge of the real object or process. In theory, one could carry out all the calculations by hand, but often the sheer amount of calculations makes this practically impossible. This is where computer simulations are useful.

Another type of computer simulation is a raytracer . A raytracer is a simulation of light. A raytracer can be used to build computer images like the ones from the Pixar movies such as Toy Story. Usually a raytracer is not

31

used to answer questions like the previous examples of simulations. Instead the actual rendered image is the goal of the simulation.

The rapid increase in speed of computers (see for example [Moo65]) has given rise to great use of computer simulators. Conway mentions the importance of computers already in a paper from 1963: The use of a digital computer to perform simulated experimentation on a numerical model of a complex system is an increasingly important technique in many disciplines today.1 Unfortunately, despite the extensive use of simulators, the theoretical foundation for what information we actually get from a simulator has not been able to keep up with the increase in use of simulators ([KH05]): In fact, there is a striking contrast between the technological sophistication that drives and is driven by computer simulation and the rudimentary state of theory underpinning the simulation endeavor.

We agree that there is a difference between theory and practice, thus, before we get to the task of actually creating a simulator for use in social epistemology, we will address the issue above and specifically the question: ‘What questions does a simulator help us answer?’. Throughout this chapter we will use an example of a simulator, namely the above mentioned raytracer.

This will help us get some insights into simulations in general. Before we get started, let us briefly look at how a raytracer works.

3.1.1

Our raytracer

As mentioned above, a raytracer is a simulator of light which can be used to create graphics like the ones known from the Pixar movies Toy Story and Up. For this example, we have written a basic raytracer in C# and we will use it to tease out important points. We will not look at the actual source code, since it will not be relevant for our points, but in case someone should be interested we have made it available on our website: http:// socialepistemology.org/. Instead, we will look more broadly at how the 1

[Con63] quoted from [KH05].

32

raytracer works and we will use it to make clear the importance of the points made above regarding new knowledge.

A raytracer works, as the name suggests, by tracing rays. The rays are both rays of light and ‘rays of sight’. When a raytracer is used to draw a picture, the first thing we need to do is to create a scene. A scene consists of one or more objects, one or more light sources, an eye and a canvas upon which the image should be drawn. The canvas should be placed somewhere between the objects and the eye as if the canvas is a window. For our raytracer we made a few assumptions to keep things simple: • There is only one light source • The objects are of a plain color • The objects are opaque • The objects are matte The canvas is made of a grid of points. In the end, these will be the pixels of the final image. We need to calculate the color of each of the points, thereby building the image. For each point in the canvas we create a line – the ray of sight – from the eye through the point and into the part of the scene that contains the objects we want to draw. We then check if the ray of sight intersects with any of the objects. If it does not, it means that the eye does not see anything in this specific point, which in turn means we can paint the point black – or another background color if we choose. If the line does intersect with an object, it means that the intersection point of the object will be the part of the object the eye sees in the canvas point – if there is light on that part of the object, that is. In case the ray of sight hits an object, we must therefore calculate whether the intersection point on the object is also hit by a light ray. If it is not, there is no light on the object, so it seems natural to paint it black. On the other hand, if a light ray hits the object, the object is visible and we can give the point the color of the object, i.e. if the object is red, we will paint the point red.

This is, in short, how our simulator of light, our raytracer, works. In the following we will see what we can learn from such a simulator.

33

3.2

What to expect from a computer

In this chapter, we will look at what we should expect, and perhaps more importantly should not expect, from a computer simulation. To accomplish this task, it will be helpful to start by looking at what we should expect from a computer in general. This is what this section is about.

One tempting way to see the computer is as a black box that produces interesting and some times unpredictable outputs. A box that, given some input, ‘does its thing’ and in mysterious ways produces a result like ‘the bridge will collapse’. This view may be sufficient once a simulator is up and running, in the same way that you do not have to know how the engine of your car works to drive it. To actually build a car, a lot more knowledge is required.

In the same way, viewing the computer as a black box is not sufficiently detailed when analyzing the theoretical foundations of computer simulations.

The black box view will, as we shall see, lead to some unfortunate misperceivings as it can give the intuition that the computer in some sense has ‘its own way of doing things’. It has not. The computer produces output based on the user input following certain rules. Nothing more, nothing less.

3.2.1

Input determines output

One area of philosophy where the benefit of computer simulations seems obvious is social epistemology. And indeed, several systems for computer simulations in social epistemology have already been written, as we saw in Chapter 2. However, the deterministic nature of the computer sets certain limits to what we can expect from simulations. Looking at the systems mentioned, it may seem tempting to build a computer simulation with a lot of bells and whistles and a huge number of agents and use it to find laws and principles for how knowledge spreads from individual to individual and from group to group. This is a very honorable goal, but as we shall see it is also quite unrealistic. As mentioned above, everything that a computer can calculate, can – in theory – be calculated by hand. However, the computer has an enormous advantage in the fact that it can make billions of calculations a second. In just a few seconds, a computer can make more calculations than a human can do in a lifetime2 . The sheer amount of computing power makes 2

Modern computers are often working at more than 2 GHz. 1 GHz is 1 billion calculations per second, which means a 2 GHz computer can make 2 billion calculations pr

34

the computer a very interesting and potentially helpful piece of machinery.

However, it is only a help if we know what to calculate. The computer will perform all the calculations on the model and there will be so many calculations that we could never do them all by hand, but the computer needs the model before it can make any calculations. In other words, if we do not have a model we will have no simulator.

3.2.2

Our raytracer

For example, when we build our ray tracer, we must know something about physics. First of all, we must know something about the behavior of light.

In this example, the raytracer described in section 3.1.1 is our model.

Using the raytracer, we can create images like the one shown in Figure 3.1. It is supposed to show three spheres – a blue, a red and a green one.

Unfortunately, it looks more like two deformed flat circles and a green banana.

This was obviously not what we wanted.

Figure 3.1: The results of a raytracer implemented using an inadequate model of light.

The first thing to notice is that even if this result may be surprising to us, it does not show that the world is flat. Instead, it shows us that something is wrong with our model: it gives us results that in a relevant way – the objects look flat – do not resemble the real world, so it obviously is not adequate. second. A rough estimate of the capabilities of a human would be maximally 5 calculations per second. This means a human would need 2 ∗ 109 /5 = 0.4 ∗ 109 seconds which is roughly 12 years to make the calculations that a computer could do in just one second.

35

Another interesting thing is to think of what would have happened if we could not compare the results to the real world. In the case of the three spheres, we are so well accustomed to the appearance of spheres that we can immediately see that something is wrong. But had this been a simulation of rational agents participating in discourse, we would probably not have been able to spot an inadequate model right away. This leads us to the next section, in which we will have a look at what we can learn from this example.

3.3

Unpredictability

As we have just seen, when it comes to computers, the input determines the output. This means that in principle nothing unpredictable will come from the computer. There is nothing we could not come to know given sufficient time. But since the computers have so much computing power, they will be able to give us results we cannot predict, because we do not have enough time to do the calculations by hand. That is, even though the results are not unpredictable, they are still not epistemically clear to us. This is one of the great strengths of computer simulations.

One objection against the statement that nothing unpredictable will come from a computer, could be this: computers can produce random numbers.

Since they are by nature random, they cannot be predicted. Hence, the entire output of a computer cannot be predicted by the input. This, however, is based on a misunderstanding regarding the randomness of random numbers in a computer. Even the apparently random numbers generated by a computer are based on a predetermined sequence of apparently random numbers.

We will not go into details with random number generators, but we would like to point to a couple of sources for more information. Two examples are [Pas10] and [PM88] which describes an algorithm for generating ‘random’ numbers. Having dispelled the myth of genuine randomness in computers, let us explore in what ways a computer can give us epistemically surprising results.

Imagine a weather simulation. A weather simulation takes extremely many calculations and would never be possible without machinery to do the calculations. For a weather simulation, it seems likely that in most cases it would be acceptable if it took an hour or two before the computer has the results ready. However, there are other kinds of simulators where an instant result is necessary, because the user needs to interact with the simulator. An 36

example of this is a flight simulator. If the pilot had to wait an hour – or just a couple of seconds – from he touched the joystick or pressed a button until the computer reacted, the simulator would be quite useless.

When the user interacts with the simulator like he does with a flight simulator, he does add a certain kind of unpredictability to the system. The computer’s reaction is not unpredictable, but depending on human nature, the input it gets from the user may be unpredictable. This means that in some sense the result of the flight simulator can be unpredictable, but if it is, the unpredictability comes from the user not the computer. This means, we can distinguish between results that are unpredictable in principle from those that are unpredictable in practice. As we have seen, the in principle unpredictable results do not exist when it comes to computers but those unpredictable in practice are very much real.

3.3.1

Surprise

The result of a calculation can be surprising, no matter if it is in practice unpredictable or not. As a simple example, imagine you are out having a few drinks and the bartender has opened a tab for you. When you are about to leave and he gives you the total amount, it is a lot higher than you had expected and you are absolutely sure he is wrong. However, when he shows you the list of all the drinks you bought, it is pretty clear that he is right, no matter how surprised you are that you have already spent all you had.

The results of the calculations were in no way unpredictable, nevertheless you were surprised because you expected something else. Comparing this to a computer simulation, we see that it is also possible to be surprised by the results of a computer simulation, maybe even more so as the results are probably in practice unpredictable.

In some cases we might have done some of the calculations of simulation by hand, for example to make sure that the simulation is written correctly.

Comparing the manual calculations to the ones made by the simulator will, of course, some times show a difference. In general, you could say that the difference will stem from either a mistake in the manual calculations or an error in the simulation. This will be a real and important difference to the programmer when he is debugging his software, but for now it is not relevant to us as it does not get us any closer to understanding what kind of information we can get from computer simulations. Another type 37

of difference is of much more interest to us, namely the difference between the results of a simulation and the real world. This is the topic of the next section.

3.3.2

Simulation results vs the world

In many cases the results of a simulation will in some way or another end up being compared to the part of the real world that was simulated. Like in Figure 3.1, where the image produced by the raytracer is very different from what we would see in the real world. How could this be? There are several possibilities, so let us look at them one by one.

3.3.2.1

Simple simulator error

The difference between simulator results and the real world may be caused by a simple error in the simulator. By a simple error, we mean something that is not part of the model, but more something like a typing error. If this kind of error occurs, it does not give us any information about the real world, it only shows us that the programmer did a sloppy job, or perhaps more realistically that all software contains bugs.

3.3.2.2

Missing known laws or forces

If there is an error in the simulator, but it is not what we called a simple error above, it can be an error in the model used in the simulator. There are different kinds of errors that can exist in the model. One of these is missing the implementation of an already known law or force. For example, if we implemented a flight simulator and did not implement gravity correctly – or maybe forgot to implement it entirely – we would obviously get some very strange results. However, the error still does not tell us anything new about the real world. We already know that gravity exists, so all we are reminded is to make sure it is implemented correctly in our software. This might actually be what happened in our raytracer. Let us have a look at it again.

3.3.2.2.1 Our raytracer As we have already seen, a simulation requires an adequate model. The model used to paint the picture in Figure 3.1 was not adequate, so we have to do better. To do that we need more knowledge

38

of how light works. Again, note that to simulate light we have to know how it works.

To enhance the model, we must first realize what is wrong. It is obvious that the objects look flat. This is because their entire surface has the same color. To get the sense of depth, the object needs to be a bit darker in some areas. But the crucial point is: which areas? Well, basically we can only – and the eye in the model should only – see the light that actually meets the eye, that is the light which is reflected from the objects. A book on physics can tell us that matte objects reflect light in all directions, but the amount of light reflected depends on the angle of the incident light ray. If the angle is equal the normal – 90 degrees to the object surface – then all of the light is reflected. If the angle to the object surface is 0 degrees, then none of the light is reflected. Anything in between can be calculated by dividing the angle by 90. For example, an angle of 45 degrees reflects 45 / 90 = 0.5, that is 50% of the light. Adding this to the model will make our raytracer generate much better looking images. The image we saw in Figure 3.1 can be seen rendered using the new model in Figure 3.2.

Figure 3.2: The results of a raytracer implemented using a better model of light.

Figure 3.2 looks much better than the original image. The new model is indeed better than the first attempt. However, it does look a little wrong that the shadows are completely black. The reason they are black is that there is no ambient light at all. If we add a bit of ambient light to the model we can get a better result. This shows another point: the process of building 39

simulations iterative is by nature – by enhancing the model we did get closer to the real world, but it reveals some shortcomings of the new model. We can see this process as a continuous attempt to falsify the model. Another account of simulations describing them this way is found in [Her91]: If the output data of the simulation model are at variance with the output data from the real world system, the model is falsified and another tentative simulation model must be proposed.3 We can make a third attempt by adding ambient light. This gives us a model that is satisfying for the purpose of this chapter. The resulting image is seen in Figure 3.3. This image looks even better than Figure 3.2 and might be adequate depending on the intended use. Still, it can be improved on in many ways, for example, the model contains no concept of field of depth, which means all objects appear equally sharp. This gives the image a little unnatural look and in many cases one would need to add the mechanics for rendering a field of depth.

Figure 3.3: The results of the raytracer using ambient light.

Depending on what purpose our raytracer is meant to serve, a more or less faithful model may suffice for a high degree of adequacy. If we do not need any field of depth, then the model may be fully adequate. This shows how a model can be highly or fully adequate even if it is only partially faithful.

Return to Laputa. As we have argued, it is clear that the model it implements 3

[Her91] p. 58.

40

is far from fully faithful. Nonetheless, full adequacy does not presuppose full faithfulness – might Laputa still be adequate for the purpose? No. As we have likewise argued, the faithfulness of the model lacks in so many aspects that it cannot serve its intended purpose well – that is the purpose of providing data that can be used in the study of socio-epistemic phenomena, as they occur in the real world.

3.3.2.3

Missing unknown laws or forces

If the error is caused by a non-simple error in the model, but not by not implementing a known law or force correctly, it may be caused by something much more interesting, namely that a law or force not yet known to us has not been implemented. This is potentially a much more interesting result to get from the simulator as it can sometimes tell os something about the real world. This situation will not give us direct knowledge of the world in the sense that it will not tell us anything about what laws or forces are missing from our model, but it will give us indirect knowledge in the sense that it tells us that some part of our model is missing or incorrect – it just does not tell us which part.

Let us say, we found out that, even with all the relevant laws implemented in our raytracer, the image still does not look like the real world. In this case, it might be caused by some unknown law we have not yet encountered. From the image of the spheres, we would not be able to know what this unknown law is, but it would be valuable information to know that there is something yet to explore when it comes to laws of light.

3.3.3

No difference

In the sections above, we discussed cases where the results from the simulator differ from either our expectations, our manual calculations, or the real world.

Of course, there are also cases where it does not differ. These, specifically the ones where the results do not differ from the real world, are the cases where we have a simulator that we can use for experiments. Like investigating whether the bridge will collapse, or whether the pilot can fly the plane. This is exactly what we talked about in section 1.2.1, where we talked about the importance of an adequate model. Of course, in some cases the results from the simulator may match the real world by chance. For example, imagine a flight simulator that is made way too easy to fly, so it is impossible for the 41

pilot to crash the plane. Let us say, the pilot flying the simulator is actually a very well-trained and experienced pilot. By looking at the simulator results, you would come to think that he is indeed an experienced pilot – but for all the wrong reasons: the simulator would have led you to think anyone without any experience at all would be an experienced pilot. This is what we could call the Gettier case for simulators. However, since the nature of the simulator is that you can actually experiment with it more or less for free, a simulator, with an error like the one just described, should be possible to spot. We will discuss another kind of Gettier case later.

3.4

What can we learn?

Basically, we can say that there are two cases in which it is possible to gain some kind of new knowledge of the world using a simulator. In the first case, the new knowledge is coming from in practice unpredictability and presupposes an adequate model of what we are simulating. Having the adequate model, we can use the simulator to do experiments that would just not be feasible doing with the real world objects or processes. The second case of new knowledge is based on not having an adequate model.

The experiments give us a hint that something is wrong with the model and thereby provide us with indirect knowledge of the processes or objects we are simulating.

3.4.1

Supervenience

The first case of acquiring new knowledge can be described as gaining knowledge through supervenience. Roughly speaking, facts of one kind can supervene on facts of another kind. B-facts supervene on A-facts if and only if there can be no differences in B-facts without a difference in A-facts. Applied to our case it means that facts about shadows supervene on facts about lights as there can be no difference with respect to shadows without a difference with respect to light. This case requires an adequate model of the part of the world. Since the model must already be adequate, this case presupposes that we have knowledge of how the relevant part of the world works. In other words, we cannot expect our experiments to give us knowledge about the objects or processes we work with directly. However, some mechanisms or laws that we do not yet know, may supervene on the laws and forces built 42

into the model.

A simple example of this is our raytracer. The raytracer simulates light.

This means that to build a raytracer, you must have a model of light. But you do not need to implement a model of shadows as they ‘come for free’ where no light is cast. This means that by experimenting with a raytracer – essentially a model of light – you can learn something about shadows.

For some simple investigations, the current model described in section 3.3.2.2.1 might be enough. For example, if we wanted to study the movements of shadows cast by spheres on other spheres, we could very well do it using this model. This takes us to one of the last points regarding what we can learn from the raytracer, namely that it shows us how we can learn about something that we did not specifically implement, but instead supervenes on what we implemented. The shadows are a perfect example of this, as our model contains no mention of shadows, only light, so in a sense the shadows do supervene on the light.

3.4.2

Comparing to the real world

The second case of acquiring new knowledge concerns a comparison to the real world. In this case, we need some model of the world, but it does not need to be adequate. We can then compare our results from the simulation to the real world to find flaws in the existing model. This, of course, is only feasible in some cases – it does not make much sense to build a real bridge just to check if your simulator contains an adequate model.

3.5

Making it explicit

Before we leave the theoretical analysis of simulations and the raytracer example, we will have a final look at the assumptions we made in the beginning of the chapter. In section 3.1.1, we made sure to make it quite explicit that we only had one light source, that the objects were matte, and so on. This is very important for users of the simulator or readers of scientific papers based on results from the simulator. If we published a paper on how to simulate light and did not spell out the assumptions, the readers of the paper would be perfectly right to assume that our model is all they need to simulate metallic as well as matte objects. But, since light on metallic surfaces requires a more

43

elaborate model, that assumption would be wrong. That is why one should be very clear about the explicit assumptions when writing a simulator.

However, we actually did make more assumptions than the ones we listed above. These other assumptions were never mentioned and one might never think of them. They are what we call implicit assumptions, and you want to do your best to get rid of them and make them explicit, so the user knows what he is working with. An example of an implicit assumption we made is regarding the distance of the objects. We assumed implicitly that distance does not matter in the sense that the light is equally strong no matter the distance from object to light source. That is a very strong assumption. And it is plain wrong – the intensity of light decreases as the inverse square of the distance. Assuming we want to make scenes where the distance between the light source and the objects matters, we still have a major flaw in our model.

But since it does not show on the images we rendered and we had never even thought of distance when writing the simulator, we might never know. The simulator might work perfectly until someday someone tried to make scenes with greater distance between the objects and the light source. If those scenes were not easily compared to the real world, we might still not know there was an error, but instead use the flawed results as if they were correct. The message here is to be very careful with all kinds of assumptions when writing the model of a simulator, as especially the implicit ones can be dangerous.

In conclusion, any assumption, implicit or explicit, is a potential source of loss of faithfulness and hence decreased adequacy, but when the assumptions are explicit our chances are higher that we will catch any inadequacies and that is why we want to make every assumption explicit.

44

Chapter 4 What did we just learn?

In the above chapters we have explored some of the theoretical foundations and limitations of simulations. The main question is: what is it possible to learn from simulations? This is very important to ask, since it is easy to be seduced by the interesting possibility of simulating human behavior with respect to knowledge in social contexts.

In general, we can learn something about human behavior in the real world, if we have an adequate model of the real world. In more specific cases, we can learn something about certain scenarios studied in the field of social epistemology. This requires an adequate model of the relevant part of the world too, and as we have seen, Laputa does not provide us with the necessary adequate model.

We have now explored some of the theoretical background for the simulations in social epistemology. In the next part of this thesis, when we get to designing our own simulation engine, we will have the theoretical foundations in mind. First of all, we should be very clear on the fact that without an adequate model we cannot expect to simulate anything. Second, we should be extremely aware of any assumptions we make and do our very best to get them out in the open, make them explicit. Let us get started!

45

Part II Designing the Simulator

46

Chapter 5 Introduction 5.1

An adequate and relevant model

We want to build a computer simulator for use in social epistemology. The simulator should include some kind of epistemic agents and should enable us to make experiments and test different kinds of epistemic strategies or practices for the agents.

In Part I, we learned the importance of an adequate model when building any kind of simulator. In short, before we start our simulations, we will need to know a lot about the processes we are simulating or at the very least make some severely restricting assumptions.

If the processes we want to simulate are by nature complex, so will a model that fully captures the processes be. This is no less true for simulations in social epistemology, than it is for simulations in general. A very complex model is obviously harder to build than a simple one. This is a good reason not to over-complicate the model. On the other hand, if the model gets too simplistic, it may no longer be able to capture the mechanisms of the simulated processes and hence not be adequate.

One way out of this might be to heavily restrict the processes we try to simulate. For example, we might choose to simulate conversations between only two agents instead of any number of agents. Even this can be extremely complex though. As we saw in Chapter 2, the implementation of the concept of trust alone has great implications for the results of a simulation of epistemic agents and to build an adequate model, we might need to implement even murkier things such as the mood of the agents. Leaving out an implementation of mood may cause the simulator to yield completely wrong 47

results, as the simulated agents will be way more rational than real life agents are.

The issue of defining an adequate and relevant model poses a real problem for any simulation in social epistemology, since very little is actually known about how trust and other concepts work in human beings. This means getting off ground is extremely hard, as you do not get anywhere without a model.

Yet we will give it a try. As described in Part I, our model will probably not be fully adequate, but we will present a model at least more adequate than the existing software. In this part, we will describe how to develop such a model and we will describe in detail every assumption we make, so that it is clear under which conditions, we can draw our conclusions.

5.1.1

Actual or optimal?

There are generally two kinds of questions one could ask when experimenting with a simulator like the one we are going to design: 1. What is the actual way the real life agents will act?

2. What is the optimal way the real life agents should act?

Say we are interested in simulating how knowledge of who to buy drugs from, flows between prison inmates. There is a big difference between the two following questions: • What is the actual way the prisoners will act to find out where to buy drugs?

• What is the optimal way for a prisoner to find out where to buy drugs?

If we had a simulator that could tell us what would actually happen given a fixed set of prison size, prisoner strategies, ways of communication and other relevant parameters, we could start trying out different parameters and given enough time we could try all parameter combinations and find the best one (or worst one, depending on whether you want the drugs to flow or not). This means that with a simulator that can tell us what will actually happen, we can – given enough time and computing power – find the optimal way (or at least one of several optimal ones). However, the reverse is not true: 48

we may have a simulator that can simulate and show optimal behavior, but still not show us what will actually happen. This is because we can have an oversimplified simulator that does not take forgetfulness, irrational behavior, less than perfect logicians and so on into account. In other words, a simulator that simulates optimal, but completely unrealistic agents. In some cases, such a simulator may be both adequate for the purpose and relevant, but it will not be able to predict what will happen in real life since the assumptions are unrealistic. However, if you are concerned with optimal strategies for perfect logical agents, it may very well be what you need. What kind of question you want your simulator to answer is something you should consider before starting designing it.

5.2

Building a game

One way of getting around the complicated and unknown workings of the simulated processes when building any kind of simulator is, as we have already seen, to be very specific about the assumptions. It may be sufficient for a raytracer not to implement light that fades with the distance, depending on how you intend to use the ray tracer. Being explicit about possible simplifying assumptions will help users of the simulator understand the results of the simulated processes. Of course, simplifying the model comes at the risk of making it useless because it is too simple. On the other hand, a complex but incorrect model will fare no better.

Our solution to this problem is to frame the simulation as a game – with rules specified by us – in which the agents compete against each other. Instead of simulating general discussion or flow of information, we will simulate human beings playing a simple game. This has several advantages. First of all, since we design the rules of the game, we also define what actions are possible in the real world. This means, we are partly in control of the real world which makes it much easier to build an adequate model. The significant similarities between the socio-epistemic practice in our game and socio-epistemic practice from the real world enable us to compare the results of our simulator to the real world – are the winning strategies the same in the simulation as they are in the real world (the real world being real human beings playing the same game)?

Also such a game can be compared to various epistemic practice from the real world. Playing games is of course a very controlled socio-epistemic prac49

tice and we all have experience with that kind of socio-epistemic practice.

But a game like this can also be compared to more complex socio-epistemic practice such as for example the practice of science, both scientific investigation and participation in scientific communities. In scientific communities the goal is to answer certain questions about the world and it can be imagined that certain strategies for answering those questions can over time prove to be superior to other strategies.

It is possible to divide games into two types: competitive and non-competitive games. Competitive games are games where you compete with other agents to answer a certain question. The first agent to answer the question correctly wins the game. A non-competitive game is a game where an agent is simply making inquiries in a social context to answer a certain question, but not competing with others to answer it. This can be thought of as a game even though is does not involve competition. This is because it still involves a notion of winning: Agent A wins the game when A answers the question.

In our framework we will focus on competitive games, but it is possible to simulate both kinds of games and compare them by investigating which strategies work best in what kind of game setting. This is interesting because it is relevantly similar to what is going on in real life, for example when scientists compete to get a certain result (a competitive setting) or when someone tries to figure out the answer to a simple question by making inquiries.

The game-approach is also a huge advantage as it will help us spot potential errors in the model. In fact, we could even setup a special variant of a Turing Test that can test if our model is adequate – if we setup a game combining human and computer players and a human cannot spot which of the players are humans and which are computers, we can say that our simulation passes this special variant of the Turing Test. Because we are simulating games (and primarily competitive games), it is clearly defined what a ‘better’ strategy means – it is better if and only if it wins more games than the other strategy. Since we can spot errors more easily in this setup, there is a better chance we can actually simulate what the agents will do, not only what they should do. That is, not only what perfect agents will do, but what actual agents will do, as described in section 5.1.1 above.

Finally, the fact that we are implementing the simulation as a game makes us get implicit assumptions out in the open and make them explicit as we have to describe the rules of the game.

50

Of course, simulating any random game will not necessarily be relevant for social epistemology. For example, simulating agents playing tic-tac-toe will not reveal anything about flow of knowledge or information. We need a game in which the information flow is an integral part of the goal of the players. In a way, our game will itself be a simulation which makes our simulation a simulation of a simulation, but since the game could be played by real humans still acting as humans, but in more controlled settings, we have better chances of building an adequate model than if we had tried to simulate an entire society.

The goal of our simulator will be to enable the user to experiment with different kinds of epistemological strategies, different levels of logical capabilities and different lines along which the communication can flow. This way, the simulation can help answering a question like: in a game that requires both logical and communicative or investigative skills, how important are the logical skills compared to the communicative?

5.3

Introducing Knowledge Game

The game we will be using is what we call Knowledge Game. It has got its name from the fact that the players compete to achieve knowledge. Knowledge Game is a turn-based game for one or more players. The goal for each player is to decide whether a specific logical expression is true or false. The first player to (correctly) announce the truth value of the expression wins the game. If an agents answer falsely, then he is out of the game, but the remaining agents will continue.

The expression is called the game statement and a new game statement can be generated for each game played or kept as you please. Each time a player takes his turn, he can choose between three kinds of actions: 1. Ask any of the other agents a question 2. Make his own investigations 3. Answer whether the game statement is true or false The choices each player makes each round – choosing between the three possible actions listed above – and his abilities as an investigator determine who wins the game.

51

5.3.1

Entities

The game is set in a world of entities. Each entity has certain kinds of properties. A number of entities can be specified before the game starts and the system will then automatically generate the specified number of entities with random properties. The kinds of properties are color, shape and size.

The following values are possible: • Color: Red Red, Green Green, Blue Blue, Purple Purple, Yellow • Shape: # (Circle), 4 (Triangle), (Square), D (Pentagon) and 7 (Hexagon) • Size: XS, S, M, L, XL For example, a world could consist of three entities: a small green circle, a medium purple triangle and large yellow pentagon. Figure 5.1 shows such a world. Note that each entity is denoted by its name shown below the entity.

The color and shape of the entities can be seen directly from the figures and the size is denoted by the capital letter to the right of each entity. We always depict the worlds using a solid border around the entities.

S a

M b

L c

Figure 5.1: It is a small world.

There is no upper limit of the number of entities in the world. We can generate as many as we like. The number of entities in the world is one parameter we can use to control the complexity of Knowledge Game. The more entities in the world, the longer it may take an agent to get to know the world well enough to answer, whether the game statement is true or false.

This is of course assuming that the agent is not just giving us more or less random answers.

5.3.2

Game statement

The game statement is an expression in a monadic predicate logic we have designed for the purpose. This predicate logic enables us to express whether 52

the entities have the properties listed above or not. An example of a game statement is shown here: (Ex)(Yellow(x) & Pentagon(x)) The example game statement states that there exists a yellow pentagon.

It happens to be true in the world we saw in Figure 5.1 – the rightmost entity is indeed a yellow pentagon.

5.3.3

Acquiring new knowledge

If the players of Knowledge Game knew the properties of the entities from the beginning of the game, it would be quite simple to answer whether the game statement is true or false. Therefore, the agents have knowledge of the number of entities and nothing more. To acquire new knowledge – and in the end be able to answer whether the game statement is true or false – the agents can ask each other questions and/or make their own investigations.

Each of the two methods has its pros and cons. For instance, each agent has a certain degree of investigative skills. If the degree is low, he is more likely to get wrong results. This means he might be better off asking the other agents. On the other hand, each of the other agents he can ask has a certain probability determining how likely they are to speak the truth.

Some agents will be better logicians than others, so some agents may require more information about the world before they are able to decide the truth of the game statement. The properties determining which agent wins can very broadly be summed up in the following: • Investigation skills • Skills for asking questions, including how patient the other agents are with the agent • Skills for deciding when to ask questions and when to investigate • Logical skills

53

5.3.4

The Representations

To distinguish between what the agents believe and how the world actually is, we introduce a distinction between the world and the representation that each agent has of the world.

The agents’ representations of the world are illustrated almost like the real world. The difference between the real world and a representation is that the representation needs not contain every color, shape and size of the entities.

If an agent has acquired no knowledge of the real world, his representation will be empty as illustrated in figure 5.2 (given there are three entities in the world).

a

b

c

Figure 5.2: An agent with no knowledge of the world.

Note that the agent always knows the number and the names of the entities in the world. In this case, it is three entities with the names a, b and c. An agent with a few more beliefs about the world would have a representation of the world like the one in figure 5.3.

S a

L b

c

Figure 5.3: An agent with a little knowledge of the world.

As seen in figure 5.3, the left-most entity, a, and the right-most entity, c, are shown as stars. This means that the shape of the entities are currently unknown to the agent. We should also note that representations are always depicted using a dashed border and a little lighter background than the illustration of the real world.

5.3.5

Common sense realism

It is intuitive to understand the distinction between the real world and the agents’ representations of it in this system against the background of common sense realism.

54

Common sense realism is the position that there exists a mind independent reality. This reality exists regardless of what we think about it. Our beliefs about the world do not influence the world. This means that statements or expressions about the world have an objective truth value, they are either true or false. This does not mean that we necessarily have access to these truth values.1 An example of such a statement about the world could be ‘There is intelligent life somewhere else in the universe’. This statement is true if and only if there actually is intelligent life somewhere else in the universe. It has a truth value regardless of the fact that we do not know anything about the truth of the statement and possibly never will know.

As mentioned, Knowledge Game can be understood against the background of this common sense realism. We define a real world and the agents have representations of this real world. The real world does not change2 and is independent of the agents’ representations of it. This means that the game statement always has an answer, it is either true or false depending on the given world. However, the agents’ beliefs about the world and the game statement do not necessarily represent the world correctly. This means that it is perfectly possible that the truth value of the game statement for all what agent A knows is ‘true’ when the answer is in fact ‘false’.

The agents’ beliefs are by default not connected in any specific way to the world. The agent’s connection to the world in this system is his ability to investigate or ask the world. This connection is part of his epistemic profile and it is therefore possible for the agents to have different abilities with respect to investigation. But this connection is not as such part of the metaphysics of Knowledge Game. The agents do not have some kind of divine inspiration giving them knowledge about the world. We will look deeper into the agents’ abilities as investigators in Chapter 6.

In Figure 5.4, we see an illustration of the world and two agents’ representations of it. Agent A knows a little about the world: entity b is a large, yellow triangle. Agent B has a lot of beliefs about the world, but none of them are true.3 1

The version of realism presented here is sometimes also called metaphysical realism, see [Mil10].

2 That is, it does not change during a single game. A new world can be defined for each game.

3 Colors can sometimes be described as properties which are not mind independent.

That is, colors are not independent of the perceiving subject. However, when we talk

55

M

L b

a The World L c

XS d

L a

S

b

a

d

c

S b

Representations S c

Agent A

S d

Agent B

Figure 5.4: Illustration of the relationship between the world and the representations of it.

5.3.6

Definition of Knowledge

The name of this system is Knowledge Game. Therefore, it seems fitting to look at a definition of knowledge and what types of knowledge and beliefs our agents can have.

The standard definition of knowledge goes like this: A knows p iff 1. p is true 2. A believes that p 3. A is justified in believing that p The three conditions above have given this account of knowledge the name ‘JTB’ for justified, true belief.4 This is a definition of knowledge in the sense about colors, we view them as wavelengths. And those are indeed mind independent.

4 [Ste06].

56

that it states a set of necessary and sufficient conditions for Agent A to have knowledge.5 It is important to notice that under this definition of knowledge, our agents will not be able to tell the difference between knowing and having a justified false belief. Standing outside the system, we can determine whether an agent really has knowledge or just a false but justified belief, but since our agents have not got a direct access to the world, they can never be sure that what they believe about the world is actually true.

5.3.6.1

Gettier examples

The JTB account of knowledge has been criticized in various ways, one of the most famous criticisms being the Gettier examples. The Gettier examples show that there are cases where one can have justified true belief, but we would not say he has knowledge. In other words, JTB does not seem to capture what we intuitively mean by knowledge. This poses a serious problem for the JTB account of knowledge. Let us have a look at a classic Gettier example presented by Edmund Gettier in [Get63].

Suppose that Smith is justified in believing 1: Jones owns a Ford. (We assume like Gettier that justified belief can be false.) From 1 Smith can infer the following disjunctions (for example using the disjunction-introduction rule in natural deduction): 2 : Either Jones owns a Ford or Brown is in Boston.

3 : Either Jones owns a Ford or Brown is in Barcelona.

4 : Either Jones owns a Ford or Brown is in Brest-Litovsk.

Smith is justified in believing 2-4, because he is justified in believing 1 and 2-4 are entailed by 1. Jones does not own a Ford, but by coincidence Brown is actually in Barcelona. Therefore Smith holds a justified true belief, 3. But this seems to have a lot to do with luck and not much to do with knowledge. Smith has no evidence of Browns whereabouts and would not be justified in believing just ‘Brown is in Barcelona’. Still by the JTB account, he has knowledge of the proposition ‘Either Jones owns a Ford or Brown is 5

Note that often in the context of social epistemology, people talks about weak knowledge which is just true belief, see [Gol01]. However, since we will be concerned with Gettier examples later on, it makes sense to look at the JTB account of knowledge.

57

in Barcelona’. This example shows that justified, true belief might not be a sufficient definition of knowledge.

In our system there is a possibility of some scenarios similar to the above Gettier example. We will look at an example shortly, but first let us look closer at why the example above actually worked as a Gettier example. Statement 3 above comprised a Gettier example because a) Smith believed it, b) he was justified in believing it, c) it was true and d) we would still not say he knew. The statement was a disjunction, and as we know, it only takes one of the disjuncts to be true to make the entire disjunction true. That fact is what makes the example work. Only one of the disjuncts was true. Smith believed the one that was false, but because he was justified in believing it, he was justified in believing the entire disjunction. And since only one of the disjuncts needs to be true to make the disjunction true, the entire disjunction was in fact true. So Smith’s belief in the disjunction was both justified and true, but still we would not call it knowledge.

We will now show a completely analogous example in our system. We will make a game statement that is a disjunction. The first disjunct will be false but be believed by our agent. The second disjunct will be true, but not be believed by our agent. This means the agent will correctly believe the truth value of the game statement is ‘true’ but for all the wrong reasons. Let us consider an example where the game statement is this: The entity a is a square or entity b is a triangle.

XS

World

b

a

Representation

L

XS a

L b

Figure 5.5: A Gettier example as it can be constructed in our system.

The agent’s representation of the world and the real world are shown in Figure 5.5. The agent believes that a is a square, but in fact a is a pentagon.

58

Entity b is a triangle – and this makes the disjunction true – but the agent does not believe that. (He believes it is a circle, but it does not matter what he believes, it is just that he does not believe it is a triangle). The agent is justified in believing that the game statement is true, but only by luck since the reason for him believing the entire disjunction is his false belief in the first disjunct – so we have a Gettier example.

We can make lots of other examples. All they must have in common is that the agent comes to believe the right thing for the wrong reasons, they do not necessarily have to refer to specific entities like the example above.

An example that does not refer to specific entities could be this: The world contains a square or a triangle.

This game statement paired with the agent and world from Figure 5.5 will be a Gettier example too. The agent is justified, but falsely believes the first disjunct stating that the world contains a square. This makes him correctly believe that the entire disjunction is true, but not because the world contains a triangle which is actually what makes the disjunction true.

The fact that we can have Gettier-like situations in Knowledge Game, where an agent can determine the truth value of the game statement correctly based on a false belief, can be seen as both a weakness and a strength of the system. On one hand, we are interested in studying reasoning in this system, we do not want agents to win based on luck. In that perspective it is a weakness of the system. On the other hand, you can argue that reasoning based on falsehoods, like in our example above, is actually part of everyday life. It actually happens that we think we know something which is also true, but we have the wrong reasons for believing we know it. These situations are also epistemologically interesting and therefore we would like to be able to simulate them.

5.3.6.2

Justification

Up until now, we have been talking about justification quite a bit, but in a very loose manner. In the Gettier example Smith was justified in his belief, because he had ‘good evidence’. One solution to the Gettier problem is to strengthen the justification-condition of knowledge. An extreme way of doing that is to say that something can only be justified if it is true. But it seems like a way of defining us out of the Gettier problems, that would make 59

justification factive. It does not seem to get us any closer to how something can be justified.

Another way is to look at the dependency structures of one’s knowledge, that is to look at which beliefs are inferred by others. This is what foundationalists with respect to epistemic justification do.6 A third way to look at justification is to look at it in a more formal framework. Justification logics are epistemic logics together with the possibility of formulating knowledge and belief modalities in justification terms.7 In justification logics it is possible to express propositions like ‘X is justified by the reason t’. The reasons can be thought of as proofs. If one can prove p, then one is justified in believing p. But the thing is, we do not always want the agents in Knowledge Game to be perfect logicians. Therefore proofs might not be the type of reasons we want available to the agents. Evidence can be a form of justification suited for the purpose of creating agents with different logical and justificational skills.

In our system, agents can have evidence for their beliefs about the world, either because they have been investigating the world or because they have been asking the other agents. These are the current two sources of evidence.

But does this mean that beliefs based on these kinds of evidence are automatically justified? In some way, yes. But since our agents are not necessarily perfect investigators or flawless inquirers, as they are not perfect logicians, they make mistakes. Their abilities as investigators, inquirers and logicians are defined in their epistemic profiles. We will discuss these in much more detail in section 6.6. For now, it is enough to know that these abilities can vary. Say we have an agent who is really bad at getting true answers from the other agents, but he is almost perfect when it comes to investigating the world. You could say that he is justified in his beliefs coming from his investigations, but not justified in his beliefs coming from other agents, since he is so terrible at getting true answers. This means that we can see justification in this system as based on different types of evidence. But there are different parameters determining how justified one is in his beliefs, depending on his abilities. This means that justification can be seen as a scale in our system, the epistemic profile of the agent determines how justified one is in for example his investigations.

6 7

[Fum10].

[AF11].

60

5.3.7

Known Unknowns

Now we have looked at a definition of knowledge. But what about the parts of the world the agents do not know about? As mentioned in section 5.3.4, the agents know at least the names and number of entities in the world. The agents have to learn the shapes, colors and sizes of the entities in the world.

This means that even if the agent does not know anything about an entity, he always knows that the entity exists. This we can call known unknowns – for example, the agent knows that he does not know what color an object is.

Those different layers in knowledge are captured by a famous quote from a 2002 Department of Defense news briefing by Donald Rumsfeld: As we know, there are known knowns. There are things we know we know. We also know there are known unknowns. That is to say we know there are some things we do not know. But there are also unknown unknowns, the ones we don’t know we don’t know.8 This quote is also used by Alexandru Baltag and Sonja Smets in [BS09] as a description of a (lack of) introspection.

Currently in our system, we do not have unknown unknowns, since the agents always know the number of entities in the world. They always know what they do not know. In section 5.3.8, we will have a look at a philosophical example of why these unknown unknowns could be interesting to implement in our system.

First, let us look a little bit closer at the known unknowns. Not only do the agents know the name and number of entities in the world, our agents also know what properties of the entities they need to investigate to determine the truth value of the game statement. This means that they are able to seek new information in a very optimized way. And this gives us a much more advanced account of our known unknowns. Say we have a game statement like this: The world contains a red square.

We will imagine that a is such an entity – a red square – but the agent only knows that entity a is red, he does not know that it is a square. This 8

[BS09].

61

XS

L

a

b M

S

c

d Agent

Figure 5.6: An agent. is illustrated in figure 5.6. Note that the agent knows that none of the other entities are squares. To determine the truth value of the game statement, the agent does not need to investigate the color of b, c and d: they are not squares so it does not matter what color they are, they can never make the game statement true. On the other hand, the agent does need to investigate the shape of a: the agent’s belief regarding the truth of the game statement will change depending on the results of that investigation. If a is a square, the truth value of the game statement is true, if not, it is false. But how can we make sure that the agent is able to figure this out himself? This is not as trivial as it might look. To reason about this, the agent needs to know the truth tables for the connectives, in this case the conjunction 9 . From the truth tables he has to be able to reason about what knowledge he lacks, in order to determine the truth value of the game statement.

The conjunction is true if and only if both conjuncts are true. First, the agent has to look at his representation and find out which entities could possibly fulfill the game statement. Since neither b, c nor d are squares, none of them are relevant to this inquiry. For all those at least one of the conjuncts is false. This leaves us with entity a. The agent knows that a is red, but to know the answer, the agent must know if it is also a square. In this simple example, the only relevant known unknown is the shape of entity a. In more complex examples, there will be a whole list of entities’ properties which need investigation. How exactly this functionality of the agents is implemented, 9

The game statement is in this case a conjunction that states that there exists an entity for which it holds that it is red and that it is a square.

62

is described in Chapter 9.

5.3.8

Hesperus is Phosphorus

One more interesting philosophical question that rises on the background of the world and the agents’ representation of it in Knowledge Game, is the question of identity. Numerical identity is a relation that holds only between a thing and itself. This is opposed to qualitative identity, where things share the same properties, but are not numerically identical.

Let us have a look at an example. At my desk I have two pens, we can call them pen A and pen B. Both pen A and pen B are black uniball pens from Mitsubishi Pencil Co. Pen A and B are qualitatively identical, because they share the same properties, being black uniball pens from Mitsubishi Pencil Co. But pen A and pen B are not numerically identical. They are not the same pen. This also relates to the difference between tokens and types. On my desk there are two tokens, pen A and pen B. But only one type of pens.

A famous example used to illustrate numerical identity in the philosophical literature, is Frege’s ‘Hesperus is Phosphorus’. Hesperus is a star seen at the sky in the morning, also known as the morning star. Phosphorus is a star seen at the exact same spot in the evening, also known as the evening star. What people might not know is that Hesperus is Phosphorus, they are actually both identical to the planet Venus. This is a metaphysical necessity, it cannot be that Hesperus is not Phosphorus or the one of them is not the planet Venus. One the other hand, it is perfectly possible that we do not know this. It is an epistemological possibility that Hesperus is not Phosphorus.

This epistemological possibility of not knowing the identity relation between two apparently different entities is an instance of the above described unknown unknowns. This scenario is currently not possible to model in the system. The agents basically know too much about the world beforehand.

No surprises can occur to agents about the identity of the entities.

Since the ‘Hesperus is Phosphorus’ example is actually an example from the real world – people did not know that the two celestial bodies were the same – it could be interesting to be able to model something like that. If for example the agents only knew the number of the entities and not their names, there would be room for surprises such as an unknown numerical identity. This would also be a way of making the Knowledge Game even 63

more complex for the agents to play.

64

Chapter 6 Agents 6.1

What is an agent?

We already know that an agent is a player in Knowledge Game and we know that the agents have representations of the world. That is, they have beliefs about the world. Our world consists of entities with different shapes, sizes and colors and it is those entities the agents have beliefs about. We also know that the agents always know, at the very least, the number of entities and their names. But there can be a lot more to an agent than that. In the following, we will take a look at how we can model the agents performing actions or moves in Knowledge Game, the relationship between agents and how all this is combined into something we call the epistemic profiles of an agent.

6.1.1

Agent diversity

In Knowledge Game, we are interested in exploring the performance of different types of agents. We ultimately want to see if there is a difference in the performance of agents with different abilities. And if there is a difference, we want to know which types of agents perform best (and worst). For example, is an agent more likely to win the knowledge game, if he lies to the other agents all the time or is an agent more likely to win the knowledge game, if he investigates the world a lot? This is the longterm goal of the simulator and Knowledge Game is our mean to measure performance of the different types of agents. You can say that we are not interested in diversity with respect to objects of reasoning (i.e., beliefs), but rather in features of agents 65

that impact the way in which they reason (i.e., trust, logical competence, intelligence, and so on).

With this goal in mind, let us try to think about how we can diversify our agents. One obvious way is to give them different belief sets when the game begins. We could imagine giving some agents lots of true beliefs, others a lot of false beliefs. We could also have some agents with only the minimal beliefs about the number and names of the entities. However, this may not be the most interesting way to diversify the agents. It seems rather obvious that – ceteris paribus – the agent starting with the greatest number of (relevant) true beliefs will have the best chance of winning the game. Luckily, there are many other ways we can vary the agents. As we have hinted to above, we can vary an agent’s degree of skepticism towards his investigations and the information he gets from others. Another way to increase agent diversity is to vary their logical abilities. All the parameters that we can change when defining a parameter make up the epistemic profile of the agent. The question is what this epistemic profile should contain, what types of agents are we interested in? This is exactly what we will look at in the following sections.

6.1.2

Are agents like humans?

To be able to create the diversity of agents, we need to be able to vary their abilities in a number of epistemically interesting ways. But what is epistemically interesting? This question opens for the discussion about what to actually simulate with our program. Are we trying to simulate how human beings form beliefs or is this some kind of sandbox, where different types of toy agents can be tested? How do we make interesting simulations, and how do we choose what characteristics are epistemically interesting?

This is where what we learned in Part I becomes extremely relevant. To build a simulator, we will need an adequate model of the processes and objects we are trying to simulate. As we have already discussed in Chapter 5, we are not simulating human beings in a real society. We are instead simulating human beings playing a game called Knowledge Game. We define the rules of the game which means we not only define the model, but also what actions are possible in the real world. If we can learn something about the information flow in more real-life scenarios than Knowledge Game from watching real humans play Knowledge Game, then maybe we can learn something from 66

our simulator as well.

That is, of course, provided our agents are adequate models of human beings playing Knowledge Game. In this discussion, there is one important thing we must remember: There is no consensus about how the human mind works, at least not with respect to forming and updating beliefs. We have no knowledge of any general laws of nature that govern human reasoning. To simulate something, you need some kind of laws to build into your simulator.

In Part I, we discussed simulating the characteristics of a bridge and how it stands to different weather conditions. This could be done by implementing laws and forces known from physics. The real problem of using simulations in social epistemology is this, we do not know similar laws of nature about the human mind. It is even possible that no such laws exist. This means that every attempt to simulate the human mind will involve a lot of assumptions about the way the human mind works. And we would have no satisfying way of justifying or defending these assumptions. Unless, of course, we could test our results against the real world. As mentioned in Part I, it is often not feasible to do this.

This is true not only for simulations in social epistemology, but simulations in general: it is way too expensive to build a real bridge just to check if the results of your simulator are correct. In social epistemology it is even worse: it is not possible to ‘build a society’ just to check your simulator. So, how do you know if your model for a simulation in social epistemology is adequate? This is where our Knowledge Game comes to rescue again: it is a lot simpler to have four or six people play a game than it is to build an entire society.

This approach has some advantages. First of all, we do not need to chase some very complex agents and network of agents and show that they behave like in the real world. We can keep the agents fairly simple, which makes it much easier for us to keep track of the different parameters influencing the outcome of Knowledge Game. Second, we can define the agents’ properties as we like and only choose the properties we like to investigate the performance of, in a very controlled environment. We just have to remember that any possible results or conclusions generated from the simulations only hold under a very specific set of assumptions.

In general, you may say that our system will lack full faithfulness with respect to both our building blocks and the social process, described in Section 1.2.2. Our building blocks might not be fully faithful simply because we 67

do not have all the relevant information about the human mind and social interaction. And therefore our social process, the Knowledge Game is possibly not fully faithful. But our system has a higher degree of adequacy than for example Laputa and might possibly have a higher degree of adequacy than other simulators in social epistemology, because our system simply fits it purpose of running simulations in social epistemology better than Laputa does. This holds for both the first and second step, that is both our agents and our social process have a higher degree of adequacy than Laputa and possibly other competing systems. Although not fully faithful, our system may be the best and most adequate available alternative.

6.2

Actions

Before we look at the epistemic profiles of the agents in detail, we should have a look at the actions or moves the agents are able to perform in Knowledge Game. This is basically a specification of Knowledge Game, and the way we set up the moves makes a big difference for the agents’ strategies.

We basically work with two types of moves: each turn the agent must choose to either answer the truth value of the game statement or try to get new information. The second option, getting new information, can be divided even further into inference, investigation and inquiry. Benthem sees the same three ways of getting information in [vB08]. He calls them observation, inference and communication: Rational agents have a great variety of ways for getting new information, and they use it for a great variety of purposes. Observation, inference, and communication all seem equally respectable ways for agents to obtain information, and in the program of ‘logical dynamics’ (van Benthem 1996), they should indeed all be handled on a par. (. . . ) Of the three mentioned sources of information that can be used in action, two seem similar, viz. observation and communication.

An observation is an answer to a question we pose to a special omniscient agent, viz. nature, while conversely, an answer to a

68

question may be seen as the observation of some fact – if we accept what the speaker says.1 Let us have closer look at each of them.2

6.2.1

Inference

In Knowledge Game, inference can be seen as a social way of getting new information, similar to the way we get new information when solving puzzles.

Every time it is an agent’s turn, he first asks himself ‘Can I determine the truth value of the game statement?’. When it is an agent’s turn in Knowledge Game, he has a set of beliefs similar to the premises in a puzzle. At some point in the Knowledge Game, these beliefs or premises will be enough to determine the truth value of the game statement. But even though the agent contains these beliefs, he still has to check at each turn, if he can determine the truth value of the game statement. He checks this using a model checker (the technical details of this will be explained in Chapter 9). If he can answer the game statement, then he does. Otherwise he moves on to either investigations or inquiries.

The step, where the agent sees that he is in fact able to determine the truth value of the game statement with his current beliefs, can be seen as an upgrade of implicit knowledge to explicit knowledge. Benthem writes: More generally, external observations create implicit knowledge by eliminating worlds, while actions of inference increase internal access to the remaining worlds, thus upgrading implicit knowledge to explicit knowledge. Why is this access useful? Many cognitive activities involve computation on syntactic code, and hence the agent’s access determines its potential for inference, but also other relevant activities, such as model checking to find truth values of formulas at worlds in the current model.3 1

[vB08] p. 1.

The attentive reader might have noticed a potential conflict with a big philosophical position in the section above. The potential conflict lies in defining inference as a way of getting new information. This is a problem Hintikka calls ‘a scandal of deduction’, namely that deductive inferences at first sight appear not to give us new knowledge. To read more about scandal of deduction see the appendix, Chapter A. 3 [vB08] p. 4.

2

69

6.2.2

Investigation vs inquiries

In section 6.2, we mentioned that investigation is a lot like asking the world – making inquiries with the world. So what is the difference between investigation and inquiries? There is no obvious answer to this question. Of course, since we define the rules of the game, we can pick anything we like, but if we have hopes of comparing Knowledge Game to some sort of society, we need to have good reasons for setting things up the way we do. Again our lessons from Part I are extremely relevant: if we ever want Knowledge Game to be a model of anything in real life, we must choose wisely when building our model or at least be very specific about our choices and assumptions. We will try to do both.

First of all, other agents have a lot more dynamic attitude towards the agent than the world does. In Knowledge Game the world is completely static, but the other agents will update their beliefs during the game, which means that asking agent A whether he thinks entity a is blue may yield a different result in the beginning of the game than it does later in the game.

The interaction between the agents – the inquiries – is the social part of Knowledge Game.

So we have differences between investigating and making social inquiries, but for all we have seen until now, it looks like the optimal strategy will always be to be a good investigator. You do not have to worry about other agents not knowing the answer and you do not have to worry about them not caring to answer you. We need another form of difference, a more fundamental one.

To come up with a fitting difference we will have to look at what kind of questions an agent might ask either the world or other agents.

6.2.2.1

Specific entity

As opposed to inference which is part of every move in Knowledge Game, investigation and communication is something the agent chooses to do. Basically, if the agent cannot determine the truth value of the game statement, then he chooses between investigation and inquiry. This choice will be represented as a probability in our system called InvestigationProbability.

This means that the agents will have a property, InvestigationProbability, defining their probability of doing investigations or inquiries in case they cannot answer the game statement.

We saw in section 5.3.7 that our agents have the ability to know what 70

knowledge they lack about the entities in the world. Because of this, we make the question asked by the agent very specific: the agent can ask about a specific property of a specific entity. Let us have a look at an example. We will use a world like the one shown in Figure 6.1. And the game statement is this: The world contains an entity with the size extra-small and the color orange.

M

L

L c

b

a S e

S

XS d

XL g

f

M h

Figure 6.1: A world with eight objects.

The value of the game statement is in fact true, because d has the size extra-small and the color orange. Let us say that the agent knows neither the size, nor the color of d. Since he knows that he does not know the size and color of d and this is necessary to determine the truth value of the game statement, he is able to directly ask the world: • What is the size of d? • What is the color of d?

In fact, he could ask even more specific questions than the ones above.

For example, our agent does not need to know the actual size of object d, he just needs to know if it is extra-small or not. It does not matter whether it is for example medium or large. This means, we could make the agent instead ask the question: Is the size of d extra-small? The ability to choose cleverly between the different kinds of questions might indeed be an interesting parameter to add to the system. However, it also increases the complexity of the system and in this first version we want to keep things relatively simple. When an agent is making inquiries, that is asking the other agents, we will use only one question form, namely the one used in the two first examples above. It can be written like this: 71

• What is the [property] of [entity name]?

It is a very specific form of question with a very specific form of answer.

Of course, in some cases the other agent does not know the answer (or for some reason does not want to answer). In those cases, the agent asking the question will receive a value indicating ‘no answer’.

In the following section, we will look at another game to get inspiration for deciding on a way to ask the questions to the world, that is making investigations.

6.2.2.2

Go Fish

In the card game ‘Go Fish’, the players get seven cards on their hand and the rest is sprawled out in a non-orderly pile called the ocean. The goal is to gather all four suits of one rank and do this the most times. The players, Ann and Ben, ask each other for cards of particular ranks. If Ann asks Ben for cards of rank four, then if Ben has any cards of rank four, he hands them over to Ann. If not, he tells Ann to go fish. Ann then draws a random card from the ocean.

This card game can be an inspiration for a way for the agents to investigate the world. Imagine an agent who wants to investigate the world. Instead of asking for a specific entity or property, he ‘goes fishing’. He simply gets a random entity or property of an entity, depending on how we set it up.

Now, what would the agents lose with that kind of investigation? Of course they would no longer be able to get the very specific information about entities and their properties. By introducing go fish to investigations, we would introduce the possibility of getting exactly the information the agent needs by pure luck. And also the risk of getting information the agent does not need at all to determine the truth value of the game statement. Or the third possibility: getting something you already know.

If we do not make the information the agent gets back completely random, this seems like an interesting way to ask questions when investigating. The agent could investigate a specific entity and get one or two random properties back. This seems to fit quite nicely with how inquiries versus investigations in everyday life works.

For example, if I want to know when Steve Jobs was born, I could ask someone I know directly: When was Steve Jobs born? This will give me a specific answer or maybe no answer at all. Alternatively I could ‘ask the 72

world’ by searching the internet for ‘Steve Jobs’. This will yield a lot of information about Steve Jobs and perhaps some of this will be his year of birth, perhaps not. If our agent wants to know whether entity d is orange, he can investigate that entity and if he is lucky, he will get to know the color. If he is less lucky, he will get to know the shape which is irrelevant to him at the moment. This is an interesting way to make the investigations a little less efficient compared to inquiries and thereby perhaps a bit more realistic. In conclusion, this means that the form the question takes when making investigations will always be this: • What information do you have for me about [entity name]?

Furthermore, the ‘go fish’ type of investigation serves as a great example of the many choices we make when building a simulator. These choices are extremely important and a lot of things in Knowledge Game would be different if we chose to change any of the decisions we made.

6.2.2.3

Quality of investigations

In our system, the way the agents do investigations is implemented as a probability of getting the correct information from the world. This is a property of the agents called InvestigationQuality. This gives us the freedom to define an agent’s investigative skills as we like. Some might get the right answer from the world every time and some might only get the right answers from the world by pure luck. An agent with an InvestigationQuality of 0.6 will have a 60% chance of getting the correct information when he asks the world.

6.2.3

Inquiries

We have now seen the kinds of questions an agent can ask when making investigations and inquiries. What we have already seen is pretty much all there is to the non-technical side of the investigations. However, there is a lot more to inquiries, which is exactly what we will discuss in this section.

There is a fundamental intuitive difference between investigating the world and asking other agents. When you investigate the world, you can of course be better or worse at doing the investigations and thereby get new information. When you ask other agents, there is a much more complex dynamic in getting correct information. Whether the information you get from 73

others is usable depends not only on your abilities as an inquirer, but also on a number of other factors like whether the agent is a liar or not, whether you trust him or not, whether he trusts you or not, and maybe also your past experiences with the agent. To model the communication between agents, we would like to have a more complex account of inquiry than what we had with investigation. The relationship between agents should be more complex than the relationship between the world and an agent (if you can even call that a relationship).

But it is clear that acquiring beliefs through communication sometimes plays an important role, for example when playing games such as Go Fish.

It is part of the game that one has to answer truthfully to questions like ‘Do you have any threes?’, so when you ask another player this, you can rely on the answer. This is part of the rules of the game. Communication is part of the way one wins the game and we can imagine that in more advanced games where one is allowed to lie and bluff, communication plays an even bigger part.

In Knowledge Game we would like the agents to be able to communicate their beliefs to each other, but with different filters so to speak. This enables us to investigate the kinds of communication strategies that pay out the best. We want our agents to behave as much like real human beings playing Knowledge Game as possible. This is in no way an easy task. As we have already mentioned, very little is known about how the human mind works when it comes to acquiring new beliefs. This means that, once again, we will have to make some assumptions and make sure to make them very clear.

First of all, one epistemic concept that has gathered a lot of philosophical attention is trust. In section 6.3, we will have a look at a way to simulate and calculate trust between our agents in Knowledge Game. Trust can serve as a filter on the communication, controlling how the agent updates his beliefs.

In Chapter 19 in Part IV of this thesis, we also suggest another epistemic concept, patience, which is our own invention. We believe that patience can serve as another kind of filter, controlling the information getting out of one agent and into another, but we will leave the implementation of it for further work.

74

6.3

Trust

Trust as an epistemic concept has been widely discussed in connection with the philosophy of testimony. Trust obviously plays an important role when we have to decide whether to believe the testimony of another agent or not.

When we ask ourselves a question like ‘what reasons do I have to believe what agent A is saying?’, one answer could simply be that I trust agent A. The interesting thing about trust is that it intuitively can serve as a reason or justification of believing something. But it also, at least under some definition, clashes with rationality. Under some interpretations, trust seems to have nothing to do with rationality: Since trust inherently involves risk, any attempt to eliminate the risk through rational reflection could eliminate the trust at the same time. Second, trust tends to give us blinkered vision: it makes us resistant to evidence that may contradict our optimism about the trustee (Baker 1987; Jones 1996). For example, if I trust my brother not to harm anyone, I will resist the truth of any evidence to the contrary. Here, trust and rationality seem to come apart.4 But it might not be so simple that trusting is always contrasting rationality. It might depend on the surrounding environment. Alvin Goldman makes this point when he considers the practice of blind trust, that is the practice of accepting and trusting everything you hear.

Consider the simple acceptance practice of BLIND TRUST, which is to believe every report or piece of testimony you hear. BLIND TRUST is an excellent acceptance practice if it complements a reporting practice that generates only truths, in other words, if it is used in an ideal reporting environment. An ideal reporting environment is one in which speakers never misperceive, misremember, or deliberately distort the facts. Unfortunately, there is no realistic prospect for such infallibility and scrupulous honesty. So in the end we need to ask how BLIND TRUST would fare in other reporting environments. Clearly, it is not optimal in every possible reporting environment; in a speech environment 4

[McL11].

75

where every report is false, BLIND TRUST is terrible. The optimal strategy in that environment is BLIND CONTRATRUST, that is, believing the negation of every report. Is there any acceptance practice that is optimal in all reporting environments, in other words, better in each reporting environment than every other acceptance practice would be? As in game theory, the answer appears to be ‘no.’5 In general, it seems that we cannot say anything about the epistemic value of a certain trust strategy, without looking at the surrounding environment in which the trust strategy is practiced. One strategy can be optimal in one environment and suboptimal in another.

The veritistic merits of a hearer acceptance practice cannot be assessed in isolation from the reporting practices that it complements. This point can be appreciated by reflecting on results from game theory. A particular strategy for playing a certain game can be very successful when pitted against a second strategy but much less successful when used against others (Axelrod 1984). Similarly, a given message-acceptance strategy may be veritistically successful when used in the environment of certain reporting practices but unsuccessful in response to different reporting practices.6 This is exactly where a simulator comes in handy. It can be extremely difficult to calculate the optimal epistemic strategies once the environment becomes more advanced and it is hard to isolate the different influencing parameters. In our system, we can set up a network of agents with a certain distribution of trust and we can let them play Knowledge Game. And if we want to, we can run the simulations a lot of times. By looking at the properties of the winning and possibly the losing agents, we can conclude something about different setups of trust. Of course, our findings will only be guaranteed true in the Knowledge Game environment we set up.

5 6

[Gol99] p. 109.

[Gol99] p. 109.

76

6.4

Trust in Knowledge Game

Knowledge Game implements a simple notion of trust. One agent’s trust in another agent is simply represented by a floating point number between 0 and 1. The value 0 is absolutely no trust, and 1 means everything the other agent says, is taken for face value.

Every agent is created with a certain amount of trust in himself, and he has a certain amount of trust in the agents he is connected to. There is no difference between how the trust an agent has in himself works and how the trust he has in his fellow agents works. So what to do with those trust values, how do they influence the agents’ behavior in Knowledge Game? Let us look at a quote from Goldman: One clear example of a rationality approach to social epistemology is that of Keith Lehrer and Carl Wagner (1981). They consider cases in which inquirers assign credal trust or “respect” to one another’s opinions. Each person, they argue, should update his opinion on a target question to reflect other people’s opinions on the same question, as a function of the degree of trust accorded to those other people. Lehrer and Wagner contend that rationality demands a series of iterated opinion changes on the part of each inquirer.7 We will not go further into the functions of Lehrer-Wagner, but we will use the general principle that the agents should update their opinion as a function of the degree of trust: When an agent gets information regarding a property of an entity, for example that entity a is red, he can assign more or less weight to that information. If he trusts the other agent a lot, he should give it greater weight than if he had not trusted the agent.

Each agent remembers the information he gets from the other agents and from investigation, and he uses this to form his beliefs. The more an agent sees something when investigating, or hears it from other agents when making social inquiries, the more he will believe it. One might think that this is a dangerous assumption, as some real life agents will indeed be extremely stubborn, and no matter how many times they hear something, they will not believe it. In some cases that may even be rational, if the agent does not 7

[Gol99] p. 109.

77

trust the agents who speak to him. This is exactly where trust comes in. If the agent does not trust the agent speaking to him, he will assign the piece of information a very low weight, which means it will have very little effect on his belief.

The agents will keep track of the sum of the weights of all information they get. A piece of information is for example ‘Entity a is red’. The weight for a single piece of information is the trust the agent has in the source of the information. If the information comes from another agent, that agent is the source of the information. If the information comes from investigation, the agent himself is the source of the information, which means his self-trust is used for the weight.

When the game starts, the agents have not yet received any information, so for any agent we can represent his weight-sums for the color of a specific entity as shown in Table 6.1. By ‘weight-sums’ we just mean the sums of weights of all pieces of information the agent has received.

Sum of weights

Red 0

Green 0

Blue 0

Yellow 0

Purple 0

Orange 0

Table 6.1: Weight-sums for an agent for a specific entity at the beginning of the game.

When an agent needs to decide, for example, what color he believes a specific entity is, he will use the weight-sums to calculate his belief. To show how this works let us look at an example. Suppose there are four agents, Alice, Bob, Charles and Dave. Alice is trying to find out what the color of entity e is, so she asks Bob. Alice trusts Bob with a value of 0.75. Bob tells Alice that e is green. Alice will then add 0.75 to her ‘belief weight-sum’ for green for the specific entity e. Now Charles comes by. Alice is a bit more skeptical towards Charles and accordingly we set her trust in Charles to 0.50.

Charles says that e is purple, thus Alice will add 0.50 to the weight-sum for purple. As the final step of the example, let Dave also tell Alice that e is purple. Alice will add the trust she has in Dave – let it be 0.10 for the example – to her existing belief in that property value. She already has a weight-sum of 0.50 in purple, so the end result will be 0.60. This is shown in Table 6.2.

The maximum weight-sum at the moment is 0.75 which belongs to the 78

Sum of weights

Red 0

Green 0.75

Blue 0

Yellow 0

Purple 0.60

Orange 0

Table 6.2: Weight-sums for Alice after having received three pieces of information. color green. This means Alice currently believes8 the entity is green. However, this does not necessarily mean she is willing to assert that the entity is green. Assertion is the topic of the following section.

6.4.1

Assertion

We have just seen how a weight is stored in each agent, for each possible value of each property, for each entity in the world. When an agent is about to decide whether the game statement is true or false, or when he is asked about a property of an entity by one of the other agents, the system must first calculate what the agent believes and then calculate whether he is willing to assert it. In other words, we distinguish between what an agent is willing to assert and what he believes. The reason we make this distinction is to enable an agent to have beliefs that are not strong enough for the agent to actually act upon them, i.e., communicate them or determine the value of the game statement based on them. As an example, suppose that you have invited a friend over for dinner. The last couple of times you have seen him, he was thirty minutes late. Therefore you believe that he will be late again tonight.

He is supposed to arrive at 19.30 and it is now 19.29. As it happens, you have just realized that you forgot to buy dessert. If your friend is late, you have plenty of time to go and buy dessert, but if he arrives on schedule, you will be out when he arrives and he will be left standing in the rain. That 8

It may seem tempting to object that defining belief as just the alternative with the greatest weight-sum is too simplistic or counter-intuitive and that we should introduce a ‘belief threshold’ which determines whether a weight-sum is large enough that we consider something a belief. However, it does not matter to the results of the simulation whether an agent considers something a belief or not, it only matters whether he be believes it enough to act on it – and we will introduce a threshold for exactly this in the next section.

If you are still not convinced, feel free to imagine that there actually is a ‘belief threshold’ that the weight-sum must exceed for the agent to consider something a belief. It makes no difference to the actual actions of the agent, those will be determined by the threshold for assertion described in the next section.

79

way, you might believe he will be late, but not believe it enough that you are willing to act accordingly. This is the way we can think of belief in Knowledge Game. Assertion, on the other hand, defines whether the agent is actually willing to act on the belief. The action can be either communicating or answering the game statement. We make no distinction between willingness to communicate p and the willingness to use it when determining the value of the game statement, i.e., if an agent is willing to assert p, it means he is willing to act on p in any way. For an agent to believe p enough, a certain criterion must be met, he must believe it with a certain strength. How this works and how it can vary from agent to agent is described below.

Before we see how to actually calculate assertion, we should make an important note: Our agents are able to lie to each other. This means that both of the following are true: 1. An agent can communicate p without actually believing p.

2. An agent can decline to communicate p, even when he believes p enough to actually assert it.

We can think of belief, assertion and the actual communication as layers of an onion. At the core we have the actual beliefs. Outside the beliefs, we have a layer that calculates what the agent is willing to assert when speaking the truth, i.e., determining the value of the game statement or talking to an agent he is willing to speak the truth to. The outermost layer is what determines what the agent will actually communicate to another agent. This is pictured in Figure 6.2.

We define an agent’s belief about a property of an entity as the value with the greatest weight-sum. If two or more values have the same maximum weight-sum, one of them will be picked at random. The agent in table 6.2 believes the entity in question is green, simply because 0.75 is the greatest value in the table. However, the agent is not necessarily willing to assert that the entity is green. This depends on whether the weight-sum for green for the entity (0.75 in this case) is greater than a certain threshold. How this threshold works is described below. And of course, as mentioned above, some agents tend to lie to the other agents. How this works is described in section 6.5.

80

Beliefs

Assertion Actual communication

Figure 6.2: The agent as an onion.

6.4.1.1

Calculating assertion

Now we should see how the agent actually decides what he believes the color of the entity in question is, and then whether he is willing to assert it.

The first thing he will do, is to find the value with the highest weightsum. We will expand our example using the numbers shown in table 6.2.

Green has a weight-sum of 0.75, which is the greatest value in the table. The agent will only assert that he believes a value if he has a certain amount of evidence for that value. Each agent can be different regarding how much evidence he requires before asserting a value. Therefore, as we mentioned above, each agent has a certain threshold which defines how much evidence the agent must have before he will assert anything. This threshold is called his AssertionProbabilityThreshold and is a floating point value between 0 and 1. Now, how should this threshold be used in our example? One way could be to just check if the weight-sum of green (the color with the greatest weight-sum) is above the threshold. If we imagine that AssertionProbabilityThreshold has a value of 0.50, the agent could assert that the color is green. This does not make much sense, though, since the agent may have

81

almost as much evidence for one of the other colors. In fact, in our example the weight-sum for purple is also above our imagined value of 0.50 for AssertionProbabilityThreshold.

Instead, we will look at the ratio of the weight-sum of green compared to the total weight-sum. The total weight-sum is 0.75 + 0.60 = 1.35, so the ratio is 0.75/1.35 = 0.55. This way we make the calculation sensitive to how much evidence the agent has of the specific color compared to how much evidence he has of the other colors (or sizes or shapes depending on the property in question). This also means that the more evidence an agent already has, the more evidence will be needed to change his belief.

6.4.1.2

Bias

The way of calculating an agent’s belief and, more specifically, what he is willing to assert described above may at first sight seem a bit odd for some cases. For example, imagine two agents, Alice and Bob, and an entity, e.

Say that at the beginning of the game Bob – in which Alice has almost no trust in – says that e is green. Now, Alice has only one piece of evidence regarding the color of e, and that piece of evidence comes from an agent she only trusts with, say, a value of 0.01. However, when calculating her belief, Alice will do as described above: first she will find the maximum weight-sum which is 0.01, then she will calculate the total sum of weights which is 0.01, as she has only one piece of evidence. At last she will calculate the ratio between the maximum weight-sum and the total which is 0.01/0.01 = 1.0.

This means that Alice fully believes e to be green, even though she has only had one piece of evidence from a source she does not trust. This is clearly not how we want (all of) our agents to behave.

One way, we could try solving the problem above, involves keeping track of how many times the agents has seen evidence for each value for each property for each entity. That is, instead of just adding to the weight-sum each time new evidence is seen, we could also increment a counter. Then we could add another attribute to the epistemic profile of the agents, say something like AssertionEvidenceCountThreshold, which would be a threshold for the minimum number of observations, i.e., pieces of evidence an agent must have for a specific property before he asserts it. For example, we might set this attribute to 3, which means our agent must have more than three observations of green to state that the color of the entity is green. In our example, the

82

agent has one observation, which means he will not assert anything. However, it turns out that this is not the best way to solve the problem since it ignores another problem: information from multiple agents which the agent has very little trust in.

Imagine that we extend our example a bit and add evidence from three more agents in which Alice has very little trust. Say the three agents all assert that e is green and Alice trusts each of them with a value of 0.01. This means the total weight-sum is 0.04, the weight-sum for green is 0.04, hence the ratio is still 1.0. However, the number of observations of green for the specific entity is now 4, which is greater than the threshold of 3. Now, Alice only has information from agents she has almost no trust in – but still she is willing to assert that the color is green. This seems quite odd and we should be able to do better.9 A simpler and more elegant solution is to not keep track of the number of observations, but instead introduce a bias for each of the agents. The bias will be the weight-sum that each cell in the weight-sum table contains when the game begins. The bias value will be part of the epistemic profile. We call it EvidenceBias. As an example, imagine that Alice from our previous example had an EvidenceBias of 0.1. Table 6.3 shows what Alice’s weightsums look like after getting evidence for green from the four other agents.

Sum of weights

Red 0.1

Green 0.14

Blue 0.1

Yellow 0.1

Purple 0.1

Orange 0.1

Table 6.3: Weight-sums calculated using a bias of 0.1.

We can now calculate the ratio like this: first, the maximum value is 0.14, the total weight-sum is 0.14 + (5*0.1) = 0.64 which makes the ratio 0.14/0.64 = 0.22. Given an AssertionProbabilityThreshold of 0.5, Alice will no longer assert that the entity is green. Combining AssertionProbabilityThreshold and EvidenceBias, we can make agents that are very diverse when it comes to how much information they require to assert something. However, that is far from all there is to the epistemic profiles of the agents. In the following sections, we will look at a few more attributes. Note 9

This approach not only fails to exclude situations with a lot of non-trusted sources, it also fails to include a situation with one very highly trusted source. So it is bad in both ways.

83

that there is a concept we do not discuss in this part of the text, and that is independence. Independence is discussed in Chapter 22 in Part IV Further work.

6.5

Liars

We also need to find out how we want to model lying in Knowledge Game.

One concept of lying could be to always just tell people the opposite of what you believe. Then you ensure that you never tell the truth. But the challenge with this kind of lying is that if the agents you lie to find out that this is the way you always behave, then you are actually still as good a source of information to them, as you were when you where telling the truth. Why is that? Let us have a look at two agents, Ann and Ben. For some reason Ben does not like Ann or just likes to lie, no matter what, Ben is always lying to Ann. This means that every time Ann asks Ben something, Ben tells exactly the opposite of what he believes. Let us say that Ann asks Ben ‘Is the color of entity e red?’. If Ben believes it is, then he will tell Ann ‘No, e is not red’.

If Ann somehow finds out that Ben is lying all the time in this way, then Ann can simply just negate the answer from Ben and then Ben is as good a source of information to Ann as he ever was. Therefore, this kind of lying is not the one we are interested in implementing in Knowledge Game.10 Instead, we want some kind of random or unpredictable lying. So if Ben wants to lie to Ann, he will just answer randomly when Ann is asking him anything. The attentive reader will notice that the above example where Ann is asking ‘Is the color of entity e red?’ is actually not possible in Knowledge Game. This is because we have restricted the questions, the agents can ask one another, to the form: • What is the [property] of [entity name]?

It is not possible for the agents to ask closed questions with the only possible answers being ‘yes’ and ‘no’. Therefore, you cannot just negate a lying agent’s answer and thereby get the correct answer. This helps us avoid 10

As discussed in Chapter 2, this is somewhat similar to what happens in Laputa. If you are an agent in Laputa and you treat another agent as a liar, then you will consequently negate all statements from this agent. This is because you are assuming that a lying agent would always negate his beliefs. But as we also discussed, this is not a very realistic way to model lying.

84

the unwanted kind of lying where the information is actually as valuable as truth-telling and instead use an unpredictable lying.

Let us say that Ann asks Ben the permitted question ‘What is the color of entity e?’ and Ben believes the color to be red. But since Ben is a liar, at least to Ann, he will pick a random answer: either yellow, blue, green, orange, purple or red, and answer that. In that way, Ann cannot infer from Ben’s answer, what Ben really believes. And that is exactly what we want in Knowledge Game.11

6.5.1

TruthProbability

We will implement an agent’s inclination towards lying as a probability called TruthProbability. Let us say that Ben has a TruthProbability of 0.1.

When Ann asks Ben something, there will be a 10% chance that Ben actually tells Ann what he believes is the truth. But there will be a 90% chance that Ben will say something random.

In Knowledge Game, we will let the TruthProbability be a static property of the communication line between two agents. This means that we can model agents who always lie or always tell the truth, and everything in between. This gives us the possibility of modeling four types of agents with respect to lying and truth-telling: • General epistemic saint – Agent A is an epistemic saint in general if and only if, for any other agent A*, agent A always tells what in his view is the true answer to the question posed by A*.

• Specific epistemic saint – Agent A is an epistemic saint specifically with respect to another agent A* if and only if, A always tells what in his view is the true answer to the question posed by A*.

11

One could wonder why Ben would be interested in answering ‘Red’ when in fact this is what he actually believes. The answer to this question is similar to the one above where one could just negate a lying agent’s answer and get to the truth. Let us say that Ann asks Ben a hundred times ‘What is the color of entity e?’ and never once gets the answer ‘Red’, but gets all the other colors, then she would still be able to infer that ‘Red’ is what Ben believes to be true. Therefore, when Ben wants to lie, he has to pick a truly random answer between all the possible answers. In that way he ensures that his answer is absolutely useless to Ann.

85

• General epistemic sinner – Agent A is an epistemic sinner in general if and only if, for any other agent A*, A always provides what in his view is a wrong answer to the question posed by A*.

• Specific epistemic sinner – Agent A is en epistemic sinner specifically with respect to another agent A* if and only if, A always provides what in his view is a wrong answer to the question posed by A*.

We will use these types of agents later on. One important thing is that TruthProbability will not change during a game, it will be a static property of the communication lines between the agents.

6.6

Epistemic profiles

In Chapter 6, we have described how our agents work and how the interaction between the agents work. As pointed out in section 1.2.2, this is the first step – our building blocks – when we want to simulate phenomena in social epistemology. One might be curious to see how all this is represented in the source code. We have chosen a very simple way to define different agents and the network of them. Both our agents and the connections between them are defined in XML – Extensible Markup Language.

Let us have a look at this and at the same time summarize the different properties of the agents and their interaction described in Chapter 6.

6.6.1

Agents

In XML, our agents look like this: <Agents> <Agent Name="A" AssertionProbabilityThreshold="0.0" EvidenceBias="0.0" SelfTrust="0.9" InvestigationQuality="0.0" InvestigationProbability="0.9" />

86

<Agent Name="B" AssertionProbabilityThreshold="0.2" EvidenceBias="0.1" SelfTrust="0.9" InvestigationQuality="0.99" InvestigationProbability="0.9" /> </Agents>

Here we see two agents named ‘A’ and ‘B’. They have different properties.

6.6.1.1

AssertionProbabilityThreshold

Agent A has an AssertionProbabilityThreshold of 0.0 and agent B has an AssertionProbabilityThreshold of 0.2. The AssertionProbabilityThreshold defines how much evidence they require before asserting anything. Agent A requires no evidence at all in order to assert something.

Agent B does not require a lot of evidence, but he does require some evidence. Agent B requires enough evidence, so that his belief needs a ratio of the belief and the weight-sum higher than 0.2 (his AssertionProbabilityThreshold) before he will assert this belief.

6.6.1.2

EvidenceBias

Agent A has a EvidenceBias of 0.0 and agent B has a EvidenceBias of 0.1. The EvidenceBias controls, in cooperation with AssertionProbabilityThreshold, how much evidence an agent requires before he is willing to assert something. Agent A requires absolutely none before he is willing to assert anything, since both his EvidenceBias and AssertionProbabilityThreshold are both 0.0. Agent B requires a little more evidence because his EvidenceBias is 0.1, which means he needs to see more evidence of the same belief before the ratio of this belief and the weight-sum will be higher than his AssertionProbabilityThreshold, and he will assert the belief.

6.6.1.3

SelfTrust

Agent A has a SelfTrust of 0.9 and agent B has a SelfTrust of 0.9. This means that they are both highly confident in their own investigations.

87

6.6.1.4

InvestigationQuality

Agent A has a InvestigationQuality of 0.0 and agent B has a InvestigationQuality of 0.99. The InvestigationQuality is the probability that the agentâ&#x20AC;&#x2122;s investigations will yield correct results. Agent A has a InvestigationQuality of 0.0 and will therefore always get a random result when he investigates the world. Agent B has a InvestigationQuality of 0.99 and will therefore almost always get the correct result from his investigations.

6.6.1.5

InvestigationProbability

Agent A has a InvestigationProbability of 0.9 and agent B has a InvestigationProbability of 0.9. The InvestigationProbability is a probability for the agent to either do inquiries or investigations, when he cannot answer the game statement. This means that both agent A and agent B have a high probability of doing their own investigations instead of asking each other.

6.6.2

Communication lines

Now we have seen an example of the definition of two agents, A and B. But as we know, this is not all there is to our model. We also have to define the properties controlling the interaction between the two agents. This is also defined in XML and below we see the communication line from agent A to agent B and the communication line from agent B to agent A. <Network> <CommunicationLine Source="A" Destination="B" Trust="0.4" TruthProbability="0.4" /> <CommunicationLine Source="B" Destination="A" Trust="0.4" TruthProbability="0.4"

88

/> </Network>

6.6.2.1

Trust

The communication line from agent A to agent B has a Trust-property of 0.4. The same holds the other way, from agent B to agent A. This means that they both trust each other to a degree of 0.4. The agents update their belief as a function of the degree of trust. The degree of trust functions as the weight, the agent assigns to his belief when he updates it. This means that beliefs from trusted sources get a higher weight than beliefs from sources with low trust.

6.6.2.2

TruthProbability

The communication line from agent A to agent B has a TruthProbabilityproperty of 0.4. The same holds the other way, from agent B to agent A. The TruthProbability is the probability that the source will tell the truth (or what the source believes to be the truth) to its destination. For both these communication lines there is a 40% chance that they will tell each other what they believe to be the truth. If they lie, they will just answer randomly.

89

Chapter 7 Designing a logic 7.1

Introduction

When building a game like Knowledge Game, we must have an unambiguous way of specifying the statements, the players are trying to determine the truth value of. Some sort of formal logic will be an obvious choice for this. A formal logic will allow us to easily specify game statements and easily have the agents calculate whether they have enough information to calculate the answer, and in case they do have enough information, then actually calculate the answer. Furthermore, the game will be able to check whether an agentâ&#x20AC;&#x2122;s answer is right or wrong. All this can be done using model checkers and/or automatic proof builders.

Since we have multiple players that are searching for knowledge and who have to update their current beliefs, a tempting choice may be a dynamic multi-agent epistemic logic. However, as we hinted to above, the logic we are using is a pretty standard monadic predicate logic. We will see why in the following.

In this first version of Knowledge Game we can make do with agents who can neither reason about beliefs nor other agents.1 We will get into a lot more details later on, but for now, let us just remember that our agents must be able to reason about entities in a world that is mostly unknown to them.

For example, let us say that an agent knows that there is an entity a and that a is green, but the shape is currently unknown to the agent. Furthermore, 1

Our agents know what they do not know, as described in Chapter 5.3.7, that is the ability that comes closest to reasoning about beliefs. They cannot reason like this about beliefs and agents: I know that Agent B knows that I know that p.

90

let us say that the game statement is to decide whether there exists a green triangle. Since the color of a is in fact green, knowing the shape of a may actually be what it takes to determine the truth value of the game statement.

On the other hand, there is no reason the agent should try to find out what size a is, since that is completely irrelevant to the game statement. So first of all we will need a way for the agents to know what information to seek. They can acquire new information by either making investigations (which can be understood as asking the world) or they can make social inquiries by talking to other agents. To ask the other agents, obviously an agent will need to know (or at least have beliefs about) what other agents are around. However, he does not need to be able to reason about the other agents’ beliefs. Let us say that agent A needs to find the shape of a and he has to decide whether he will ask agent B. If agent A could reason about B’s beliefs, he might had been able to figure out that there was no reason to ask B, because B had no chance to know.2 .

In Knowledge Game, the game statement will not be a statement about the agents, but about objects in the world which themselves have no beliefs.

Of course it would be handy if the agents had a way to figure out whether another agent is worth asking. Implementing this using multi-agent epistemic logic would complicate things, though, and it might not completely fit the purpose. A multi-agent epistemic logic would require the agents to have beliefs about what the other agents have been up to, what they knew to begin with and so on. If these beliefs were not justified, the reasoning would just be random. Our agents should not necessarily know what the other agents have been doing. Instead, they should have beliefs about whether the other agents are trustworthy or not. Since they do not have to really reason about trust, it does not have to be part of the logic. By ‘not being part of the logic’ we simply mean that the logical language cannot express anything about trust. Trust will be handled by the surrounding machinery3 . Actually not having trust part of the logic will make some parts of the game mechanics more flexible, as we can change the mechanics without having to change the 2

Depending on the game statement and the structure of communication lines between the agents, A might even be able to calculate the truth value of the game statement by reasoning about B’s knowledge. This is the case in the Muddy Children Problem where the game statement is concerning the agents/children themselves rather than objects in the world. The muddy children problem is explained in Chapter B on page 182 in the appendix.

3 This is explained in detail in Chapter 6.

91

logic. In other words, we can do without multi-agent and epistemic logic.

What can we not do without, then? The agents need to be able to reason about the entities in the world. They need to have beliefs that there are entities and that these entities have certain properties. The logic we choose must be able to express things like ‘there is a red circle, but there are no green circles’. To keep things simple, we will not have expressions like ‘there are less than three blue squares’ or entity x is the same entity as entity y’.

In other words, we can do without being able to count and without having identity.4 Now, what we have expressed need of in the above is almost exactly a classical monadic predicate logic. The only place where we diverge a bit is when the agents need to figure out what they do not know and what they need to know. We need some way to encapsulate the fact that an expression can, at the moment, be neither true nor false, as its truth value is currently unknown. Luckily, this does not have to be part of the logic either – the logical language needs only be concerned with the boolean values ‘true’ and ‘false’, it needs not have power to express that the value of an expression is ‘unknown’. Instead ‘unknown’ can be implemented as part of the mechanisms surrounding the logical language5 . This means we can use a pretty standard monadic predicate logic. To make the expressions easy to type on a standard computer keyboard, we will be avoiding symbols like ∀ and ∃ (and instead use letters like A and E). The semantics, however, are completely standard.

7.2

Decidability

Before continuing, we should make a single remark on decidability. Formal systems of a certain complexity have been proved to be undecidable, which means that they contain sentences that are true, but which the systems cannot compute the truth value for. Choosing a logic that is not decidable for a system like Knowledge Game, may yield all sorts of difficulties. For example, what happens if the game statement is one of the sentences which the system cannot decide the truth value of? Furthermore, an undecidable logic 4

Adding identity to our logic would also allow us to express numerical statements like ‘there are exactly two blue pentagons’. In this first version however, we can do with simple statements like ‘there is a red circle, but there are no green circles’. More advanced numerical statements will be left for further work.

5 This will be explained in detail in Chapter 9.

92

would expose a potential difference between the computer agents and real life agents: it has been argued that while formal systems of a certain complexity always contain a true sentence which they cannot determine is true, human beings have no such limit. For a discussion of the philosophical implications of this potential difference between computers and humans see [Luc63]. For related discussions see for example [Sea90], [Boj97], [Cha], [Dum78], [Gai], [OD03], [Pen56], [Göd57] and [Smu46].

Luckily, even though not all predicate logics are decidable, the one we choose – monadic predicate logic – is indeed decidable, so we can continue without worrying further about this.

7.3

Expression trees

To the computer, a string containing an expression in monadic predicate logic is just like any other string. The computer has no knowledge or understanding of the contents of the string. It does not even know about the structure of the expression – to the computer it is just random characters in a row. To be able to check the truth of an expression, the computer must at the very least have a structured representation of the expression. To better understand this, we will see a simple mathematical example in the following section.

7.3.1

2+3*4

Let us imagine we are trying to build some software that reads input from the user. The user simply types the input on the keyboard. If the input is a simple mathematical expression like 2 + 2, the computer should calculate the result. If the input is not a simple mathematical expression, the computer should just ask the user to enter something else. For now, we will just assume the input is in fact a sentence in the language, so we should start calculating the result. But how? How are we supposed to make the computer ‘understand’ the contents of the input enough to actually calculate the result?

Where should it start? A simple example like 2 + 2 might seem easy enough, one could just start from the left, but how about 2 + 3 * 4? If we were just starting from the left, reading one character at the time we, would get the result 20: first we would add 2 and 3, which gives us 5, and then we would multiply by 4, yielding a result of 20. Unfortunately this is the wrong 93

result, because the calculation does not take the structure of the sentence into account. The structure of the sentence – including the implicit priority of the operators – specifies that 3 * 4 must be the first step of the calculation and adding 2 must be the second step. In other words, the correct result is 14.

Now, how can we represent this implicit structure? An expression tree will do the trick. Expression trees (and trees in general in computer science) by convention grow downwards. They consist of nodes connected by branches.

A node can have one or more nodes below it called child nodes. A node with children is called the children’s parent node. An example, showing the expression tree for the expression 2 + 3 * 4, can be seen in Figure 7.1. The idea is that instead of just starting from the first character of the string, we start from the top of the tree: the top node of the tree is the addition-step.

This step tells us to add the left and right branches of the tree. The left branch is just the number 2, but the right branch is actually a mathematical expression in itself, namely 3 ∗ 4. So before we carry out the addition, we need to calculate the value of the right branch that is multiplying 3 and 4. + ∗

2 3

4

Figure 7.1: Correct tree for 2 + 3 ∗ 4.

After having done this, we have a tree like the one in Figure 7.2. Now the remaining part is easy, we can just add 2 and 12 and we get the correct result, 14.

+ 2

12

Figure 7.2: The remaining tree after carrying out the first step of calculating 2 + 3 ∗ 4.

Now we see how the expression tree is a way to represent the expression in a structured way that makes it possible to calculate the right result. The 94

challenge ahead of us is to find a way to convert the original input string to the structured data that the expression tree is. This is what a parser can do for us. The parser will be built on top of a grammar . A grammar is a way to describe the language in a recursive and unambiguous way. Before looking at the actual parser, we will have a more general look at grammars.

7.4

Context-free grammars

The syntax of a formal language can be described in a recursive and unambiguous way using a formal grammar. There are several ways to write grammars, one of them being the Backus-Naur Form (BNF). Grammars in general, and BNFs in particular, have several advantages. First of all, they are relatively easy to read and write and secondly, they aid when writing a parser for the language. We will dive into an example shortly, but for starters let us have a few general words on formal grammars.

A formal grammar can be used to help us both parse a string by creating an expression tree and more simply just checking if the string is a sentence in our language. Formal grammars consist of rules called production rules and the grammar is used by applying those rules on an input string recursively.

Each recursive step will get us a bit closer to knowing if the string is a sentence in the language and, if needed, an actual expression tree. The rules are built using terminals and non-terminals and the goal of applying the rules is to convert all non-terminals to terminals. There are different types of formal grammars generating different types of languages, some more general than others. The grammars, we are going to use, are called context-free grammars (CFG). All rules in a context-free grammar convert from non-terminals6 . But some convert to terminals and others to non-terminals. Some even convert to a mixture of terminals and non-terminals. When everything has been 6

A grammar is context-free if and only if all rules have the form A â&#x2020;&#x2019; w where A is a non-terminal and w is a string of terminals and/or non-terminals. There are also context-sensitive grammars which generate context-sensitive languages. They have the form wAx â&#x2020;&#x2019; wyx where A is a non-terminal, w and x are strings of terminals and/or nonterminals, and y is a string of terminals. In other words, whether a terminal is appropriate in a certain place cannot depend on the context in the context-free grammars (as the lefthand side of the rules contains only non-terminal). In the context-sensitive grammars, it can depend on the context (as the left-hand side can contain terminals). For example, the context-sensitive language an bn cn (where n is a positive integer) cannot be expressed using a context-free grammar. See for example [Wik12b] or [Wik11].

95

converted to terminals, you are done and you can conclude that the string is a sentence in the language. If you are stuck and there are no more rules to apply, you can conclude that the string is not a sentence in the language.

To make things a bit more clear, let us have a look at a simple production rule. This is not a complete grammar (we will get to that!) but just one of several rules making up the grammar: <Expressions> ::= <Expressions><Expression> | <Expression>

Each rule consists of a left-hand side and a right-hand side divided by ::=. On the left, you have the non-terminal you are converting from and on the right, you have the terminals and/or non-terminals the rule can convert to. Non-terminals are written with brackets, for example <Expression>, so this is a rule that converts to non-terminals only (no terminals). Sometimes, there are several possible outcomes. In those cases, each possible outcome is separated by |. You can think of ::= as meaning ‘can be converted to’ and | as ‘or’. In other words, the rule above can be read as: Convert an <Expressions> to either both an <Expressions> and an <Expression> or to just an <Expression>.

You can only apply a rule if the left-hand side of the rule matches the type of the current input. The grammar defines a start symbol. The grammar considers the input string as being of the type defined by the start symbol.

The first possible outcome of the rule, <Expressions><Expressions> requires that the input is split in two. The first part must be a non-terminal of type <Expressions> and the second part must be of type <Expression>.

If you have a long input string, there are obviously many ways to split it, for example, the string abc could be split in many ways: (a, b, c), (ab, c) or (a, bc). We will leave out the details on how to actually do this split, but we will include the results of the split where necessary.

Eventually, we will use a grammar to help us build the expression trees for the expressions in our predicate logic. For now, we will just have a look at a couple of simple languages and the grammar will be used only to check whether an input string is part of the language or not.

7.4.1

The language of ab. . .

One very simple language is the one consisting of only the symbols a and b, and in which a string is a sentence if and only if it has exactly the same 96

number of as and bs, and each b is preceded by an a. One sentence would be ab as it has the same number of as and bs, and each b is preceded by an a. On the other hand, aabb is not a sentence of the language. Even though it contains the same number of as and bs, the second b is not preceded by an a. This language can defined using BNF as seen in figure 7.3 1

"Start Symbol" = <Expressions>

2 3

<Expressions>

::= <Expressions><Expression> | <Expression>

<Expression>

::= â&#x20AC;&#x2122;abâ&#x20AC;&#x2122;

4 5 6

Figure 7.3: Grammar for the language of ab.

Figure 7.3 shows a simple grammar. Let us go through it line by line. Line 1 defines what type of input we start with. In other words, when we start applying the rules, the input string is considered an <Expressions> token.

Line 2 is empty and has no effect â&#x20AC;&#x201C; it only serves to make the grammar easier to read. Line 3 and 4 constitute the first rule of the grammar. This rule is exactly the same rule we saw on page 96, only this time we wrote the second disjunct of the right-hand part of the rule on the next line. This split has no effect and is only done to make the grammar easier on the eyes. Line number 5 is empty and has no effect. Line 6 is the second rule of the grammar. It is the first rule that we have seen that actually converts to a non-terminal. Note that the non-terminal is not surrounded by brackets but by single quotes.

7.4.1.1

Applying the rules

An example of how we apply the rules will make things more clear. Let us try checking the string ab. First, as noted above, because of line 1, the input string (in this case ab) is considered a non-terminal of the type <Expressions>. Starting from the top, we have to look for rules that can convert this type of non-terminal. Luckily that is exactly what the rule in lines 3 and 4 do, so let us try applying that. The rule converts the <Expressions> non-terminal to either both an <Expressions> and an <Expression> (the first disjunct) or just an <Expression> (the second disjunct).

97

We will try the first disjunct first. To do this, we would have to split the string ab into two non-terminals, the first being of type <Expressions> and the right being of the type <Expression>. When we first saw the example rule above, we noted how splitting the input string is something we will not go into details with. Instead, we will just assume that the input string was already split in two: (a, b). A single non-terminal can consist of one or more tokens. This means that in case the input had been split into three or more tokens, there would be several ways to combine the token. These would have to be tried one by one. In our case however, there are only two tokens, namely a and b, so we will let the a be the <Expressions> and the b be the <Expression>. Now we have to apply rules to both of these non-terminals one by one. We can try the <Expressions> first. Trying to apply the rule in lines 3 and 4 again, this time to a, we will see that the first disjunct will not work as we cannot split the a further. So we will try the second disjunct. This just converts the a to an <Expression> which we still have to apply rules to.

The only rule working on an <Expression> is the one in line 6, but it only works if the <Expression> can be converted to ab. Our <Expression> only contains a, hence it will not work.

When we originally tried to apply the rule in lines 3 and 4, we decided to try the first disjunct first. This step is what got us into a dead end. Now let us instead try the second disjunct which just converts the entire input, consisting of ab, to an <Expression>. We then need to apply a rule to this and again the only rule that works on an <Expression>, is the one in line 6, so we will try that one. Now, this time it works, as our input is now exactly ab. This means we are done â&#x20AC;&#x201C; we have converted everything to terminals.

So what did we just do? We started by trying to apply the first disjunct of the rule in lines 3 and 4 which did not succeed. Then we tried applying the second disjunct, thereby converting our start symbol <Expressions> to the non-terminal <Expression> which could in turn be converted using line 6 to the string ab and then we were done. The fact that we are done means that the input string was a sentence in the language. It also means that we could build an expression tree for the string. We will save the tree building part for a little later, though.

98

7.4.1.2

Example

Let us look at one more example. This time we will try a string that is not a part of the language: abc. First we will try matching the left disjunct of the rule in lines 3 and 4. Again we will not explain how to split the string, but we will assume that process yields three tokens: (a, b, c).

We can assign these to the resulting non-terminals in several ways, but none of them will work. As an example, we will just try one of the assignments, namely (a, b) for <Expressions> and c for <Expression>. Applying the rules to the <Expressions> non-terminal is exactly like the previous example, so we know that this will work â&#x20AC;&#x201C; we will convert to <Expression> using the second disjunct of lines 3 and 4 and then to the string ab using line 7. However, converting our <Expression> containing the string c will not work, as the only rule converting from an <Expression> is line 6 which only works if the contained string is ab and we just have c. So there is no way we can apply rules until we are done, hence the string is not part of the language.

7.4.2

Adding parentheses

Before we are ready to construct more advanced grammars, let us just have a quick look at an extension of the first example grammar. We would like to construct a language that is a lot like the previous one (figure 7.3) with the addition of parentheses. So instead of only accepting strings like ab and abab, we would like to accept strings like (ab), (abab)(ab) and ((ab)(ab))ab.

We will not accept strings with non-matching parentheses, i.e., (ab or (ab)).

This can be done by just adding one line to the grammar, as seen in figure 7.4.

7.5

Building a tree

Now that we have seen how the basics of simple grammars work, we are ready to see how we can build a tree using a simple grammar. Before building a tree, we will find the sequence of rules, we need to apply to parse the entire string. This is done like we just saw above. Then when we have the actual sequence â&#x20AC;&#x201C; which is in fact a tree itself called a parse tree â&#x20AC;&#x201C; we can start building the expression tree.

99

1 2

"Case Sensitive" = True "Start Symbol" = <Expressions>

3 4

<Expressions>

::= <Expressions><Expression> | <Expression>

<Expression>

::= ’ab’ | ’(’ <Expressions> ’)’

5 6 7 8

Figure 7.4: Grammar for the language of ab and parentheses.

The grammar, we will use for our first example, describes a very simple mathematical language. The only thing it can do is multiply numbers. The grammar is seen in figure 7.5 1

"Start Symbol" = <MultiplyExpr>

2 3 4

<MultiplyExpr>

::= <MultiplyExpr> ’*’ Number | Number

Figure 7.5: Grammar for the language for multiplying numbers.

We can now use the grammar defined in figure 7.5 when parsing a string like 2 * 3. We will build the tree by adding one node at a time. Each time we apply a rule, we add a new node and each remaining terminal and non-terminal will add branches. Let us try building a tree for 2 * 3. First we need to find the sequence of rules to apply. There is only one rule, so it is pretty simple to decide which rule to try. The rule has two parts, the first and second disjunct. The first one applies. In the next step, the second disjunct applies. Both steps can be seen in figure 7.1.

Now we know the steps to parse the input string. The steps are as follows: 1. Apply first disjunct of the rule to 2 * 3.

2. Apply second disjunct of the rule to 3.

Now let us build the tree. In the first step, we create the top node. There is one terminal and one non-terminal in the right-hand side of the part of the 100

2 <MultiplyExpr>

* ’*’

3 number

2 number Table 7.1: Applying rules to 2 * 3. rule we are applying. This means we will add two children to the top node, one for the non-terminal and one for the terminal. Since the non-terminal is exactly that – a non-terminal – we still do not know all the details about it, hence the details of the second child of the top node are still unknown. The partial tree we have created after step one is shown in figure 7.6.

∗ 3

?

Figure 7.6: The partial tree after performing the first step.

Now we can perform the second step which just creates a node for the number 2 and adds it where the question mark is in figure 7.6. This gives us the tree shown in figure 7.7.

∗ 2

3

Figure 7.7: The complete tree for 2 * 3 after performing the second step.

Now that was not too hard. We have created the expression tree for a mathematical expression and the result can now easily be calculated. In the next section, we will look at an example that is a bit more advanced.

101

7.6

Getting the priorities right

Now let us turn back to the example expression in section 7.3.1 on page 93.

The example was used to show how an expression tree is helpful to make sure we get the correct result when calculating 2 + 3 * 4. We saw how two interpretations of the expression are possible and only one of them is correct.

The correct one is essentially 2 + (3 * 4) and the wrong one is (2 + 3) * 4. In other words, the expression contained a set of implicit parentheses.

Those implicit parentheses can actually be worked into the grammar, so you do not have to worry about them later on.

The implicit parentheses mentioned above arise from the effects of operator precedence. Operator precedence means that some operators have priorities over others. We are used to this from simple calculations like 2 + 3 * 4 – we know that we have to multiply before adding, unless explicit parentheses tell us otherwise.

To see how operator precedence works, let us add the addition operator to our simple grammar from figure 7.5. We can do this in several ways. We will see two examples, one where multiplication is given higher priority than addition and one the other way around.

Before looking at the actual grammars let us look at the resulting trees.

They are shown in figure 7.8. The left is formed by giving addition priority over multiplication and the right one is formed by giving multiplication priority over addition. ∗ + 2

+ 4

∗

2 3

3

4

Figure 7.8: Two possible trees for 2 + 3 ∗ 4.

Notice how the trees differ. Remember that the result of a node can only be calculated if the results of the children have already been calculated. This means that though we start from the top, we must calculate the bottom nodes first. This is used to control the priority of the operators. The tree to the left makes sure that addition is done first by having it in the bottom of the tree. The tree to the right has multiplication in the bottom and therefore calculates multiplication first. We can think of this as the multiplication 102

operator binding stronger than the addition operator in the tree to the right and vice versa in the tree to the left. By writing our grammar correctly, we can make sure that our trees turn out like the tree to the right. Let us have a look at that grammar, it is shown in figure 7.9.

1 2 3

"Start Symbol" = <AddExpr> <AddExpr> ::= <AddExpr> ’+’ <MultiplyExpr> | <MultiplyExpr>

4 5

<MultiplyExpr>

::= <MultiplyExpr> ’*’ Number | Number

6

Figure 7.9: Correct grammar for the language for adding and multiplying numbers: multiplication has priority over addition.

Comparing the grammar in 7.9 to the original grammar which only supported multiplying numbers, we see that two lines have been added. These lines define rules for the non-terminal <AddExpr>. We have also changed the start symbol to <AddExpr>. Remember that we wanted to make the multiplication operator bind stronger than the addition operator. We do this by putting the multiplication operator in the bottom of the grammar. Thereby we make sure that the multiplication operator is always placed in the bottom of the tree. Since any given node, including the top node, can be calculated only after the children have been calculated, we are thereby making sure that the multiplication will be done as the first step. <AddExpr> <AddExpr>

+

<MultiplyExpr>

<MultiplyExpr>

<MultiplyExpr>

2

3

∗

4

Figure 7.10: Parse tree for 2 + 3 * 4.

Figure 7.10 shows the parse tree for the expression 2 + 3 * 4 using the grammar in 7.9. The parse tree expresses the applications of the rules neces103

sary to parse the string. The start symbol is <AddExpr> which is converted to an <AddExpr>, an addition symbol and a <MultiplyExpr>. The resulting <AddExpr> is converted to a <MultiplyExp> and then the number 2. The <MultiplyExpr>, resulting from the first step, is converted to another <MultiplyExpr>, a multiplication symbol and the number 4. At last the resulting <MultiplyExpr> is converted to the number 3.

For completeness we will just have a quick look at the grammar producing the tree to the left in Figure 7.8. This actually produces an incorrect tree, but we will show it to point out how it differs from the correct grammar. It can bee seen in Figure 7.11.

1

"Start Symbol" = <MultiplyExpr>

2 3

<MultiplyExpr>

::= <MultiplyExpr> ’*’ <AddExpr> | <AddExpr>

<AddExpr>

::= <AddExpr> ’+’ Number | Number

4 5 6 7

Figure 7.11: Incorrect grammar for the language for adding and multiplying numbers: addition has priority over multiplication.

The conceptual difference between the two grammars in Figure 7.9 and Figure 7.11 is the order of the rules. The change is accomplished by simply swapping the names of the non-terminals and the operator symbols * and +.

Now that the first rule is the one for the multiplication operator, we make sure that multiplication always gets in the top of the tree which means that addition gets in the bottom and gets calculated first.

What we have just seen is all there is to operator precedence and priority.

We will use this, when writing the actual grammar for our predicate logic.

However, we have not yet seen all there is to implicit parentheses. We have seen how the implicit parentheses work when forming trees with different operators, but they also play a role when combining multiple expressions using the same operator. We will get to that in the next section.

104

7.7

Operator associativity

In section 7.6, we saw how implicit parentheses work for operators of different precedence. However, that was only part of the story of implicit parentheses.

We need to define what to do with expressions that combine operators of the same precedence. Let us take the expression 1 + 2 + 3. How should we interpret this? There are to possibilities (1 + 2) + 3 and 1 + (2 + 3).

The property determining the position of the parentheses is called associativity. In the first case, the addition operator is left associative because the expression is grouped from the left. The latter is right associative. For this particular example, the expression yields the same result, no matter what associativity we choose for the operator. This is because of the associative laws of the addition operator. In other cases, the result depends on the implicit interpretations. Usually addition and subtraction are given the same priority which means that implicit parentheses for expressions containing only addition and subtraction must come from the associativity of the operators. An expression like 1 - 2 + 3 yields different results depending on the implicit parentheses: (1 - 2) + 3 = 2 and 1 - (2 + 3) = -4. The difference can also be seen from the expression trees of the two different interpretations as shown in Figure 7.12. This must be taken into account when writing a grammar. Usually both multiplication, division, addition and subtraction are left associative. â&#x2C6;&#x2019;

+ â&#x2C6;&#x2019; 1

3

+

1

2

2

3

Figure 7.12: Two possible trees for 1 â&#x2C6;&#x2019; 2 + 3.

We are not going to write an example showing all four operators, but we will show how the multiplication operator of the simple grammar in figure 7.5 on page 100 is left associative. The left disjunct of the rule in lines 3 and 4 can convert a <MultiplyExpr> to a <MultiplyExpr>, the string * and a Number.

The thing to notice here is that the Number is to the right of the *. This ensures that we are always parsing the right-most element of the expression first, thereby placing it in the top of the tree. Conversely, this means that the left-most element will always be parsed last, thereby placing it in the 105

bottom of the tree and making sure the results of the left-most elements are calculated first. The effects of this can also be seen in Figure 7.6 on page 101 where only the right-most element of the expression is parsed.

106

Chapter 8 Our monadic predicate logic 8.1

Syntax

Now that we have seen a few example grammars, we are ready for the real thing. Figure 8.1 shows the grammar for our monadic predicate logic. We will look at it in more detail later, but for now we should just notice a couple of things and then run trough a simple example. First let us notice that there are rules supporting each of the connectives, quantifiers and parentheses – for example in line 20, there is a rule converting a non-terminal (do not worry about the specific non-terminal for now) to two other non-terminals surrounding the string |. This is how the grammar recognizes the or-symbol of the language.

8.1.1

Example

A basic sentence in monadic predicate logic could be something like this: (Ax)Fx This says that for all x, x is F. For our purpose it makes sense with some longer predicate names for making the expressions easier to read. For example, if the predicate F really means ‘is red’, then another version of the sentence would be this: (Ax)Red(x) By making the interpretation of the predicates implicit (the predicate Red(x) picks out the same property that the natural language predicate ‘ 107

1 2 3 4 5 6

"Name" = ’ExpressionParser’ "Author" = ’Soren Lind Kristiansen and Alette Holmberg-Nielsen’ "Version" = ’1.0’ "About" = ’Predicate Logic’ "Case Sensitive" = True "Start Symbol" = <Expression>

7 8 9

{Lowercase} = [abcdefghijklmnopqrstuvwxyz] {Uppercase} = [BCDFGHIJKLMNOPQRSTUVWXYZ]

10 11 12 13

Variable = {Lowercase}+ Predicate = {Uppercase}{Lowercase}+ Sentence = {Uppercase}+

14 15

<Expression>

::=

<ToExpr>

<ToExpr>

::= <OrExpr> ’->’ <ToExpr> | <OrExpr>

<OrExpr>

::= <OrExpr> ’|’ <AndExpr> | <AndExpr>

<AndExpr>

::= <AndExpr> ’&’ <Quantifier> | <Quantifier>

<Quantifier>

::= ’(E’ Variable ’)’ <NegateExpr> | ’(A’ Variable ’)’ <NegateExpr> | <NegateExpr>

<NegateExpr>

::= ’-’ <Quantifier> | <Predicate>

<Predicate>

::= | | |

16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36

Predicate ’(’ Variable ’,’ Variable ’)’ Predicate ’(’ Variable ’)’ Sentence ’(’ <Expression> ’)’

Figure 8.1: Grammar defining our predicate logic used in Knowledge Game.

108

is redâ&#x20AC;&#x2122; picks out) we can make the sentences in our logic a lot easier to read.

Therefore our language only accepts predicate names with at least two letters and it requires parentheses after the predicate.

Let us try parsing the sentence (Ax)Red(x) using the grammar. As usual we will start from the top. Our start symbol is the non-terminal <Expression>. There is only one rule matching this, namely line 15, so let us just apply this and convert to a <ToExpr>. This can be used with the rule in line 17 and 18. We can see already that line 17 will not work, as our input string does not contain a conditional (the string ->). So we apply the rule from line 18 and we get an <OrExpr>. Continuing like this, we â&#x20AC;&#x2DC;drain downâ&#x20AC;&#x2122; the rules till we reach the rule for the quantifiers in line 26, 27 or 28. Line 26 will not work, as we do not have an existential quantifier, so instead we will try line 27. To make this work, we need to have a Variable between the leftmost set of parentheses. As seen in line 11, a Variable is just at least one lowercase letter and since what we have is just the letter x, we have a match.

We are not done yet though. The rule in line 27 requires us to also match a <NegateExpr> with the string Red(x). The <NegateExpr> can be converted using line 30 to a <Predicate> which in turn can be converted using line 34 to Predicate consisting of an uppercase letter and at least one lowercase letter and a <Variable> inside a pair of parentheses. This perfectly matches the string Red(x) which means we are done and can conclude that the string (Ax)Red(x) is indeed a sentence in our language.

8.1.2

Syntax

Now that we have decided on a decidable logic, let us have a look at the syntax. As mentioned above, it is pretty much standard for monadic predicate logic but with a few changes for easier typing. The language consists of the following symbols: Parentheses: (, ).

Connectives: & (and), | (or), -> (conditional), - (not) Variables and constants: a . . . z Propositions: A . . . Z Quantifiers: A (universal), E (existential) 109

Predicates: Consisting of at least two letters, the first letter must be upper case, all others lower case.

The attentive reader may have noticed that we use the same symbols for variables and constants. This may at first seem a bit odd as it appears we cannot distinguish the two sentences (Aconstant )Red(constant ) and (Avariable )Red(variable ). However, since we are designing this logic specifically for Knowledge Game we can assume that the names of all constants are given before parsing a sentence. Thus, when the parser meets a symbol that can be either a variable or a constant, it will check whether a constant with that name exists. If it does, the parser will assume that the symbol is a constant. If it does not, the parser will assume it is a variable.

The reason for doing it this way is that it enables us to make the grammar a bit simpler.

110

Chapter 9 Model checker 9.1

Introduction

In chapters 7 and 8, we saw how to design a logic for the purpose of our simulations. In this chapter, we will see how that logic actually comes to use. The reason, we needed a logic to begin with, was so we could easily specify game statements and have our agents calculate the answers. We have already seen how the logic can be used to specify the game statements, but we have not yet seen how it can be used to calculate the truth values of the game statements.

9.2

The task at hand

The task we need to solve is this: Given a model consisting of entities with certain properties, we must check whether a specified expression is true for the model. This is exactly what a model checker can do. If for example the specified expression states that the entity a is red (Red(a)), a model checker can look at a and check whether the predicate Red is true for that entity.

9.2.1

Known unknowns

A model checker will be useful for checking an expression against the actual game world where all properties of all entities are known. However, when an agent must decide whether he thinks the value of the game statement is true or false, he does not always have access to all information about the entities in

111

the world. In some cases, the agent should be able to come up with an answer even if he does not have full information and in others, he should come to the conclusion that more information is needed. For example, if he does not know the shape of a, he could still answer the question above as long as he knows the color of a. Had he known the shape, but not the color, he should come to the conclusion that he needs to find out the color of a before he can answer.

None of these two abilities are part of a standard model checker. However, we can create a standard model checker and extend it a bit to achieve the desired effect. In other words, we need to build a basic model checker with two additions: support for returning a value representing ‘unknown’ and support for calculating what relevant information might be missing. ‘Unknown’, in the sense of being a value returned by the model checker, will be written unknown to indicate that it is a specific value representing that the answer is at the moment not known to the agent.

9.2.2

Tautologies, valid expressions and contradictions

The specified expression can be either a contingent expression which can be either true or false depending on the contents of the model, a tautology or valid expression which are always true no matter the model, or a contradiction which is always false. Tautologies, valid expressions and contradictions are special in the sense that their truth value can be calculated without looking at the model. For example the expression (Ax)Red(x) | -(Ax)Red(x) is a tautology. No matter what entities are in the world, either all of them are red or not all of them are red1 . However, since a model checker does not ‘know’ this, it will need at least one non-red entity or information on all entities being red before it can determine that the tautology is true. Thus, our agents cannot determine the value of an expression like the tautology above without looking at the model. This could be dealt with by implementing a satisfiability solver which would be able to identify tautologies and valid expressions. This is discussed in Chapter 20 in Part IV Further work.

1

Note that ‘not all of them are red’ is different from ‘all of them are not red’. The latter would be written (Ax)-Red(x) and had this been the second disjunct, the disjunction would no longer have been a tautology.

112

9.3

Top-down

There are different approaches when building a model checker. In general, model checkers can be either top-down or bottom-up. A top-down approach starts checking the top-node and works its way down, like we did when calculating the results of the mathematical expressions in section 7.3.1 on page 93. The bottom-up approach, on the other hand, starts from the bottom and works its way to the top. We use the top-down approach because we find it intuitive and relatively simple to implement.

9.3.1

Example

Before we get into the details, let us have a look at an example. We will specify a simple world and check an expression against it. The world can be seen in figure 9.1. The expression we want to check against it is this: (Ax)Red(x) | (Ex)Small(x) In other words, we want to check whether it is true that all entities in the world are red or at least one entity is small.

XS

L

a

b M

S

c

d World

Figure 9.1: A world.

The first thing we need to do is to build an expression tree for the expression. Luckily, we already have our grammar and parser for this, so we will just assume that this has been done. The resulting expression tree can be seen in Figure 9.2.

The top node of the tree is an â&#x20AC;&#x2DC;orâ&#x20AC;&#x2122; operator. We know from the truth table for disjunction that it is true if at least one of the disjuncts are true.

113

|

A x

E x

Red x

Small x

Figure 9.2: The expression tree for (Ax)Red(x) | (Ex)Small(x).

Hence, we need to check at least one of the children. If the first one we check is true, then the entire expression is true. If it is false, then we have to check the other child. Let us start with the left disjunct. The top-node of the left disjunct is a universal quantifier. This binds the variable x (the left child node of the universal quantifier) and states that x must be red.

In other words, we need to run through all the entities in the world and check whether they are red. If we find just one that is not red, we know the universal quantifier is false. If we do not find any non-red ones (that includes not finding any entities at all), the universal quantifier is true. Let us try actually doing this. The first element of the world is a red triangle. Since this is red, we need to check the next entity which is a yellow pentagon. The pentagon is not red, so we have found a counterexample and we now know that the universal quantifier is false. Going back up the tree, we now know that the left disjunct is false, so we have to check the right disjunct. This one is an existential quantifier. The existential quantifier binds the variable x and then the right child states that x is small. In other words, there must be at least one small entity for the existential quantifier to be true. So we have to go through all the entities till we find a small entity. If we find one, the right disjunct is true, otherwise it is false. We start by checking the red triangle. This is extra small, so it will not satisfy the condition. We then try each of the other entities until we reach the blue circle. This is indeed small which means the existential quantifier is true, hence the right disjunct is true and therefore the entire expression is true. That is basically how our model checker works. The example does not show what to do when we encounter 114

unknowns and specifically it does not show how to calculate what relevant information is missing. We will get to that in the coming sections.

9.4

Semi-trivalent system

When an agent tries to calculate the truth value of a game statement, he will not always have enough information to come up with an answer. In those cases, the agent must be able to return a value indicating that the answer is unknown. One way to do this would be by using a trivalent concept of truth.

However, our logic only implements standard boolean values, true and false.

We could of course choose another logic and make sure it is one supporting both true, false and unknown. This way the agents would be able to reason about what is unknown, as well as what is true and false. However, what we need to realize is that our agents do not need to reason about unknowns â&#x20AC;&#x201C; they just need to be able to return the value unknown. This makes a big difference, as we do not have to change the logic at all to achieve this. We just have to extend the model checker a little. Whenever an agent is not certain enough of the truth of a predicate for a given entity to actually assert it, he will instead just communicate a value indicating that the truth of the predicate is unknown. How the internals of the agents work, including how certain the agent must be to communicate, is discussed in Chapter 6. In this section, we will assume that the agent has the internal mechanism to decide that the value of a specific predicate for a specific entity is unknown. We will then focus on how this value is propagated through the model checker.

We need to solve two problems. First of all, we must be able to correctly calculate the value of an expression when one or more predicates are unknown. The result must be either true, false or unknown. The result must be unknown if and only if the answer is in fact unknown. For example, if the game statement is a disjunction and the first disjunct is unknown and the second disjunct is true, then the result should be true, since the value of the first disjunct does not matter when the second one is true. On the other hand, had the second disjunct been false, we would need to calculate the first disjunct to know the truth of the expression. Secondly, in those cases where the truth value of the game statement is unknown, we must be able to calculate what more information is needed to calculate the answer.

115

9.4.1

Propagating unknowns

Let us first see how we can make sure that unknowns are propagated correctly through the model checker. We will look at an example shortly, but first we have to find out in which cases we should propagate the unknowns when performing the check using our model checker. We can write a standard truth table for each of the connectives. This can be extended to include unknown. Having done that, we know when to propagate unknown and when we can actually give one of the answers true or false. Such an extended truth table can be seen in Table 9.1. The value ‘U’ in the table means unknown. Note that three of the Us in line number 9 are marked with a ∗.

This is because they can potentially contain tautologies, valid expressions or contradictions which the model checker cannot detect. They are discussed in section 9.4.1.3.

No. 1 2 3 4 5 6 7 8 9

p T T T F F F U U U

q T F U T F U T F U

p∧q T F U F F F U F U∗

p∨q T T T T F U T U U∗

p→q T F U T T T T U U∗

¬p F F F T T T U U U

Table 9.1: Extended truth table We can group the lines of the truth table in three: one where both p and q are known, one where they are both unknown and a third one where one is known and one is unknown. We will look at these groups one by one.

9.4.1.1

Both p and p are known

This group consists of lines 1, 2, 4, and 5. In those cases all is known, so there is no doubt what the value of the connectives will be. In other words, these never give rise to the question of what value to propagate.

116

9.4.1.2

Either p is known and q is unknown or vice versa

This group consists of the lines 3, 6, 7, 8. In this group, we start to see the unknown values, so we must figure out when to propagate unknown and when we can actually calculate the truth value of the entire connective. Still, this is the easy part of the unknowns, as we can just look at what is needed to make the connectives true. For example, let us look at the ‘or’-operator for line 3. Here p is true and q is unknown. However, the value of p ∨ q is known (it is in fact true). How come? Well, to make a logical (inclusive) ‘or’ true, at least one of the disjuncts has to be true. This is satisfied by the fact that p is true, so it does not matter whether q is true or false. We can easily see this in the truth table: The only other lines than line 3, where p is true, are lines 1, 2 and in both of these p ∨ q is true. Likewise for the ‘and’-operator, if at least one of the conjuncts is false, then the entire conjunction is false. So in line 6 where p is false and q is unknown, we can conclude that p ∧ q is false.

Again, this can also be seen by noting that p ∧ q is false in both lines 4 and 5 which are the only lines apart from line 6 where p is false. In other cases we need to know the value of both p and q to calculate the value of the entire connective. For example, let us look at the ‘and’-operator in line 3. Here p is true but q is unknown. So depending on the value of q, it is both possible that p ∧ q is true and that it is false (but not both true and false at once, of course!) In other words, in this case where p is true, we need to know the value of q before we can calculate the value of the conjunction. Again, this can be seen by looking at the two other lines where p is true, namely lines 1 and 2: in line 1 the conjunction is true, but in line 2 it is false. So given that p is true, the value of the conjunction depends on whether q is true or false. This was the easy part. Now, let us look at the last group of lines.

9.4.1.3

Both p and q are unknown

This group is actually only line number 9. Here both p and q are unknown.

This might look simple enough: we may think that since both p and q are unknown, there is no chance we could know the value of the connectives.

After all, how could we for example know whether a conjunction is true if we do not know the value of at least one of the conjuncts? Well, in some cases we can. Let us imagine that p is in fact short for the expression r and q is short for ¬r. So p ∧ q is actually short for r ∧ ¬r. That is a contradiction and it is always false. Or consider a similar situation where p and q are the 117

same as in the first example, but instead of a conjunction we are looking at the disjunction p ∨ q. This is now in fact r ∨ ¬r, which is indeed a tautology – it is always true.

The examples above show us that in some cases, we can indeed know the value of a connective, even if we do not know the value of any of its parts2 . However, this is not something we can determine with just the model checker as it is, because the problem arises from the fact that the model checker could not determine the value of the parts. One idea for how to solve this is to temporarily assign known values to the unknowns: If p was just the proposition R and q was ¬R, then for the conjunction we would only get two possible outcomes: either R is true or it is false. In both cases the conjunction R ∧ ¬R is false, hence the conjunction is false. However, this is not as easy if p was ∀(x)Red(x) and q was ¬∀(x)Red(x). The conjunction is still false, but we cannot see this by just assigning the different possible truth values to a single proposition. One might argue that it is obvious that q was just a negated p, but this does not help much as we shall now see. First of all, the fact that the second conjunct is the first conjunct negated might be well ‘hidden’ by a larger formula. Secondly – and more importantly – not all cases are a matter of just having an expression and its negation. Take for example this: (∀(x)Red(x)) ∨ ∃(x)¬Red(x). It is always true, but it is not a tautology as we cannot turn it into a propositional tautology by substituting parts of it with propositions like we could in the examples above. It is called a valid expression. We will not go into the details of differences between tautologies and valid expressions, but we will note that the valid expressions can be harder to spot.

We have not extended our model checker to deal with tautologies, valid expressions and contradictions in a special way, thus we will simply propagate unknown when both p and q are unknown and let the agent gather the information about the world he needs to know the values for p and q instead (or just one of them, if it turns out that is all he needs). Chapter 20 in Part IV Further work discusses the possibility of extending the system to allow for detection of expressions that are always false or always true.

2

Our extended truth table is var similar to that of Strong Kleene (K3 ). In fact, had it not been for our asterisks marking the possibility of logical truths/falsehoods it would be identical to Strong Kleene. However, Strong Kleene does not contain logical truths. See for example [Fit94].

118

Part III Running the Simulator

119

Chapter 10 Experiments in Knowledge Game 10.1

Introduction

Now that we have laid the foundations for more adequate simulations in social epistemology, we are ready to actually use our simulator to make some experiments.

It should be clear that with the epistemic profiles of our agents, described in section 6.6, it is possible to create a wide range of different agents with respect to requiring evidence, self-trust, trust in others, the quality and frequency of ones investigations and truth-telling. This means that even though our model of agents and interaction and our process, Knowledge Game, might only be partially adequate (see section 6.1.2), we still can do a lot of interesting experiments with our system.

Below, we have selected a few experiments to show what we can do with this system and what we can learn from it. Here is an overview of the experiments: â&#x20AC;˘ How the smart beat the stupid : experiment designed to test the epistemic significance of differences in quality of investigations in a simple social context, modeled as a Knowledge Game. â&#x20AC;˘ How the outspoken beat the careful : experiment designed to test the epistemic significance of differences in thresholds of assertion in a simple social context, modeled as a Knowledge Game.

120

• How the stupid and outspoken beat the smart and careful : experiment designed to test the epistemic significance of differences in both quality of investigations and threshold of assertion at the same time.

• How the stupid and outspoken beat the smart and careful – revisited : experiment designed to test the impact of the formulation of the game statement.

• Saints and sinners: experiment designed to test the epistemic significance of truth telling and lying in a simple social context, modeled as a Knowledge Game.

These experiments represent only a small part of what can be done in our simulator. For example would it be possible to experiment with all five features of the epistemic profiles of the agents and also with more agents and more complex combinations of features of their epistemic profiles. Nonetheless, the following experiments contain very interesting results.

121

Chapter 11 How the smart beat the stupid 11.1

Description

Let us start with a rather simple experiment. We want a smart agent and a stupid agent to compete against each other in Knowledge Game. It is fairly easy to make a prediction about the outcome. If our simulator works like expected, the smart agent should beat the stupid agent.

The game statement is a simple one: Red(a). In other words, the agents need to determine the truth value of â&#x20AC;&#x2DC;Entity a is redâ&#x20AC;&#x2122;.

In order to create a stupid and a smart agent, we create two identical agents except for their InvestigationQuality. Agent A, the smart agent, has an InvestigationQuality of 0.9. Agent B, the stupid agent, has an InvestigationQuality of 0.001. Each of them has an EvidenceBias of 0.05 and a AssertionProbabilityThreshold of 0.9.

This is the simplest possible way to create two agents, one of which we can call smart and one of which we can call stupid. Of course, we could make the agents even more different by changing some other parts of their epistemic profiles and still have one smart and one stupid. But we want to make the least possible change because it makes it a lot easier to isolate the influencing parameters. In this case, since the agents only have different InvestigationQualities, we can be sure that this is what causes the results.

In section 12.2 and 12.3, we see the XML-files defining our experiment and agents. We will not be looking at the communication lines between the agents, they are not relevant since the agents will almost never talk to each other.

122

11.2

Experiment definition

<Experiment Name="Experiment1" AgentsFile="1-Agents.xml" NetworkFile="1-Network.xml" GameStatement="Red(a)" Description="Smart agent beating stupid agent."

WorldSize="10" RandomSeed="" />

11.3

Agents

<Agents> <Agent Name="A" AssertionProbabilityThreshold="0.9" EvidenceBias="0.05" SelfTrust="0.9" InvestigationQuality="0.9" InvestigationProbability="0.9" /> <Agent Name="B" AssertionProbabilityThreshold="0.9" EvidenceBias="0.05" SelfTrust="0.9" InvestigationQuality="0.001" InvestigationProbability="0.9" /> </Agents>

11.4

Results

We let agent A and agent B play against each other 10.000. Here is what we get: 123

â&#x20AC;˘ Agent A: 6433 wins â&#x20AC;˘ Agent B: 87 wins â&#x20AC;˘ No winner: 3480 matches

11.5

Analysis

As one would expect, agent A wins almost every time there is a winner.1 This is no surprise, since both A and B require the same amount of evidence before they will answer the game statement. Since A has a lot better chance to get the correct evidence, he wins almost every game.

This simple experiment shows us that in this particular game setting, our simulator works as expected. The smart agent is able to beat a stupid agent.

1

One might wonder why there is such a high number of games with no winner. This is simply due to the fact that every game has a maximum of 50 rounds. If none of the agents has a correct answer in the first 50 rounds the game will stop without a winner.

This limit can be as high or low as we like, of course the more rounds allowed, the longer our experiments will take to run. All experiments in this thesis have the same limit of rounds in each game. In this particular experiment the high number of games with no winner is due to the high AssertionProbabilityThreshold. Both agents require a high amount of evidence before they will assert something. And it takes time to get evidence.

Therefore the high number of games with no winner.

124

Chapter 12 How the outspoken beat the careful 12.1

Description

In the above experiment we saw how a smart agent (high InvestigationQuality) beat a stupid agent (low InvestigationQuality). All other features of the agents were alike. In this experiment instead of isolating the InvestigationQuality like above, we want to isolate the impact of the AssertionProbabilityThreshold. We want to compare a careful agent (high AssertionProbabilityThreshold) who requires a lot of evidence before asserting anything to a more outspoken agent (low AssertionProbabilityThreshold) who just asserts almost anything. Agent A is the careful agent and thus has an AssertionProbabilityThreshold of 0.9. Agent B is the outspoken agent and thus has an AssertionProbabilityThreshold of 0.0001.

12.2

Experiment definition

<Experiment Name="Experiment2" AgentsFile="2-Agents.xml" NetworkFile="2-Network.xml" GameStatement="Red(a)" Description="Outspoken agent beating careful agent."

WorldSize="10"

125

RandomSeed="" />

12.3

Agents

<Agents> <Agent Name="A" AssertionProbabilityThreshold="0.9" EvidenceBias="0.05" SelfTrust="0.9" InvestigationQuality="0.9" InvestigationProbability="0.9" /> <Agent Name="B" AssertionProbabilityThreshold="0.0001" EvidenceBias="0.05" SelfTrust="0.9" InvestigationQuality="0.9" InvestigationProbability="0.9" /> </Agents>

12.4

Results

We let agent A and agent B play against each other 10.000 times. Here is what we get: • Agent A: 1613 wins • Agent B: 8387 wins • No winner: 0 matches

126

12.5

Analysis

There is no doubt that agent B (the outspoken agent) is the winner. Apparently it pays off to just assert things and answer very quickly without requiring a lot of evidence. In this case it is important to note that both agents have a very high InvestigationQuality at 0.9. This means that they both are almost always correct in their investigations, and they do a lot of investigations because they have an InvestigationProbability of 0.9.

So it is actually not that surprising that the outspoken agent wins. Since both agents have the same chances in answering correctly, but one agent answers more quickly than the other, and the agent who answers correctly first wins, then of course the outspoken agent wins the most games.

127

Chapter 13 How the stupid and outspoken beat the smart and careful 13.1

Description

Now we have seen a smart agent beat a stupid agent, exactly like we would expect. We have also seen an outspoken agent beat a careful agent. Let us then have a look at another experiment, a combination of the ones above.

We want to keep the difference in InvestigationQuality from the first experiment, where agent A was very smart (InvestigationQuality of 0.9) and agent B was very stupid (InvestigationQuality of 0.001). Also, we want to keep the difference of the agents from Chapter 12, actually we want to make them a bit more different with respect to carefulness. Specifically, we want agent B to require even less evidence by adjusting his EvidenceBias also, this makes him even more outspoken. Let us set his EvidenceBias to 0 and keep his AssertionProbabilityThreshold at 0.0001. Agent A is still careful (with an AssertionProbabilityThreshold of 0.9), but agent A is also very smart (with an InvestigationQuality of 0.9). The game statement stays exactly the same as in the experiments above.

What would one think will happen in this case? One could think that we have made agent B even more stupid than in the experiment in Chapter 11 because he now is not just terrible at making investigations, but also has no requirements for evidence and will just assert anything he hears or sees the slightest evidence of. And agent A seems pretty smart, he is not only good at making investigations, he also requires some evidence before he speaks.

128

Let us see what happens when we run the experiment defined in section 13.2 and 13.3.

13.2

Experiment definition

<Experiment Name="Experiment3" AgentsFile="3-Agents.xml" NetworkFile="3-Network.xml" GameStatement="Red(a)" Description="Stupid agent beating smart agent."

WorldSize="10" RandomSeed="" />

13.3

Agents

<Agents> <Agent Name="A" AssertionProbabilityThreshold="0.9" Bias="0.05" SelfTrust="0.9" InvestigationQuality="0.9" InvestigationProbability="0.9" /> <Agent Name="B" AssertionProbabilityThreshold="0.0001" Bias="0.0" SelfTrust="0.9" InvestigationQuality="0.001" InvestigationProbability="0.9" /> </Agents>

129

13.4

Results

Again we let agent A and agent B play against each other 10.000 times and we get the following results: â&#x20AC;˘ Agent A: 3071 wins â&#x20AC;˘ Agent B: 6929 wins

13.5

Analysis

Suddenly, things changed quite a bit since we saw the smart agent beat the stupid agent in Chapter 11. Here we have a stupid (and outspoken) agent beating a smart (and careful) agent a lot more than half of the times. How can that be? The explanation is found in the AssertionProbabilityThreshold, the EvidenceBias and the particular game statement. Since agent B has no EvidenceBias and a very lowAssertionProbabilityThreshold, he will try to answer the game statement as soon as he gets the chance. Furthermore, he has almost no ability as an investigator, so almost all his investigation will yield wrong results. But how does that make him win? One should think that he would lose pretty quick since an agent will be out of the game if he makes an incorrect guess. The answer is that in the way the game statement is set up, it is extremely easy to guess the right answer.

The game statement says that entity a is red. There are six possible colors of entity a. Because agent B has no abilities whatsoever as an investigator, he will get a completely random result when investigating. This means that five out of six times he will come to the conclusion that a is not red. Since he has no EvidenceBias and a low AssertionProbabilityThreshold, he will be ready to answer the game statement after investigating just once. In five out of six games, he will answer that a is not red, after just one investigation.

Let us calculate the probability that agent B will be right when he answers the game question. First of all, let us look at the situation where entity a is not red. Entity a is not red five out of six times, because we generate the world for each game randomly. This gives us the probability 56 that a is not red. Five out of six times, agent B will make the guess that a is not red because his investigations are random. This means that if a is in fact not red, then there is a 56 chance that agent B guesses right.

130

If a is in fact red, then agent B will be guessing correctly one out of six times. This gives us a 16 chance that B is right. And a is red 16 of the times.

The probabilities are illustrated in Figure 14.1. All this of course depends on the way we generate our worlds (see Chapter 5), namely the fact that the worlds are generated randomly, which means equal probabilities for each of the properties of the entities.

1 6

a is red a is not red

5 6

B is right

B is wrong

1 6 5 6

5 6 1 6

Table 13.1: The probabilities that agent B is right, when he guesses the truth value of the game statement.

We can calculate the probability that agent B is right like this: , which is about 0.7. This means that because B ( 16 â&#x2C6;&#x2014; 16 ) + ( 56 â&#x2C6;&#x2014; 65 ) = 13 18 tries to answer whenever he get the chance, he has about a 70% to answer correctly. This matches our results where agent B wins about 69 % of the games.

At first sight, it seems strange that a stupid agent is able to continuously beat a smart agent. But as we have seen, this is because the game statement is so easy to guess that an agent who answers randomly has a better chance than one who gathers a lot of evidence before he answers.

131

Chapter 14 How the stupid and outspoken beat the smart and careful â&#x20AC;&#x201C; revisited 14.1

Description

Now we have seen both how a smart agent can beat a stupid agent and how a really stupid and outspoken agent can beat a smart and careful agent because of the probabilities of guessing the right answer. This shows us how important the game statement is and how big influence it has on the chances of winning. As we saw above, when the game statement is true in only one out of six times, it is better to be a random guessing agent, than one with high requirements for evidence. Basically, in certain situations it seems like it is better to act random and quick than to have good reasons and act accordingly.

Let us have a look at the exact same experiment as the one in section 13, except for one thing: the game statement. In this experiment, we will create a game statement which has a 50% chance of being true and see what happens.

14.2

Experiment definition

<Experiment Name="Experiment4"

132

AgentsFile="4-Agents.xml" NetworkFile="4-Network.xml" GameStatement="Red(a) | Orange(a) | Blue(a)" Description="Stupid agent beating smart agent - revisited."

WorldSize="10" RandomSeed="" />

14.3

Agents

<Agents> <Agent Name="A" AssertionProbabilityThreshold="0.9" Bias="0.05" SelfTrust="0.9" InvestigationQuality="0.9" InvestigationProbability="0.9" /> <Agent Name="B" AssertionProbabilityThreshold="0.0001" Bias="0.0" SelfTrust="0.9" InvestigationQuality="0.001" InvestigationProbability="0.9" /> </Agents>

14.4

Results

The results are as follows: â&#x20AC;˘ Agent A: 5188 wins â&#x20AC;˘ Agent B: 4812 wins

133

14.5

Analysis

This means that just because we have changed the game statement, it now suddenly again pays off a lot better to be smart and careful. And again, this has its reasons in the probability of guessing the game statement. Let us have a look at the probabilities for agent B to win by guessing the game statement.

1 2

a is red, orange or blue a is not red, orange or blue

1 2

B is right

B is wrong

1 2 1 2

1 2 1 2

Table 14.1: The probabilities that agent B is right, when he guesses the truth value of the game statement.

We can calculate the probability that agent B is right like this: ( 12 â&#x2C6;&#x2014; 21 ) + ( 12 â&#x2C6;&#x2014; 21 ) = 12 Now the probability of agent B to guess the truth value of the game statement is 50%. And this matches our results from above, where agent B is right 48.12% of the games played.

This lesson learned is that the game statement we choose has a big influence on who wins the game when we compare a smart agent to one who makes random guessing.

134

Chapter 15 Saints and sinners 15.1

Description

Let us make another type of experiment than the previous ones. In section 6.5, we saw a definition of an epistemic saint and an epistemic sinner. A general epistemic saint is always telling the truth, no matter who he speaks to. A general epistemic sinner is always telling lies, no matter who he speaks to.

In this experiment we shall see if there is any difference in performance of an epistemic saint and an epistemic sinner. Where the previous experiments only involved a social aspect in the sense that they contained more than one agent competing, this experiment involves a more complex social interaction between the agents - namely the one of truth telling and lying.

We will keep the agents equal in any other way. Both agents A and B have a AssertionProbabilityThreshold, EvidenceBias, SelfTrust, InvestigationQuality and InvestigationProbability of 0.5, so no one has an advantage with respect to these parameters. We give agent A a TruthProbability of 0.999 and agent B a TruthProbability of 0.001.

As can be seen in section 15.4, agent A always tells agent B the truth, and thus he is an epistemic saint. Agent B is an epistemic sinner because he always lies to agent A.1 1

Note that in this experiment the network specification is very relevant, because the agents communicate with each other. In the previous experiments the agents were almost always investigating on their own, and therefore the network specification was not relevant for the experiment.

135

15.2

Experiment definition

<Experiment Name="Experiment5" AgentsFile="5-Agents.xml" NetworkFile="5-Network.xml" GameStatement="Red(a) | Orange(a) | Blue(a)" Description="Epistemic Saints vs epistemic sinners."

WorldSize="10" RandomSeed="" />

15.3

Agents

<Agents> <Agent Name="A" AssertionProbabilityThreshold="0.5" EvidenceBias="0.5" SelfTrust="0.5" InvestigationQuality="0.5" InvestigationProbability="0.5" /> <Agent Name="B" AssertionProbabilityThreshold="0.5" EvidenceBias="0.5" SelfTrust="0.5" InvestigationQuality="0.5" InvestigationProbability="0.5" /> </Agents>

15.4

Network

<Network> <CommunicationLine

136

Source="A" Destination="B" Trust="0.5" TruthProbability="0.999" /> <CommunicationLine Source="B" Destination="A" Trust="0.5" TruthProbability="0.001" /> </Network>

15.5

Results

The results of the experiment are as follows: • Agent A: 105 wins • Agent B: 1733 wins • No winner: 8162 matches

15.6

Analysis

Many of the games have no winner. What is also interesting is that agent B wins a lot more than agent A. Remember that agent B is the epistemic sinner. It seems like agent B has an advantage in this setting, even though most of the games have no winner.

So why is it that agent B wins? Is it because he lies to agent A or is it because agent A always tells the truth to B? Or maybe it is a combination of the two? If we change our agents a little bit, we might find out. Let us first try to make agent A neither a sinner nor a saint, this means a TruthProbability of 0.5 and see what happens: • Agent A: 103 wins • Agent B: 327 wins

137

• No winner: 9570 matches Agent B still wins more than agent A, but not as much as before. Let us now try the other way around, let agent A be an epistemic saint with a TruthProbability of 0.999, but change agent B to neither a sinner nor a saint with a TruthProbability of 0.5. The results are: • Agent A: 297 wins • Agent B: 1677 wins • No winner: 8026 matches If we look at the ratio between agents in the three cases, we see that: • A (Saint) vs agent B (Sinner): Agent B wins about 16 times more than agent A. • A (None) vs agent B (Sinner): Agent B wins about 3 times more than agent A. • A (Saint) vs agent B (None): Agent B wins about 6 times more than agent A. This tells us that in this setting it is epistemically beneficial to always lie (more than your competitor). Or stated in another way, it seems epistemically beneficial to get more true information as input than you give out to your competitor. Another thing that is worth mentioning is that lying comes at no cost in the current setting. It might seem a bit too simplified, for example one could imagine that in real life lying to someone would mean that the person lied to would lower his trust in you - at least if you were caught in the lie. In Chapter 18 we will discuss the possibility of adding dynamic trust to the system.

138

Chapter 16 Conclusions 16.1

Conclusions

We have now seen how our simulator can be used to run experiments involving the intelligence of the agents and their probability of telling the truth. Here is an overview of the results of the different experiments: • How the smart beat the stupid : the result came out as expected - the agent with a high quality of investigations beat the agent with a low quality of investigations.

• How the outspoken beat the careful : the outspoken agent beat the careful agent in this experiment.

• How the stupid (and outspoken) beat the smart (and careful): this experiment showed us that an agent with low investigation quality and low threshold of assertion can easily beat an agent with high investigation quality and high threshold of assertion, even though this seems a bit counter-intuitive. The reason lies in the formulation of the game statement.

• How the stupid (and outspoken) beat the smart (and careful) - revisited : this experiment showed exactly how much the formulation of the game statement matters for the probability of being able to guess the truth value of the game statement.

• Saints and sinners: the result of the comparison of truth tellers and liars was that under certain conditions it is epistemically beneficial to 139

lie.

These examples show us that our simulator works as expected in most cases. In the case where it did not (in Chapter 13), we saw that the reason for this was to be found in the setup of the game statement.

It should be clear that our simulator can be used for a lot of other experiments, for example involving trust in one self, trust in others, bias and the like. And of course, one can also run experiments with a lot more agents.

From the experiments above, we can see that we actually are able to draw some conclusions from our simulator. For example can we draw the conclusion that under all the assumptions described in this thesis in general and specifically in Chapter 15, it actually is epistemically more beneficial to lie to the opponent, if the opponent tells you the truth (at least half of the times). This is actually a valid conclusion in this simulator. But it took us an awful lot of work of describing our assumptions and simulator to get there.

And still, we cannot say for sure say that this holds in the real world as well, since our simulator is not fully adequate, it might not even be partially adequate. Even though we need to make a lot of explanation to make sure every assumption made in the simulator is out in the open, it still sure is promising that we were able to get this far.

Let us look back at what it actually took to get us so far that we actually have a working simulator. First of all, we learned from Laputa, how we did not want our model to work with respect to for example trust. Second, we built a controlled social process our own, Knowledge Game, which can also be replicated in the real world because it is a game. Third, we built the actual model of the agents and their interaction. For this purpose we designed a logic for the agents to reason in, and we created a model checker able to check the validity of the statements in our logic. And we did not just write about all this, we actually developed the software in which all this is implemented, which makes us capable of running the above experiments.

The software is written in C# and in its current version it consists of several different parts including the social process, Knowledge Game, the logic and the model checker. The fact that this thesis contains not only the conceptual development of a simulator, but also the actual software makes it a positive project. It would have been very easy to just take on the negative task of criticizing the adequacy of the existing systems, but not only did we develop the conceptual framework for more adequate simulations in social epistemology, 140

we also developed a software application containing this, that is the actual simulator.

So now we have a first version of a simulator for use in social epistemology.

At the same time we have pointed at some of the general thoughts and challenges for simulations in social epistemology. Even though there is no more space left in this thesis, there is of course a lot of further work to be done with this simulator, both with respect to the adequacy of the model and with respect to the complexity of the experiments run in it. We will go into detail with a few in Part IV.

141

Part IV Further work

142

Chapter 17 Introduction 17.1

First version of our simulation software

This thesis describes the first version of our simulation software for use in social epistemology. In this thesis, we have built both the agents, the interaction between the agents and a social process to simulate, namely the Knowledge Game.

Even though this version of the system might not be fully faithful, it should be clear that it is indeed more adequate than the existing competitors.

And even though our social process, Knowledge Game, is not a complex social interaction similar to the processes described by for example Goldman in [Gol99], it hopefully is clear that we are able to switch Knowledge Game with another social process of our liking, possibly an even more complex one.

Therefore, our system should be able to yield more interesting results than competing systems, when switching to such complex social processes.

17.2

What to do next?

We have hinted at some improvements of the existing model at various places above. In the following, we will have a look at how our model could be improved in order to make it even more adequate. Among these possible improvements is the need for a dynamic trust function, so the agents can adjust their trust in each other during the game. Also a patience function would be interesting to work with, it would make it possible for the agents to have some sort of memory of their interaction. It is also obvious that a

143

lot more work on the diversity of the agents could be done, especially with respect to their logical abilities. In some ways, our agents in general are just too smart compared to human beings. The last thing, we want to visit in this part, is the possibility of modeling peer disagreement. Modeling peer disagreement can be seen as both something which should make Knowledge Game even more adequate as a social process, but it can also be seen as a social process of interest in itself, in social epistemology.

144

Chapter 18 Dynamic trust 18.1

Introduction

We have already seen how trust in both the agent himself and in other agents is used, when we calculate what an agent believes about the entities in the world. The trust is used for weighing the incoming information. However, if the trust an agent has in another agent (or himself for that matter) is a static value, our model could very well be too simplistic.

In real life, trust seems to be a quite dynamic phenomenon: if you realize that someone is lying to you or maybe just does not know what he is talking about, it seems very likely that you will lower your trust in him. Thus, we would need to somehow make trust a dynamic relation. There are obviously many ways to do this and no matter how we may choose to do it, we will have to make some limiting assumptions. Hence we should make it very clear what we choose and that our choice will most likely limit the adequacy of our model. Nevertheless, we should do our best to build a mechanism that at least sounds realistic.

18.2

Keeping track of statements

Our agents do â&#x20AC;&#x201C; as we have seen â&#x20AC;&#x201C; keep track of the weight-sums of evidence for each property, for each entity. However, they do not keep track of who told them what, that is they do not remember that for instance Alice said that entity e was green. That is a quite strong assumption about how our minds work. But it sounds like a reasonable assumption: it seems plausible

145

that real people do not keep track of who told them what, for all statements they ever heard. However, when it comes to adjusting trust, it may very well be too limiting. An example will show us why.

Imagine that you had a wonderful teacher in middle school. You really enjoyed coming to his lessons and you learned more from him than from any other teacher you ever had. This means you probably have very high trust in him. Now, imagine that you somehow learn that a lot of what he taught you was simply wrong. Not necessarily because he wanted to deceive you and your classmates, but because he just did not know better. After learning this, it seems plausible that you not only should, but actually would lower your trust in the old teacher. In other words, even though it seems plausible that we do not keep track of the sources of everything we hear, it seems most likely that in some cases, we do remember who the source is, and we use this to adjust our trust in the source later on, maybe even years later. We have no way of implementing this in Knowledge Game yet, since we do not keep track of the source of each piece of evidence. Our current way of keeping track of evidence is therefore potentially limiting the faithfulness and more importantly the adequacy of our model.

Creating a dynamic trust function would therefore be high on our list of priorities when making the second version of this system.

18.3

Rough ideas for an implementation

Before implementing even the simplest form of dynamic trust there are a few things to consider. It may be tempting to think of implementing something along the lines of â&#x20AC;&#x2DC;when agent A receives information from agent B then A will downgrade his trust in B if the information is false, otherwise he will upgrade the trustâ&#x20AC;&#x2122;. This, however, seems to ignore the real problem: agent A may not, at the time of information exchange (t0 ), know whether the information is true or false, and we can only base the updates on what the agents know, otherwise it would be some sort of omniscient trust updates. This means we actually have several problems to solve: â&#x20AC;˘ At t0 how do we determine if the agent already has strong enough belief with regard to the truth of the new piece of information to update the trust in the source?

146

• If it is determined that the agent should update the trust at t0 , how should he do it?

• If the agent should not update the trust in the source yet, how should he track the source of the information for later use?

• If the agent is later (t1 ) convinced that the information received at t0 was false, how should he update the trust?

The most basic of the questions above seems to be the first one: how do we determine if we should update the trust at t0 ? Fortunately this has a relatively straight-forward answer: The agent receiving information will be in a position to adjust trust based on the received information if and only if he is willing to assert whether the information is true or false before adjusting his belief set using the new information. This seems plausible: Agent A would be willing to lower the trust in B based on something B says just in case A already thinks he knows differently. And conversely, if A does not yet have enough information about the issue at hand to assert anything about it, he would have no way of knowing how to update the trust, and hence should not update it at all.

Now we can start trying to answer the rest of the above questions. Actually we can describe a strategy that will answer all of the questions at once.

But first let us sketch out a situation where dynamic trust is relevant: Imagine that at t0 agent A tells B that entity e is red. This is the first A ever hears regarding the color of e so before receiving this information he is not willing to assert anything about it, and hence will not update the trust in B. In the following rounds of the game A receives a lot of evidence that e is green and finally has enough evidence to assert that e is green.

Now, what should agent A do? Very broadly we can distinguish between two strategies: updating the trust or using on-the-fly trust. If we choose to use a strategy that updates the trust we will have to detect that a change has occurred in what the agent is willing to assert and then adjust trust in all the agents who have been sources for that information (or the negation of it) accordingly. This raises another question: • If new information from agent C causes agent A to lower his trust in agent B, how should old information from agent B be treated?

147

Let us see an example of how this question can come up: Agent A starts out with fairly high trust in B. B convinces A that e is red and that f is green. Later on A is convinced by other agents that e is in fact blue. This makes A lower his trust in B. But should it also make A recalculate his belief in the color of f using the new and lowered trust in B? If beliefs are recalculated using the updated trust value it appears that there is a risk of apparently paradoxical updates: Imagine that A is convinced by B that e is red and that f is green. Now imagine that the fact that A is convinced that f is green makes him trust agent C, D and E who also say that f is green. Now agent C, D and E say that e is not red but instead blue. Since A has come to trust C, D and E (based on their agreement with B out the color of f ) this convinces A that e is blue. This will make A trust B less. But A only had high enough trust in C, D and E because they agreed with B who A trusted.

And now A no longer trusts B. So what should A do? If he lowers his trust in B, the reason for trusting B, C and D disappears and thus he should not be convinced that e is blue instead of red and hence he should not lower his trust in B. On the other hand, if he does not lower his trust in B, how can the trust be dynamic?

One way of dealing with the problem of the seemingly paradoxical update described would be to simply downgrade the trust in both B and B, C and D. Another way would be to change the concept of trust a bit to make an agent not just have trust in another agent but have trust in another agent regarding a specific entity (or perhaps a specific property). This way A would only have to downgrade his trust in B regarding either the entity e or the entity f . Yet another way to deal with the challenge of paradoxical updates would be to simply not recalculate other beliefs based on old information from the source in which the trust has been changed. A relatively simple way to do this is to use what we will call on-the-fly trust by which we mean that the trust used when updating the belief set is always calculated on-the-fly. It can be described along the following lines: Each agent has a base trust in each other agent. When agent A receives new information from agent B he will look 148

at all the pieces of information he has received from B and for each of them calculate whether he is at the moment willing to assert the same thing. The on-the-fly trust will be calculated as a function of the results for each received piece of information and the base trust.

18.3.1

Memory quality

To make any of the suggested ways of making trust dynamic work the agents must of course be able to track from who they have received each piece of information. This is not technically problematic but it may be fruitful to to consider different strategies. A very simple strategy would be to simply make the agents remember the source of all the information they get. This may however be a bit unrealistic as mentioned in section 18.2 above. A better approach may be to add a parameter that defines the probability that the agent will remember the source when new information is added. This will be much more realistic and will enable us to test agents who are good at remembering the source (and hence good at updating the trust) against agents who almost never remember the source. However, an even better approach may be to add a parameter that defines the probability that the agent remembers the source correctly. That way, the agent will not only be able to have low quality memory in the sense that he cannot remember the source but he can have remember things wrong: Agent A may be convinced that agent B was source of a specific piece of information when it was in fact C. Furthermore, a parameter for how quickly the memory fades may be relevant.

18.3.2

Other adjustments needed

On a final note we should mention that the examples described above could not actually happen the way Knowledge Game works right now. The examples are based on agents that get new information from other agents regarding a specific property of a specific entity when they are already willing to assert a value for that property and entity. This cannot happen in the current version of Knowledge Game as the agents simply stop asking when they have enough evidence to make an assertion. Hence we will somehow need to adjust the mechanics of the game to actually make the dynamic trust relevant to the game. The only exception to this is self-trust: when an agent makes his

149

own investigations he may receive information about an entity and property he already knows about.

150

Chapter 19 Patience 19.1

Introduction

When playing Knowledge Game the agents can, as we have already seen, ask each other questions or carry out their own investigations. As we have seen, making investigations is a bit less specific than asking questions, in the sense that an investigation returns a random property for the specified entity instead of the value of the specific property for the specified entity. Thus it might seem that there is no reason at all for the agents to make their own investigations â&#x20AC;&#x201C; it is easier to just ask the other questions. This may indeed be true in the current version of the Knowledge Game, but we could imagine another version of the system where if an agent keeps asking the other agents and not doing any inquires himself, the other agents may lose their patience with him.

Introducing some kind of patience attribute would give our agents a memory with respect to their previous history of interaction with other agents.

This could possibly make a huge difference to the results the system generates, and it would most certainly make a difference in terms of the adequacy of the system. In the following, we will have a look at a suggestion of such a patience attribute.

19.2

The Patience Feature

Contrary to the static TruthProbability, we can imagine concept of patience which is dynamic and concerns the individual relationship one agent

151

has to another agent.

We would want the patience not only to be dynamic in the way that it should decrease under certain circumstances. We would also want it to be possible for the patience to increase under certain circumstances. As mentioned, an agent’s patience should decrease he is asked a question by another agent. To keep it simple, we could restrict the situations of increasing patience to two situations: • Round without question – If one round of the game gets by without agent B getting asked by agent A, then agent B’s patience with agent A should increase a little, but not as much as: • Round with outgoing question – If agent B in one round himself asks agent A a question, then agent B’s patience with agent A should increase even more because he now also has got some information from agent A. and the patience should decrease in only one situation: • Round with incoming question – When agent A asks B a question, B’s patience with A should decrease.

To keep this simple, we would not use probabilities to calculate this, but instead we would use a kind of barometer. We could imagine the values of the different situations as follows: • Round without question – A round without a question from agent A to agent B should increase agent B’s CurrentPatience with A with 0.5. But only up to a certain limit, namely the MaxPatience.

• Round with outgoing question – A round where B asks A a question should increase agent B’s CurrentPatience with agent A with 1.0. On this kind of increase, there is no limit. But if this increase goes beyond the MaxPatience, then the MaxPatience should be increased along with the CurrentPatience.

• Round with incoming question – A round with a question from agent A to agent B should decrease agent B’s CurrentPatience with agent A with 1.0.

152

MaxPatience = 3 Patience

0

Impatience

CurrentPatience = - 2

Figure 19.1: An example of the patience feature.

This could be our patience feature and what we imagine it is illustrated in Figure 19.

The benefits of this way of modeling patience would be several. First of all, it is very simple. Basically, CurrentPatience would just move up and down according to the rules above. This means that it would be transparent and easy to understand how the patience in our system would work. It is likely that some of the results generated in this system would depend strongly of this notion of patience. If that happens, our chances of understanding why and how the results are determined by patience would be better because of the simplicity.

Another benefit would be that the patience is sophisticated enough that we could represent the different values of the two patience-increasing situations. We would be able to represent the intuition that if you just get left alone for a while, then your patience with another agent would automatically increase a bit. But if you get some valuable information from another agent, then you would feel you owe something to that agent and this would not only increase your current patience with this person, but also your potential maximum patience with this agent.

A third benefit would be that this function of patience would allow the

153

agents to have different MaxPatience values. This means that we could represent different types of agents, some would have a very high general patience with everyone else or someone would have a very low general patience.

The question now would of course be what the agent would do if the CurrentPatience is larger than 0 (the green zone in Figure 19.1) and what action would an agent perform if the CurrentPatience is below 0 (the red zone in Figure 19)?

When we talk about patience, we need to decide what should happen, when an agent loses patience with another agent. Let us say that Ann has been asking Ben questions a lot of times. Ben has not asked Ann anything at all and he is losing patience with Ann. What action should Ben take with Ann? We could imagine that Ben simply gets so tired of agent Ann trying to use all Ben’s information and Ben getting nothing in return that Ben simply starts to lie to Ann every time Ann asks Ben. This is however, a very drastic action to take, when you get annoyed with someone. An impatient agent like Ben could lie, but this might be too strong a reaction. Instead he maybe should just ignore the question he is being asked by someone he has no patience with. And of course then adjust his CurrentPatience accordingly. If an agent has patience with the asking agent, what he answers should depend on the rest of his epistemic profile.

19.3

Saints and sinners revisited

With this kind of patience feature implemented in our simulator, we would be able to make an even more sophisticated definition of epistemic saints and sinners, namely patient saints and impatient sinners. The types could be defined as follows: • General epistemic saint – Agent A is an epistemic saint in general if and only if, for any other agent A*, agent A always tells what in his view is the true answer to the question posed by A* regardless of how many times A* has asked A questions previously (and regardless of how many times A has asked A* questions).

• Specific epistemic saint – Agent A is an epistemic saint specifically with respect to another agent A* if and only if, A always tells what in his view is the true answer to the question posed by A* regardless of 154

how many times A* has asked A questions previously (and regardless of how many times A has asked A* questions).

• General epistemic sinner – Agent A is an epistemic sinner in general if and only if, for any other agent A*, A always ignores inquiries from A* or provides what in her view is a wrong answer regardless of how many times A* has asked A questions previously and regardless of how many times A has asked A* questions.

• Specific epistemic sinner – Agent A is en epistemic sinner specifically with respect to another agent A* if and only if, A always ignores inquiries from A* or provides what in her view is a wrong answer regardless of how many times A* has asked A questions previously and regardless of how many times A has asked A* questions.

If we started to run an experiment, we might not want any of the competing agents to be saints or sinners in general or with respect to specific other agents. But as the game would evolve, it could be interesting to see how many of the agents would end up being either general or specific epistemic saints or sinners. The patience feature might be able to turn agents into either of the four types during a game, because of the social interaction between the agents.

Since the goal of the social interaction between the agents is to win a game, the agents will interact only in order to achieve the goal, that is to win the Knowledge Game. They may not act in the optimal way because they not necessarily have optimal epistemic profiles, but they will always act in their own self-interest within the limits of their epistemic profile. For example, an agent could have a very low ThresholdOfAssertion and therefore be ready to answer the game statement very early – and maybe before an optimal agent would answer – but he would still answer in his own self-interest, namely to actually win the game.

The only way to win Knowledge Game is to gather information about the world (if we leave out the strategy, where one just answers the game statement randomly). You could therefore say that it is only in an agent’s epistemic self-interest to interact with other agents if he thereby is able to gather new information. But then, why would an agent ever want to answer a question posed by another agent?

With the proposed patience feature, we can answer that question fairly straight forward: An agent would want to answer a question from another 155

agent because it would be in his epistemic self-interest to make sure that the other agent, at some point, would give some information back in return.

With the patience feature, the free-riders, who just benefit from other agents’ information without contributing anything, would not have an easy time.

This is because the patience feature would allow the agents to, in some way, store their history of interaction with other agents and thereby make sure not to interact with free-riders.

The patience feature would also allow the agents to ‘owe’ each other favors. If you ask another agent a question, then you would adjust your patience with that particular agent accordingly. This means that the patience feature would allow the agents to act in a sort of favor economy. Implementing such a feature would therefore make it possible to make a whole new kind of experiments regarding the agents’ relationships with each other, and at the same time simply make their relationships a little more complex and closer to reality than it is in the current system.

156

Chapter 20 Diversifying agentsâ&#x20AC;&#x2122; logical abilities 20.1

Imperfect logicians

We saw in Chapter 9 how the agents, in the current version of Knowledge Game, use a model checker to determine the truth value of the game statement. We noted how this means that the agents cannot spot that an expression is a tautology, a valid expression or a contradiction. Thus, they will have to gather information about the entities of the world to determine that a given expression is true, even if that sentence is a tautology. Had our agents been perfect logicians, investigation would not have been necessary in the case of expressions that are always true or always false. This raises two questions: 1. Would agents, who are perfect logicians, be more realistic and thus increase the faithfulness and adequacy of the model?

2. How could we implement perfect logicians?

We will address the questions in the following two sections.

20.2

Perfect logicians

Would implementing our agents as perfect logicians make the system more adequate? This question does not have a short and simple answer. In some 157

cases, some humans are obviously better than our model checker. Take for example this sentence: â&#x20AC;&#x2DC;This ball is black or it is not blackâ&#x20AC;&#x2122;. It seems reasonable to assume that most human beings could determine that the sentence is always true without actually looking at the ball (unless, perhaps, they thought it was a trick question). This could indicate that implementing our agents as perfect logicians would be more adequate than just using the model checker. However, in other cases, humans seem to be no better than a model checker and in some cases maybe even worse. If the sentence had been longer with several connectives and multiple layers of negations, it would be a lot harder for a human being to spot that it is always true without looking at the entities the sentence mentions. Given complex enough sentences, a human being may not even be able to tell if a sentence is true, even if he has access to all information about the properties of the mentioned entities. This makes the human worse off than the model checker and a lot worse than the perfect logician. In other words, just turning the agents into perfect logicians does not seem to increase adequacy, but in some cases better logical abilities than those found in the model checker, seem to be desirable.

One way to go about this could be to make the logical abilities of each agent definable by the user. For example, a maximum level of complexity1 could be specified and the agent would have the abilities of a perfect logicians when checking the expressions (or partial expressions, that is sub-trees) with complexity below the specified level. This would enable os to compare how agents with different logical capabilities fare. In section 20.4 below, we will see a famous experiment that seems to show that even very simple expressions involving conditionals can give rise to problems for a lot of people, but first a few words on the possibility of actually implementing perfect logicians.

20.3

Satisfiability solver

We already hinted to a possible solution for implementing perfect logicians in section 9.2.2, namely a satisfiability solver. Very broadly speaking, a satisfiability solver works by checking if an expression can be made true or in other words, if it can be satisfied, hence the name. This can be used to check if a given expression is always true if we start by negating the expression: If 1

This, of course, would also require a way of specifying the complexity of the expressions.

158

an expression is always true, the negation of that expression is always false and since the negation is always false, it cannot be satisfied. Thus, if we want to check if p is always true, we can check if ÂŹp is satisfiable.

There are different strategies for building satisfiability solvers. It is relatively easy to build one for sentential logic, for example by simply trying all combinations of truth values for each variable to see if an interpretation, which makes the entire expression true, can be found. For monadic predicate logic, however, this approach is not feasible since the number of possible combinations explodes. However, restricting the use of the satisfiability solver to only expressions of low complexity could make the number of calculations more manageable.

20.4

Wason selection task

The Wason selection task is famous psychological task devised in 1966 by Peter Cathcart Wason2 . The task is a logical puzzle which seems extremely simple, but most people give the wrong answer. One instance of a Wason selection task is this: Four cards are put on a table. Each card has a letter on one side and a number on the other side. The visible faces are A, 7, J, 8. Now you must determine for each card if it is true that if the card has a vowel on one face, then it has an even number on the other face. Which cards do you need to turn?3 If the test subject turns a card that needs not be turned (because the answer is given by the already visible face) or fails to turn a card that needs to be turned, it counts as an error.

20.4.1

Solution

The proposition, which we must determine the truth of, says that if one face has a vowel then the other has an even number. In other words, the proposition is a conditional, p â&#x2020;&#x2019; q, where p and q are as follows: 2

See for example [Wik12d].

This exact formulation of a Wason selection task is our own. There are other variations, the example task described in [Wik12d] uses colors instead of letters. Wasonâ&#x20AC;&#x2122;s original task used number and letters like the one we describe [Wik12d].

3

159

p: ‘The letter is a vowel’ q: ‘The number is even’ Thus, all it takes to answer correctly is to know and use the truth table of the conditional. A conditional is false if and only if the antecedent is true and the consequent is false (see line 2 in Table 20.1). Since we can only see one face of each card before turning it, we will know either the antecedent or the consequent, but not both. If we can see the letter and it is not a vowel (the antecedent is false), then we can conclude that the conditional is true no matter what is on the other face, since both lines 3 and 4 in 20.1 make the conditional true. Likewise, if the card has the number up and it is even (the consequent is true), we can conclude that the conditional is true no matter the value of the antecedent, since both lines 1 and 3 in Table 20.1 make the conditional true. In all other cases (that is the card showing a vowel or an odd number) we need to turn the card to determine the truth value of the conditional. This is summed up in table Table 20.2.

No. 1 2 3 4

p T T F F

q T F T F

p→q T F T T

Table 20.1: Truth table for the conditional.

No. 1 2 3 4

p T F U U

q U U T F

p→q U T T U

Action Turn the card Don’t turn the card Don’t turn the card Turn the card

Table 20.2: Solution table for the Wason selection task. ‘U’ means unknown.

We can now easily give the correct answers by finding the right line in Table 20.2. The first card shows the letter ‘A’. This is a vowel which means the antecedent is true and the consequent unknown and this puts us in line 1 160

which says, we should turn the card as the value of the conditional depends on whether the other face of the card is odd or even. Second card shows the odd number 7 and thus we are in line 4 which says, we should turn the card.

Third card is the letter ’J’. This makes the antecedent known, but false, so we are in line 2 and should not turn the card. And finally, the fourth card shows the even number 8 making the consequent true and thus the conditional true independently of the antecedent, hence we should not turn the card.

20.4.2

Wason’s results and Knowledge Game

In Wason’s own study, less than 10 percent of the test subjects were able to answer correctly. Later tests have shown that the ability to answer correctly depends on the context of the task. If it is put in a social context, the subjects seem to fare better4 . Still the point remains valid: At least in some cases the vast majority of people are not able to either grasp the truth table of the conditional or put it to use when needed.5 This is interesting in relation to how Knowledge Game uses the built-in model checker: If the game question contains a conditional, it basically contains a Wason selection task. The way our model checker propagates knowledge all the way up the expression trees (see section 9.4.1 on page 116) makes them able to answer the Wason test correctly every single time. In the light of Wason’s results, this does not seem to be faithful, and, since it will probably make a difference to the results of the simulations, not adequate. Thus, a very interesting way to improve the adequacy of our system would be to implement a less effective model checker, which is not able to pass the Wason selection task and thereby making the agents seek more information than they will using the existing model checker.

4

[Wik12d] It is not only conditionals people have problems with, when it comes to logic. Also conjunctions tend to cause problems. The problem is known as the conjunction fallacy, that is when the conjunction itself is assigned a higher probability than any of the conjuncts.

An example of the fallacy would be to assign ‘Ben is a philosopher and a rugby player’ a higher probability than just ‘Ben is a philosopher’ or ‘Ben is a rugby player’. But the probability of the conjunction is always less than or equal to the probability of either of the conjuncts. [Wik12a].

5

161

Chapter 21 Peer disagreement 21.1

Introduction

Strategies for how to update belief has seen quite a bit of attention in social epistemology. A set of special cases concerns situations in which two or more persons, who consider themselves epistemic peers, come to disagree about something. The reason, the situations are particularly interesting, is that it does not seem obvious what you should do when disagreeing with someone you regard as your epistemic peer. Generally, there are two ways to go about it, conformism or non-conformism. Very broadly speaking, a conformist will adjust his original belief â&#x20AC;&#x201C; he will conform. The non-conformist, on the other hand, will say that, in at least some cases, he should not change his belief â&#x20AC;&#x201C; he will not conform.

Setting the details about how to change the belief aside, it still seems very likely that a society consisting of only non-conformists will yield a different epistemic result than a society of only conformists, especially if the nonconformists tend to stick to their belief most of the time. In other words, our choice of strategies for our agents seems to be crucial for the outcome of our simulations. Before going into details with disagreement strategies, we will have a look at a famous example by David Christensen called Mental Math which clearly explains the issue. We should imagine a group of friends who often go out to dinner. They regard each other as epistemic peers when it comes to mental math. This means that none of them has any reason to believe that any of them are better or worse than any of the others at doing calculations in their head.

162

Suppose that five of us go out to dinner. It is time to pay the check, so the question, we’re interested in, is how much we each owe. We can all see the bill, we all agree to give a 20 percent tip and we further agree to split the whole cost evenly, not worrying over who asked for imported water, skipped desert or drank more of the wine. I do the math in my head and become highly confident that our shares are $43 each. Meanwhile, my friend does the math in her head and becomes highly confident that our shares are $45 each. How should I react upon learning of her belief?1 A conformist would lower his belief in $43 and raise his belief in $45.

A non-conformist may just stick to his belief that the share is $43, or not change his belief as much as the conformist. David Christensen suggests the first option, that is lowering his belief in $43 and raising his belief in $45: If we set up the case in this way, it seems quite clear that I should lower my confidence that my share is $43 and raise my confidence that it’s $45. In fact, I think (though this is perhaps less obvious) that I should now accord these two hypotheses roughly equal credence.2 David Christensen may very well be right that one should think of the two shares, $43 and $45, with roughly equal credence, but nevertheless, it is easy to think of real life agents who would have a more non-conformist approach.

Furthermore, as we mentioned above, different disagreement strategies seem likely to yield very different epistemic results (after all if they did not, why bother discussing them?). This means, we should at least investigate if and how the different strategies can play a role in our simulation software. If different disagreement strategies could plausibly be implemented in a future version of our system, we would not have to just take David Christensen’s word, we could actually show if he is right that one ought to be a conformist.

Of course, before deciding on how to handle disagreement among epistemic peers, we must have a way to define what makes two agents epistemic peers.

We will do this using a notion of trust. Before we see how this is done, we will need to look a little closer at how trust is implemented in Knowledge Game, which is exactly what the next section is concerned with.

1 2

[Chr07].

[Chr07].

163

21.2

Implementing update strategies in Knowledge Game

We saw in Chapter 6 how our agents in the initial version of Knowledge Game can update their beliefs. They do not take disagreement into account or in other words, they update exactly the same way, no matter if they agree or disagree with the new information they get. In this section, we will discuss three different strategies for belief updates, not only in case of disagreement and not only among epistemic peers. They are as follows: Probability-based This is the only strategy implemented in the first version of Knowledge Game. This is the simplest of the strategies we are going to discuss. It is not concerned with whether or not the agents disagree, nor is it concerned with whether the agents are epistemic peers.

This strategy simply uses the trust in the other agent as a weight.

Conformism This strategy is designed to make an agent assign equal credence to the different views.

Non-conformism This strategy is designed to make an agent, at least in some cases, not lower his belief to equal credence with the other view.

Before we look at the details regarding conformism and non-conformism, we need two definitions. First we need to define what it means to disagree and secondly what it means that an agent regards another agent as his epistemic peer.

21.2.1

Disagreement

In Chapter 6, we saw how our agents keep weight-sums for each propertyvalue for each entity. The property-value of the agent is simply the one with the greatest weight-sum. How much he believes it is determined by the ratio between the greatest weight-sum and the sum of all weight-sums, possibly including a bias. For example, Albert has a bias of 0.05. He has one piece of evidence for the color of entity e being red from a source he trusts with 0.9.

He has no other evidence. There are six different colors in Knowledge Game.

Consequently we can easily calculate Albertâ&#x20AC;&#x2122;s belief-ratio as 0.9 / (0.9 + 0.05 + 0.05 + 0.05 + 0.05 + 0.05 + 0.05) = 0.75. Now, what does it take for 164

Albert to disagree with another agent? The following section describes one suggestion.

21.2.1.1

Disagreement defined by differing ratios

One way of defining disagreement that may seem rather obvious, is to say that two agents disagree over a proposition if and only if they have different ratios for that proposition. After all, if the ratios are different, the beliefs are different and that seems to be what matters. However, this is nowhere as simple as it sounds. First of all, another agent, Bob, may completely agree with Albert that e is red, but he may have even more evidence (or less bias) which makes his ratio for red 0.9, which is greater than Albertâ&#x20AC;&#x2122;s 0.75.

That means, we should say they disagree. It does seem a bit odd to say that they disagree when they both believe that e is red. At the very least we should then acknowledge that there is a stronger and perhaps different kind of disagreement if one of them had believed that e was green.

To get closer to a plausible definition of disagreement, let us imagine that Albert and Bob start talking to each other. Albert asks Bob what color e is. We could now imagine several different outcomes. First of all, Bob only asserts that e is red if his ratio is above his assertion threshold. If it is not, he will just say he does not know what color e is. Does it not seem a bit odd to say that Albert and Bob disagree over the color of e if Bob says he does not know what color it is? Furthermore, what if Albertâ&#x20AC;&#x2122;s ratio is below his threshold for assertion? Then both agents would say they do not know what color e is â&#x20AC;&#x201C; how could it be plausible to say they disagree? These examples indicate that something else, than just the ratio, should at least play a role in defining disagreement. Of course, someone may remind us that we should think of disagreement as something having only to do with the beliefs and not with what the agents actually are willing to assert and maybe even less to do with what they actually assert. We think this is, if not a mistake, then at least not the most relevant way to think of disagreement in the context of a simulator. Here is why.

In Chapter 6, we described the agent as a layered onion. The beliefs are at the core and on top of them is calculation of assertion and then at the outmost is the actual communication. Figure 6.2 on page 81 shows this way of describing the agent. We can imagine a definition of disagreement for each layer of the onion. The way of defining disagreement, we mentioned above,

165

can be seen as defining disagreement for the core, the beliefs. We could think of this as some kind of ‘metaphysical disagreement’ or ‘deep disagreement’ as this is concerned with the inner beliefs of the agents (the beliefs they may never communicate). The fact that they may never communicate it makes it completely irrelevant to a simulator like Knowledge Game. Imagine that neither Albert nor Bob has strong enough belief in e being red to assert it. When asked, Bob will just say he does not know. Albert has no way of knowing what is actually inside Bob when Bob does not communicate it3 . We may say that Albert and Bob do not know, but deep down they do disagree. However, we must not lose our track: the reason we are discussing disagreement is to enable the agents to have different update strategies for when agreeing and disagreeing. Since Albert does not know the fact that he disagrees with Bob, how could he ever use that fact to decide to use his disagreement-strategy4 ?

Instead, we may jump to the second layer of the onion, the assertion layer, and define disagreement by what the agents are willing to assert. This is a bit better, since it only involves beliefs that are strong enough for the agents to communicate. However, it still completely disregards what the agents actually do communicate: it ignores the fact that the agents sometimes lie to each other. Imagine two agents, Charlie and Don, who both have beliefs in the color of e that are above their respective assertion thresholds. Charlie believes e is red and Don believes it is green. However, Don is a notorious liar, so when asked, he tells Charlie e is red. Again, you could say that on some deep level they do disagree, but for all Charlie knows they agree. And this is the only thing that matters: If Charlie does not know about it, he cannot use it when deciding how he should react to the information he gets from Don. Hence we should define disagreement in relation to what is actually communicated. Furthermore, since Don can lie to Charlie even if Charlie always tells Don the truth, disagreement is a directed relation, in the sense that it is not necessarily symmetric: Don lies to Charlie when saying e is red and thereby makes Charlie think they agree, while Charlie tells Don that e is red 3

And Albert does not even know what assertion threshold Bob has, so he could not even know that the ratio is less than the value of the threshold.

4 Indeed, since we are the programmers, we know that they disagree, so we could make Albert pick his disagreement-strategy instead of the agreement-strategy. This would, however, be best described as giving the agents telepathic abilities.

166

thereby making Don think they disagree.

21.2.1.2

Disagreement defined by communication

In the light of the above discussion, we define disagreement like this: Agents can only disagree on the value of a specific property of a specific entity. For example two agents can disagree on whether or not entity e is red. Agent A disagrees with agent B about the property p of entity e if and only if agent A has a sufficiently strong belief in a value of the property p of the entity e that he is willing to assert it and B actually asserts a value of the same property of the same entity that is different from the value A is willing to assert.

This way, disagreement is defined from the view of one specific agent. He disagrees with another agent if that other agent asserts something different from what, he is himself willing to assert.

21.2.2

Epistemic peers

We define epistemic peers like this: Agent A regards agent B as his epistemic peer if and only if agent A has roughly the same trust in agent B as agent A has in himself.

Again, note that being epistemic peers is defined from the view of one specific agent. Agent A may very well regard B as his epistemic peer, while B regards A as inferior or superior. Of course â&#x20AC;&#x2DC;roughlyâ&#x20AC;&#x2122; needs to be defined as well. This could be done when configuring the simulation.

21.2.3

Is the existing implementation conformism?

Now that we have defined both disagreement and epistemic peers, it is about time to look at a few of the challenges that pop up when implementing disagreement strategies. This first version of Knowledge Game does not implement different disagreement strategies, so we are not giving a detailed technical account of how this is implemented. Instead, we will look at suggestions and see what problems they pose. Since our existing agents do tend 167

to listen to each other and in some way conform to each other’s assertions (depending on the level of trust) it might turn out that the existing implementation could be described as some sort of conformism. Even if this turns out to be the case, we should note that the current agents do not care whether they agree or disagree with the other agent. So we should not describe the current system as implementing a certain disagreement strategy, but more of a general update strategy used when both agreeing and disagreeing. In the following, we will however focus on the specific cases where the agents disagree.

An agent, A, can either agree or disagree with another agent, B, and A can regard B as either inferior, peer or superior. This makes for six different cases which we should take into account. They are listed in Table 21.1. When we write TrustA−B we mean the amount of trust that agent A has in agent B. Note that the first three rows say ‘Do not disagree’ instead of simply ’Agree’.

This is because they include cases where the agents would assert the same thing and cases where only one of the agents has a belief strong enough that he would assert anything. We will not describe the latter cases as agreement, but on the other hand, nor are they disagreement.

1 2 3 4 5 6

TrustA−A TrustA−A TrustA−A TrustA−A TrustA−A TrustA−A

> = < > = <

TrustA−B TrustA−B TrustA−B TrustA−B TrustA−B TrustA−B

Do not disagree Do not disagree Do not disagree Disagree Disagree Disagree

Table 21.1: Peers and disagreement Since what we are concerned with is conformism as a disagreement strategy, we will in this section not worry about the cases in the first three lines.

Conformism is a strategy for updating your belief when disagreeing with a peer, so it is reasonable to begin our considerations with the situation shown in line 5 in Table 21.1. For example, imagine two agents, Ava and Boyd, and imagine that Ava trusts herself with 0.8 and she trusts Boyd equally.

In other words, Ava regards Boyd as her epistemic peer. Now, imagine that Ava is asking Boyd about the color of the entity e, and Ava’s weight-sums regarding e are as shown in table 21.2. As a final detail add that Ava has a 168

AssertionProbabilityThreshold of 0.95 which means she will only assert anything if the ratio for that value is at least 0.95.

Sum of weights

Red 0.1

Green 20.0

Blue 0.1

Yellow 0.1

Purple 0.5

Orange 0.1

Table 21.2: Weight-sums for Ava. We can quickly calculate that Ava is willing to assert that e is green: Green has the maximum weight-sum, 20.0, and the total sum of weights is 0.1 + 20.0 + 0.1 + 0.1 + 0.5 + 0.1 = 20.9, hence the ratio is 20.0/20.9 which amounts to 0.96. 0.96 is greater than 0.6, so Ava is willing to assert that e is green. And now to the disagreement: Boyd tells Ava that e is red.

What should Ava do? According to our basic probability-based strategy, Ava should simply add 0.8 (the trust she has in Boyd) to the weight-sum for red, yielding a result like shown in table 21.3.

Sum of weights

Red 0.9

Green 20.0

Blue 0.1

Yellow 0.1

Purple 0.5

Orange 0.1

Table 21.3: Weight-sums for Ava after having updated with information from Boyd using the basic propability-based update-strategy implemented in the initial version of Knowledge Game.

The new evidence will change the total sum of weights to 0.9 + 20.0 + 0.1 + 0.1 + 0.5 + 0.1 = 21,7 and thus, the ratio is changed to 20.0/21.7 which roughly equals 0.92. In other words, in this example case, Ava would no longer be willing to assert that e is green since 0.92 is less than her threshold of 0.95. This means that in a way, Ava has conformed to Boyd’s belief (or at least Boyd’s asserted belief - who knows if he is lying?). But there are several reasons that this is not as strong as the conciliationism described by David Christensen. First, even though Ava is no longer willing to assert that she believes e is green, she still believes e being green more than she believes it being red – the weight-sum for green is 20.0 versus 0.9 for red, so she does not regard the two colors equally possible or with ‘roughly equal credence’.

Secondly, just by lowering Ava’s assertion threshold, we could have her still willing to assert that e is green. In other words, using the basic probability169

based strategy, it is far from always the case that the agent will conform so much that she is no longer willing to assert her original belief.

Now, how could we make sure the agent conforms? A simple suggestion would be to simply make the weight-sum for red equal the weight-sum for green. Now that they have the same weight-sum, the agent is guaranteed to believe them equally. However, our troubles are far from over. Imagine that Boyd is telling the truth, but that he only has little evidence, for example a weight-sum of 1.0, for the color he asserts. If we are to change Ava’s weight-sum for red from 0.1 to 20.0, we add way more weight – that is, more evidence – than Boyd has ever seen for the color he asserts. This does seem quite odd, and it is most probably not what we are looking for. However, the goal of this section is not to come up with a complete implementation of disagreement strategies, but merely suggest ways and challenges. There are probably other and better ways, but there are also other challenges. Some of these are described below, but first a few word on non-conformism.

21.2.4

Non-conformism

According to non-conformism, there are at least some situations in which an agent, who disagrees with an epistemic peer, should not change his belief. It seems this is easier to implement than conformism as no change is needed.

But it does raise at least three other questions: First, if you do not change your belief when disagreeing with a peer, what should you do if you regard the agent as inferior? One suggestion would be to still just not change the belief.

But this seems to blur the difference between a peer and an inferior. This may not be a problem, but at least it needs some consideration. Another suggestion would be to actually lower one’s belief in what the (perceived) inferior says. This however seems even more odd. The second question is how to decide when the non-conformist agent should ignore the statements of his peer and when he should actually change his belief. The third question is how the agent should update in the cases where he decides to update when disagreeing with a peer. All of these questions need to be addressed before one can hope to have an implementation of non-conformism.

170

21.2.5

Updating when not disagreeing

We mentioned above that we were interested in disagreement strategies.

However, one should also consider what strategies to use when not disagreeing (the first three rows in Table 21.1). There may be good reasons to use another strategy when not disagreeing and this strategy may not be identical for conformists and non-conformists. In any case, the considerations above show that an example involving friends dining is one thing, but an actual implementation in a simulator is a completely different thing. Still, an implementation of different disagreement strategies is high on the list for a future version of knowledge game.

171

Chapter 22 Independence 22.1

What is independence?

Back in Chapter 6, we mentioned that we would discuss independence here in Chapter 22. The kind of independence we are interested in is independent evidence. This is best described by an example of evidence that is not independent. Imagine five agents, Albert, Bert, Charlie, Don and Ernie. Now imagine that Ernieâ&#x20AC;&#x2122;s abilities as an investigator leave a lot to be desired. He is trying to find out whether the entity e is red. In fact e is red, but Ernie comes to the conclusion that it is yellow. Now Don asks Ernie about the color of e and Ernie is an honest guy, so he speaks what he believes to be the truth and tells Don that e is yellow. Then Charlie comes along and asks Don what color e is. Don tells Charlie that it is yellow. Bert then asks Charlie who repeats what he got from Don. Albert is a little different from the other guys. He needs more than one piece of evidence before he will assert anything. (Either he does not trust the other ones enough or he has a high bias). So he goes to ask each of the others one by one what color e is. After asking all of them and getting the same from each one (that e is yellow), he is satisfied â&#x20AC;&#x201C; he now has enough evidence that he is willing to assert that e is yellow. This is pretty much how Knowledge Game works. However, all the evidence of e being yellow comes from one source: Ernieâ&#x20AC;&#x2122;s investigation that yielded an incorrect result. In other words, all the other sources (Bert, Charlie and Don) were directly (Don) or indirectly (Bert and Charlie) dependent on Ernie. Thus, Albert does not have any better evidence for e being yellow than had he only asked Ernie. Had both Bert, Charlie, Done and Ernie made their own investigations, we would say that the evidence Albert 172

got from each of them was independent.

22.2

Why does independence matter?

Independence matters because dependent evidence leads an agent to believe that his evidence is better than it actually is and thereby possibly leads him to draw incorrect conclusions and perform actions, he would have avoided, had he known the evidence lacked independence.

The concept of independence is as important in real life as it is in a simulator. It is easy to see how an example like the one above could be created in Knowledge Game, and it is just as easy to see how it could happen in real life.

22.3

Implementing independence

Having seen that wrong conclusions stemming from lack of independent evidence not only occurs in the simulator, but also in the real world, we should, in the terms of adequacy, think of it as a good thing: it actually seems to increase our adequacy, not decrease it. Still, it does lead to wrong conclusions. Since we would like to be able to experiment with optimal behavior and not only actual behavior, it seems natural that a future version of Knowledge Game should implement some kind of ‘dependence detection machinery’ which could be enabled for at least some of the agents. One way to go about this could be to simply track the source of each and every piece of information. Whenever an agent communicates a property, he could add who his sources were, and that way the other agent has a chance to decide if he considers that piece of information independent or not. This would be a little like giving the agents the ability to ask ‘How do you know?’ and then act accordingly when getting the answer.

173

Chapter 23 Conclusion In this thesis, we have taken on the positive project of laying the foundations for more adequate simulations in social epistemology and developing an actual software application – a simulator – for use in social epistemology.

The first part of the project was to look at existing systems for simulations in social epistemology. We looked at one in particular, namely Laputa.

Laputa unfortunately contains some very limiting assumptions in the model of the agents, so limiting that Laputa cannot serve as an adequate model for the simulations in social epistemology that we are after. Our main objections against the model of Laputa are that the doxastic comprehension of the agents are simply too limited – the agents only contain one proposition, p, and the modeling of the agents’ reaction to low trust is too strong: If the source is not trusted then information from this source is simply negated.

This assumption has a very large impact on the results generated in Laputa and thus gives us reason to doubt the fruitfulness of simulations in the system, at least in its current form.

This leads us to an analysis of the theoretical foundations of simulations in general and in social epistemology in specific. Before we could get started in developing our own simulator, we had to answer questions like ‘What is a simulation?’, ‘What can we learn from a simulation?’ and ‘What is it we want to simulate?’. These questions are all fundamental in the work with designing a simulator, since using simulations in philosophy is a very new and unexplored field. The main achievement of this part of the thesis was the insight that we need an adequate model in order to be able to learn anything from our simulations. And an adequate model is very hard to construct, when we know so little about the domain we are simulating – namely knowledge in 174

a social context. This means that even though one might be able to create a simulator and generate some results, these results most likely will only hold under all the assumptions made in the simulator, not necessarily in the real world.

Exploring the challenges of making an adequate model and generating interesting results with a simulator in social epistemology, sets us of on the path of creating our own simulator. First of all we needed a controllable and simple social process to simulate, so we developed a game â&#x20AC;&#x201C; Knowledge Game. In Knowledge Game agents compete against each other with the goal of being the first to correctly determine the truth value of a statement about a toy world. The agents can investigate the world or ask each other in order to gain the relevant knowledge. Therefore Knowledge Game is a game involving both epistemic concepts like knowledge and belief, and social interaction between agents. But of course developing the process we wanted to simulate was far from all needed in the simulator. We also needed a model of the agents and the interaction between the agents. And as a part of that, we needed a model of trust, a model of assertion, a model of inquiry, a model of investigation and also a model of how to lie. Some of this ended up as features of the agentsâ&#x20AC;&#x2122; epistemic profiles, features that the user of the system is able to control in order to create different types of agents.

In order to specify the game statement in Knowledge Game in an unambiguous way, we created a monadic predicate logic. This logic also enables the agents to reason about the entities in the world. Our monadic predicate logic enables the agents to have both complex beliefs about the world and reason about it, unlike the agents in Laputa. This logic is implemented in our software application as expression trees, and we are able to parse a sentence in monadic predicate logic to an expression tree by using a grammar. The grammar ensures that we get both operator precedence and priority right in our representation of the logical expressions in the computer.

To calculate the truth value of the actual game statement (to check if the agentsâ&#x20AC;&#x2122; answers are right or wrong) we needed a model checker. Given a model consisting of entities with certain properties (our toy world and the entities in it), we have to check whether a specified expression is true for the model. The model checker is not only used when we want to know the truth value of the game statement, it is also used by the agents to determine their belief about the truth value of the game statement based on what they know.

This work constitutes our simulator for use in social epistemology. We 175

now have simulator in which we are able to experiment with different types agents, for example liars, truth tellers, good investigators, bad investigators, agents with high trust in others, agents with low trust in others and much more. The agents in our simulator are able to reason about the world and compete against each other in gaining knowledge about the world. This simulator enabled us to run several experiments showing among other things how a stupid and outspoken agent is able to beat a smart and careful agent in our system, and how lying seems to be epistemically beneficial if your competitor is a truth teller.

Even though we have achieved quite a bit in terms of laying the foundations for more adequate simulations in social epistemology and developing an actual simulator, there are lots of work left. In part IV we have pointed to some of the interesting things left to do, for example developing a way to work with dynamic trust, implementing a patience feature and simulating peer disagreement. We have come a long way in this thesis, but there is still a lot of unexplored territory.

176

Part V Appendix

177

Appendix A Scandal of deduction The problem Hintikka calls ‘a scandal of deduction’ is that deductive inferences at first sight appear not to give us new knowledge. This is so, because the conclusion in a deductive inference is already contained in the premises.

Hintikka writes about the scandal of deduction that: [...] there is [...] [a] scandal of deduction. Its urgency can be brought home to each of us by any clever freshman who asks, upon being told that deductive reasoning is “tautological” or “analytical” and that logical truths have no “empirical content” and cannot be used to make “factual assertations”: In what other sense, then, does deductive reasoning give us new information?1 Let us have a look at an example of a deductive inference: P1 If it rains, then the street is wet P2 It rains ∴ The street is wet This is a valid deduction, because the conclusion follows from the premises P1 and P2. We cannot in anyway imagine that the premises are true and the conclusion is false. This is an example of use of the rule modus ponens, because the inference has the following form: P 1 a→b 1

[Hin70].

178

P2 a ∴ b This is a logical truth. If a → b and a are true, then the b is necessarily true. The question is, how can b be new information to us, if it is already contained in the premises and is true by necessity? This is Hintikka’s scandal of deduction.

We can ask ourselves: what is actually the problem? It seems like everyone agrees that the conclusion in a valid, deductive inference cannot be new knowledge. This however, conflicts with our intuition that we do not always know the conclusion, even though we know the premises. Puzzles in general are good examples of cases where we know the premises, but not immediately the solution. An example of such a puzzle can be found below.

It seems like the intuition about new knowledge by deduction is something that has to do with complexity. In the first example, which was just a simple modus ponens, we did not need a lot of reasoning to see that the conclusion was true. But in puzzles it is not at all obvious how the conclusion is entailed by the premises. We need to do some reasoning before we see this. In that way, the conclusion actually is new information to us.

Benthem has a similar answer to the scandal of deduction: This is a real problem when we look at logical practice. For, deduction is clearly informative. Natural tasks show a clear interplay of update through observation, or recollection, or whatever source, with more combinatorial inferential steps. When solving a puzzle, we do not just update information spaces, we also make appropriate deductions which highlight some important aspect of the solution, or at least, the road toward it.2

A.1

The asylum

Let us have a look at a puzzle by Raymond M. Smullyan3 . First he has an introduction: 2 3

[vBM07] p. 50.

[Smu82].

179

Inspector Craig of Scotland Yard was called over to France to investigate eleven insane asylums, because it was suspected that something was wrong with them. In each of these asylums, the only inhabitants were patients and doctors – the doctors constituted the only staff. Every inhabitant of an asylum (patient or doctor) is either sane or insane. Moreover, the sane ones are totally sane and one hundred percent accurate in all their beliefs; all true propositions they know to be true and all false propositions they know to be false.The insane ones are totally inaccurate in their beliefs; all true propositions they believe to be false and all false propositions they believe to be true. It is to be assumed that all inhabitants are always honest; whatever they claim, they really believe.

This background information serves, of course, as scene setting for the puzzle, but it also gives us valuable information about the premises of the puzzle. Now, let us have a look at the actual puzzle: The First Asylum. In the first asylum that Craig visited, he spoke on two separate occasions to two inhabitants whose last names were Jones and Smith.

‘Tell me,’ Craig asked Jones, ‘what do you know about Mr. Smith?’

‘You should call him Doctor Smith,’ replied Jones. ‘He is a doctor on our staff.’

Sometime later, Craig met Smith and asked, ‘What do you know about Jones? Is he a patient or a doctor?’

‘He is a patient,’ replied Smith.

Craig mulled over the situation for a while and then realized that there was indeed something wrong with this asylum: either one of the doctors there was insane, hence didn’t belong there, or worse still, one of the patients was sane and certainly shouldn’t be there.

How did Craig know this?

Now we have every premise of the puzzle and this is the information we need to solve it. But is it obvious how Craig came to his conclusion that 180

something is wrong in the asylum? No, we need to do some thinking before we can see why Craig is right.

It is sufficient to look at an informal proof, but if we wanted we could have formalized it in a suiting logic. What we need to prove is that either Jones or Smith must be either an insane doctor or a sane patient. In that case, there is indeed something wrong at the asylum.

We know that Jones is either sane or insane. If he is sane, then he has the true belief: Smith is a doctor. If Smith is insane, then he is an insane doctor and hence there is something wrong in the asylum. If Smith is sane, then he has the true belief: Jones is a patient. But then there would be something else wrong in the asylum, since we are assuming that Jones is sane. Jones would be a sane patient. Up until now we have proved that if Jones is sane, then either he is a sane patient or Smith is an insane doctor.

Let us now suppose that Jones is insane. Then he has the false belief that Smith is a doctor. Smith is therefore a patient. If Smith is sane, he would be a sane patient and there would be something wrong in the asylum. If Smith is insane, then his belief, that Jones is a patient, would be false. In that case, Jones would be an insane doctor and there would be something wrong in the asylum. This proves that if Jones is insane, then either Jones is an insane doctor or Smith is a sane patient.

This proves that either there is a sane patient or an insane doctor in the asylum and Craig was right. Now we know why Craig is right, but it took some reasoning to get there.

181

Appendix B Muddy Children Problem B.1

Introduction

The Muddy Children Problem is a logical puzzle often used as an example when introducing epistemic logic. In fact, it is so famous that it deserves at least a brief explanation even though Knowledge Game is not implemented using epistemic logic. Thus, this chapter describes the Muddy Children Problem.

B.2

The puzzle

Let us get directly to the puzzle. We can set it up like this: A man has three children called Albert, Betty and Charlie.

This is not unusual in itself, but the father is a bit unusual â&#x20AC;&#x201C; he likes to drag his children through all sorts of logical puzzles and tests. The children are unusual too â&#x20AC;&#x201C; they are unusually smart.

In fact, all this logical banter has made all the children perfect logicians. One day, the children have been out playing in the forest. When they get home they are greeted by their father who shakes his head and says: Oh, you troublemakers! Playing in the forest and getting dirty again! Well, I will give you a little test.

Stand so you can all see each other. At least one of you has mud on the forehead. When I say so, each of you 182

must try to figure out if you have mud on your own forehead. If you can figure it out, step up over to me, otherwise keep standing where you are. Let’s start – can you figure out whether you have mud on your own forehead?

None of the children move. In fact they stand completely still, looking at each other. Then after a little while, the father asks exactly the same question again: Can you figure out whether you have mud on your own forehead? Nothing happens. Again the father waits a little while and then repeats the question: Can you figure out whether you have mud on your own forehead? This time something happens! All the children quickly leave their spot and move over to the father, shouting ‘I know, I know!’ Now, the question is, how many of the children have mud on the forehead?

And how did they figure it out?1 Before we have a look at the solution, let us make a remark on its relevance for social epistemology. It should be quite clear that the puzzle is about knowledge, since the children’s goal is to obtain knowledge of the cleanliness of their foreheads. However, it may not be completely obvious where the social part comes into the picture. Nonetheless, as soon as we see the solution, it will be very clear that this puzzle is indeed very much about the social interaction between the children.

B.2.1

Solution

Let us now look at how to solve the puzzle and find out how the children figured out whether they had mud on their foreheads. We are given several relevant premises in the text of the puzzle. First of all, we should note that children are all perfect logicians, which means if and only if something can be logically inferred, they will indeed infer it. Furthermore, they are siblings, so it is safe to assume that they all know that they are all perfect logicians and that they all know this, and so on. This means that each child can reason about what the other children know.

1

This exact formulation of the puzzle is our own, but it is inspired by the version found in [FHMV95].

183

B.2.1.1

Assuming one dirty child

We will solve the puzzle by first assuming that only one child has mud on the forehead. Let us just say that Albert is the one with mud on the forehead.

Making this assumption, we will ask ourselves what Charlie will do when the father has asked the question once. The answer is that he will look at Albert and Betty and see both a muddy forehead and clean one. The father said at least one forehead was muddy. Charlie can see Albert’s dirty forehead, so he knows that the father’s statement is true, no matter if he himself has mud on the forehead or not. In other words, he cannot figure out whether he is clean or dirty. Now, what about Betty? She sees Albert’s dirty forehead and Charlie’s clean one. In other words, her situation is identical to Charlie’s, so she does not know either. For Albert, however, the situation is different. He sees only two clean foreheads. For the father’s statement to be true, Albert must himself be dirty, so assuming that only Albert is dirty, he can actually know whether he is dirty or not. This, of course, works for each of the other children as well: If only one of the children is dirty that child can puzzle out whether he or she is dirty. Having realized this, we must of course cancel our initial assumption that only one child was dirty, since none of the children could in fact figure out if they were dirty.

B.2.1.2

Assuming two dirty children

Now let us try assuming that two of the children are dirty. Say that the dirty ones are Albert and Betty. Again, we will think through how each of the children can reason. Let us once again start with Charlie. He sees two dirty foreheads which ensure the father’s statement is true, no matter if Charlie himself is dirty or not. Betty sees only one dirty forehead (remember, she cannot see her own), but that is enough to make the father’s statement true.

In other words, she cannot know whether she is dirty. Albert also sees only one dirty forehead, so he is in the same situation as Betty and thus cannot know about his own forehead. Summing up, none of the children can figure out whether they are dirty and hence they all stand still.

Now the father asks again: Can you figure out whether you have mud on your own forehead? One might think that this is pretty dumb. After all, the father has just asked the question once and none of the children could answer, so what difference should it make that he asks again? As it turns out, it makes all the difference in the world. Let us continue our investigation 184

into how the children can reason. We know by now that if only one of the children had been dirty that child had been able to figure out the state of his forehead. Since none of the children moved after the father asked the first time, each of them now know that there is more than one dirty child. Again let us start by seeing how Charlie reasons. He now knows that at least two foreheads must be dirty, but he also sees two dirty foreheads, so he cannot use this to conclude whether his own forehead must be dirty. Betty is in a different situation. She also knows (by the fact that none of the children moved after the father asked the first time) that at least two children are dirty, but she only sees one dirty child. The only way there can be at least two dirty children is if she is dirty. Albertâ&#x20AC;&#x2122;s situation is identical to Bettyâ&#x20AC;&#x2122;s, so he can also know. Thus, assuming two children are dirty, the two dirty ones will be able to know that they are dirty after the father asks twice.

However, as we have stated the puzzle, none of the children knows until after the father asks three times, so we can conclude that our assumption about exactly two dirty children was wrong.

B.2.1.3

Assuming three dirty children

Now given that there are only three children, that at least one is dirty, and that we have just seen that we cannot have exactly one or exactly two dirty children, we can simply conclude that all three children must be dirty, since there is no other way to make the premises true. However, we can also realize this by continuing our line of reasoning. First, the father asks for the third time: Can you figure out whether you have mud on your own forehead? All the children see the same thing: two dirty foreheads. Since they know that had there only been two dirty foreheads, the two dirty ones would have known after the father had asked only twice, they can conclude that there must be at least three dirty children. When a child, Charlie for instance, looks at the two other children and sees two dirty foreheads, he knows that there can only be three dirty ones if his own is dirty too. This, of course, goes for all the children, so they will indeed all be able to figure out that they are dirty.

B.2.1.4

The social part

The fact that the children are able to answer the question, just by having the father repeat the question, is indeed quite amazing. And it is also what makes the example relevant for social epistemology. One might think that 185

there is nothing social about this example, since it seems the children are not communicating or interacting in any way. In fact, they are very much communicating and this communication is exactly what enables them to answer the question: Each time the father asks the question, they communicate something by not communicating anything â&#x20AC;&#x201C; they effectively say I cannot make out whether my forehead is dirty and knowing that the other children could not know is exactly what enables the dirty ones to know.

B.2.2

General problem and solution

The problem can be generalized to any number of children and any number of muddy foreheads and it can be shown that if there are n muddy foreheads, the father must ask the question n times before the muddy children will know that they are muddy.

B.3

Further reading

The problem is known under many other names, for example Josephineâ&#x20AC;&#x2122;s Problem and Cheating Husbands Problem. See [Wik12c] for more information on variations and other similar problems. For a famous and detailed discussion of the problem and reasoning about knowledge in general, see [FHMV95]. A brief and slightly more mathematical approach can be found in [Mis98]. For a more general approach to modal logic see [Gar06].

186

Bibliography [AF11]

Sergei Artemov and Melvin Fitting. Justification Logic. Stanford Ecyclopedia of Philosophy, 2011.

[Ang10]

Staffan Angere. Knowledge in a Social Network. page 24, 2010.

[Boj97]

Damjan Bojadziev. Mind versus gödel. In Mind Versus Computer. IOS Press, 1997.

[BS09]

Alexandru Baltag and Sonja Smets. Dynamic Logics for Interactive Belief Revision. In ESSLLI, 2009.

[Cha]

David Chalmers. Minds, Machines, And Mathematics Minds, Machines, And Mathematics – A Review of Shadows of the Mind

by Roger Penrose.

[Chr07]

David Christensen. The Epistemology of Disagreement:The Good News. Philosophical Review, pages 187–217, 2007.

[Chr09]

David Christensen. Disagreement as Evidence: The Epistemology of Controversy. Philosophy Compass, pages 756–767, 2009.

[Con63]

R. W. Conway. Some tactical problems in digital simulation.

Management Science, 10(1):47 – 61, 1963.

[Dou09]

Igor Douven. Introduction: Computer simulations in social epistemology. Episteme, 2009.

[Dum78]

Micheal Dummett. Truth and Other Enigmas, chapter 12. The Philosophical Significance of Gödel’s Theorems. Harvard University Press, 1978.

187

[EH07]

Lisa Eriksson and Alan Hájek. What Are Degrees of Belief. Studia Logica: An International Journal for Symbolic Logic, 86(2):183 – 213, 2007.

[Elg07]

Adam Elga. Reflection and disagreement. Noûs, pages 478–502, 2007.

[Fel06]

Richard Feldman. Epistemology Futures, chapter Epistemological Puzzles about Disagreement. Oxford: Clarendon Press, 2006.

[Fel07]

Richard Feldman. Philosophers without Gods, chapter Reasonable Religious Disagreements. Oxford University Press, 2007.

[FHMV95] Ronald Fagin, Joseph Y. Halpern, Yoram Moses, and Moshe Y. Vardi. Reasoning About Knowledge. The MIT Press, 1995.

[Fit94]

Melvin Fitting. Kleene’s three valued logics and their children.

Fundam. Inf., 20:113–131, March 1994.

[Fol91]

Richard Foley. Rationality. Synthese, 89(3):365 – 392, 1991.

[Fum10]

Richard Fumerton. Foundationalist Theories of Epistemic Justification. Stanford Ecyclopedia of Philosophy, 2010.

[Gai]

Haim Gaifman. What Gödel’s Incompleteness Result Does and Does Not Show. Philosophy, pages 1–7.

[Gar06]

James W. Garson. Modal Logic for Philosophers. Cambridge, 2006.

[Get63]

Edmund Gettier. Is Justified True Belief Knowledge? Analysis, 1963.

[Göd57]

Kurt Gödel. On Formally Undecidable Propositions of Principia Mathematica and Related Systems. Basic Books, Inc., 1962, normalsider: 57.

[Gol99]

Alvin Goldman. Knowledge in a Social World. Oxford University Press, 1999.

[Gol01]

Alvin Goldman. Social Epistemology. Stanford Ecyclopedia of Philosophy, 2001.

188

[Hae09]

Rolf Haenni. Non-Additive Degrees of Belief. Degrees of Belief, 342:121 – 159, 2009.

[Haw05]

James Hawthorne. Degree-of-Belief and Degree-of-Support: Why Bayesians Need Both Notions. Mind, 114(454), 2005.

[Her91]

P. J. Herskovitz. A theoretical framework for simulation validation: Popper’s falsificationism. International Journal of Modelling and Simulation, 11:56 – 58, 1991.

[Hin70]

Jaakoo Hintikka. Information, Deduction, and the A Priori.

Noûs, Blackwell Publishing, page normalsider: 19, 1970.

[HK02]

Rainer Hegselmann and Ulrich Krause. Opinion Dynamics and Bounded Confidence Models, Analysis, and Simulation. Journal of Artifical Societies and Social Simulation, 5(3), 2002.

[HMS09]

Stephan Hartmann, Carlo Martini, and Jan Sprenger. Consensual Decision-making Among Epistemic Peers. Episteme, 2009.

[How09]

Colin Howson. Epistemic Probability and Coherent Degrees of Belief. Degrees of Belief, 2009.

[Hub09]

Franz Huber. Belief and Degrees of Belief. Synthese Library, pages 1 – 43, 2009.

[Jr.06]

Henry E. Kyburg Jr. Belief, Evidence and Conditioning. Philosophy of Science, 73(1):42 – 65, 2006.

[Kel05]

Thomas Kelly. Oxford Studies in Epistemology 1., chapter The Epistemic Significance of Disagreement. Oxford University Press, 2005.

[Kel08]

Thomas Kelly. Disagreement, Dogmatism and Belief Polarization. The Journal of Philosophy, CV(10), 2008.

[KH05]

Esther E. Klein and Paul J. Herskovitz. Philosophical foundations of computer simulation validation. Simulation & Gaming, 36:303 – 329, 2005.

[Lac10a]

Jennifer Lackey. Social Epistemology, chapter What Should We Do When We Disagree? Oxford University Press, 2010.

189

[Lac10b]

Jennifer Lackey. Social Epistemology, chapter A Justificationist View of Disagreement’s Epistemic Significance. Oxford University Press, 2010.

[Lon06]

Helen Longino. The Social Dimensions of Scientific Knowledge.

Stanford Ecyclopedia of Philosophy, 2006.

[Luc63]

J. R. Lucas. Minds, Machines and Gödel. Philosophy, pages 112–37, normalsider: 21, 1963.

[LW81]

Keith Lehrer and Carl Wagner. Rational Consensus in science and society. Springer Science and Business, 1981.

[McL11]

Carolyn McLeod. Trust. Stanford Encyclopedia of Pholosophy, 2011.

[Mil10]

Alexander Miller. Realism. Stanford Ecyclopedia of Philosophy, 2010.

[Mis98]

Jayadev Misra. The muddy children puzzle. http://www.cs. utexas.edu/users/misra/Notes.dir/MuddyChildren.pdf, 1998. [Online; accessed 15-January-2012].

[Moo65]

G. E. Moore. Cramming more components onto integrated circuits. Electronics, 38(8):114 – 117, 1965.

[OD03]

Graham Oppy and David Dowe. The Turing Test, 2003.

[Ols10]

Erik J. Olsson. 2nd copenhagen conference in epistemology.

In The Epistemology of Group Polarization: A Simulation Approach. Social Epistemology Research Group, August 2010.

[Ols11]

Erik J. Olsson. A Simulation Approach. Will be published in Episteme, 2011.

[Pas10]

Waldemar Dos Passos. Numerical Methods, Algorithms and Tools in C#, chapter 10, pages 283 – 382. CRC Press, 2010.

[Pen56]

Roger Penrose. The Emperor’s New Mind. Oxford University Press, 1989, normailsider: 456.

190

[PM88]

Stephen K. Park and Keith W. Miller. Random number generators: Good ones are hard to find. Communications of the ACM, 31(10):1192 – 1201, 1988.

[Ros01]

Gideon Rosen. Nominalism, Naturalism, Epistemic Relativism.

Philosophical Perspectives, pages 60–91, 2001.

[Rot09]

Hans Rott. Degrees all the way down: Beliefs, non-beliefs and disbeliefs. Degrees of Belief, 2009.

[Sea90]

John Searle. Is the Brain’s Mind a Computer Program? Scientific American, 1990.

[Smu46]

R. M. Smullyan. Gödel’s Incomleteness Theorems. Oxford University Press, 1992, normalsider: 146.

[Smu82]

Raymond M. Smullyan. The Lady or the Tiger. Dover Publications, 1982.

[Ste06]

Matthias Steup. The Analysis of Knowledge. Stanford Ecyclopedia of Philosophy, 2006.

[Sun02]

Cass R. Sunstein. The Law of Group Polarization. Journal of Political Philosophy, 10(2):175–195, 2002.

[Sun09]

Cass R. Sunstein. Going to extremes - how like minds unite and divide. page 199, 2009.

[vB08]

Johan van Benthem. Merging Observation and Acces in Dynamic Logics. 2008.

[vBM07]

Johan van Benthem and Maricarmen Martinez. The Stories of Logic and Information. 2007.

[Wed07]

Ralph Wedgewood. The Nature of Normativity. Oxford University Press, 2007.

[Wik11]

Wikipedia. Context-sensitive language — wikipedia, the free encyclopedia. http://en.wikipedia.org/w/index.php?title= Context-sensitive_language&oldid=441323641, 2011. [Online; accessed 23-January-2012].

191

[Wik12a]

Wikipedia. Conjunction fallacy — wikipedia, the free encyclopedia. http://en.wikipedia.org/wiki/Conjunction_fallacy, 2012. [Online; accessed 3-March-2012].

[Wik12b]

Wikipedia. Context-free language — wikipedia, the free encyclopedia. http://en.wikipedia.org/w/index.php?title= Context-free_language&oldid=470655559, 2012. [Online; accessed 23-January-2012].

[Wik12c]

Wikipedia.

Induction puzzles — wikipedia, the free encyclopedia. http://en.wikipedia.org/w/index.php?title= Induction_puzzles&oldid=470206512, 2012. [Online; accessed 15-January-2012].

[Wik12d]

Wikipedia. Wason selection task — wikipedia, the free encyclopedia. http://en.wikipedia.org/w/index.php?title= Wason_selection_task&oldid=471751545, 2012. [Online; accessed 19-January-2012].

192