Lecture solurion manual manual programming language pragmatics fourth edition michael scott complete

Lecture solution manual

Chapter 1 :: Introduction

Programming Language Pragmatics, Fourth Edition Michael L. Scott

Copyright © 2016 Elsevier

Introduction

• Why are there so many programming languages?

– evolution -- we've learned better ways of doing things over time – socio-economic factors: proprietary interests, commercial advantage – orientation toward special purposes – orientation toward special hardware – diverse ideas about what is pleasant to use

Introduction

• What makes a language successful?

– easy to learn (BASIC, Pascal, LOGO, Scheme) – easy to express things, easy use once fluent, "powerful” (C, Common Lisp, APL, Algol-68, Perl) – easy to implement (BASIC, Forth) – possible to compile to very good (fast/small) code (Fortran) – backing of a powerful sponsor (COBOL, PL/1, Ada, Visual Basic) – wide dissemination at minimal cost (Pascal, Turing, Java)

Introduction

• Why do we have programming languages?

What is a language for?

– way of thinking -- way of expressing algorithms – languages from the user's point of view – abstraction of virtual machine -- way of specifying what you want – the hardware to do without getting down into the bits – languages from the implementor's point of view

Why study programming languages?

• Help you choose a language.

– C vs. C++ vs. C# for systems programming – Fortran vs. C for numerical computations – PHP vs. Ruby for web-based applications – Ada vs. C for embedded systems – Common Lisp vs. Scheme vs. ML for symbolic data manipulation – Java vs. .NET for networked PC programs

Why study programming languages?

• Make it easier to learn new languages.

• Some languages are similar; easy to walk down family tree.

– concepts have even more similarity; if you think in terms of iteration, recursion, abstraction (for example), you will find it easier to assimilate the syntax and semantic details of a new language.

– Think of an analogy to human languages: good grasp of grammar makes it easier to pick up new languages (at least Indo-European).

Why study programming languages?

• Help you make better use of whatever language you use – understand obscure features: • In C, help you understand unions, arrays & pointers, separate compilation, varargs, catch and throw • In Common Lisp, help you understand first-class functions/closures, streams, catch and throw, symbol internals

Why study programming languages?

• Help you make better use of whatever language you use (2) – understand implementation costs: choose between alternative ways of doing things, based on knowledge of what will be done underneath: – use simple arithmetic equal (use x*x instead of x**2) – use C pointers or Pascal "with" statement to factor address calculations – avoid call by value with large data items in Pascal – avoid the use of call by name in Algol 60 – choose between computation and table lookup (e.g. for cardinality operator in C or C++)

Why study programming languages?

• Help you make better use of whatever language you use (3) – figure out how to do things in languages that don't support them explicitly: • lack of suitable control structures in Fortran • use comments and programmer discipline for control structures • lack of recursion in Fortran, CSP, etc • write a recursive algorithm then use mechanical recursion elimination (even for things that aren't quite tail recursive)

Why study programming languages?

• Help you make better use of whatever language you use (4) – figure out how to do things in languages that don't support them explicitly: – lack of named constants and enumerations in Fortran – use variables that are initialized once, then never changed – lack of modules in C and Pascal use comments and programmer discipline – lack of iterators in just about everything fake them with (member?) functions

Imperative languages

• Group languages as – imperative • von Neumann

(Fortran, Pascal, Basic, C)

• object-oriented

(Smalltalk, Eiffel, C++?)

• scripting languages

(Perl, Python, JavaScript, PHP)

– declarative • functional

(Scheme, ML, pure Lisp, FP)

• logic, constraint-based

(Prolog, VisiCalc, RPG)

Imperative languages

• Imperative languages, particularly the von Neumann languages, predominate – They will occupy the bulk of our attention

• We also plan to spend a lot of time on functional, logic languages

Compilation vs. Interpretation

• Compilation vs. interpretation – not opposites – not a clear-cut distinction

• Pure Compilation – The compiler translates the high-level source program into an equivalent target program (typically in machine language), and then goes away:

Compilation vs. Interpretation

• Pure Interpretation – Interpreter stays around for the execution of the program – Interpreter is the locus of control during execution

Compilation vs. Interpretation

• Interpretation: – Greater flexibility – Better diagnostics (error messages)

• Compilation – Better performance

Compilation vs. Interpretation

• Common case is compilation or simple preprocessing, followed by interpretation • Most language implementations include a mixture of both compilation and interpretation

Compilation vs. Interpretation • Note that compilation does NOT have to produce machine language for some sort of hardware • Compilation is translation from one language into another, with full analysis of the meaning of the input • Compilation entails semantic understanding of what is being processed; pre-processing does not • A pre-processor will often let errors through. A compiler hides further steps; a pre-processor does not

Compilation vs. Interpretation

• Many compiled languages have interpreted pieces, e.g., formats in Fortran or C • Most use “virtual instructions” – set operations in Pascal – string manipulation in Basic

• Some compilers produce nothing but virtual instructions, e.g., Pascal P-code, Java byte code, Microsoft COM+

Compilation vs. Interpretation

• Implementation strategies: – Preprocessor • Removes comments and white space • Groups characters into tokens (keywords, identifiers, numbers, symbols) • Expands abbreviations in the style of a macro assembler • Identifies higher-level syntactic structures (loops, subroutines)

Compilation vs. Interpretation

• Implementation strategies: – Library of Routines and Linking • Compiler uses a linker program to merge the appropriate library of subroutines (e.g., math functions such as sin, cos, log, etc.) into the final program:

Compilation vs. Interpretation

• Implementation strategies: – Post-compilation Assembly • Facilitates debugging (assembly language easier for people to read) • Isolates the compiler from changes in the format of machine language files (only assembler must be changed, is shared by many compilers)

Compilation vs. Interpretation

• Implementation strategies: – The C Preprocessor (conditional compilation) • Preprocessor deletes portions of code, which allows several versions of a program to be built from the same source

Compilation vs. Interpretation

• Implementation strategies: – Source-to-Source Translation (C++) • C++ implementations based on the early AT&T compiler generated an intermediate program in C, instead of an assembly language:

Compilation vs. Interpretation

• Implementation strategies: – Source-to-Source Translation (C++) • C++ implementations based on the early AT&T compiler generated an intermediate program in C, instead of an assembly language:

Compilation vs. Interpretation

• Implementation strategies: – Bootstrapping

Compilation vs. Interpretation

• Implementation strategies: – Compilation of Interpreted Languages • The compiler generates code that makes assumptions about decisions that won’t be finalized until runtime. If these assumptions are valid, the code runs very fast. If not, a dynamic check will revert to the interpreter.

Compilation vs. Interpretation

• Implementation strategies: – Dynamic and Just-in-Time Compilation • In some cases a programming system may deliberately delay compilation until the last possible moment.

– Lisp or Prolog invoke the compiler on the fly, to translate newly created source into machine language, or to optimize the code for a particular input set.

– The Java language definition defines a machine-independent intermediate form known as byte code. Byte code is the standard format for distribution of Java programs.

– The main C# compiler produces .NET Common Intermediate Language (CIL), which is then translated into machine code immediately prior to execution.

Compilation vs. Interpretation

• Implementation strategies: – Microcode • Assembly-level instruction set is not implemented in hardware; it runs on an interpreter.

• Interpreter is written in low-level instructions (microcode or firmware), which are stored in readonly memory and executed by the hardware.

Compilation vs. Interpretation • Compilers exist for some interpreted languages, but they aren't pure: – selective compilation of compilable pieces and extrasophisticated pre-processing of remaining source.

– Interpretation of parts of code, at least, is still necessary for reasons above.

• Unconventional compilers – text formatters – silicon compilers – query language processors

Programming Environment Tools

• Tools

An Overview of Compilation

• Phases of Compilation

An Overview of Compilation

• Scanning: – divides the program into "tokens", which are the smallest meaningful units; this saves time, since character-by-character processing is slow – we can tune the scanner better if its job is simple; it also saves complexity (lots of it) for later stages – you can design a parser to take characters instead of tokens as input, but it isn't pretty – scanning is recognition of a regular language, e.g., via DFA

An Overview of Compilation

• Parsing is recognition of a context-free language, e.g., via PDA – Parsing discovers the "context free" structure of the program – Informally, it finds the structure you can describe with syntax diagrams (the "circles and arrows" in a Pascal manual)

An Overview of Compilation

• Semantic analysis is the discovery of meaning in the program – The compiler actually does what is called STATIC semantic analysis. That's the meaning that can be figured out at compile time – Some things (e.g., array subscript out of bounds) can't be figured out until run time. Things like that are part of the program's DYNAMIC semantics

An Overview of Compilation

• Intermediate form (IF) done after semantic analysis (if the program passes all checks) – IFs are often chosen for machine independence, ease of optimization, or compactness (these are somewhat contradictory) – They often resemble machine code for some imaginary idealized machine; e.g. a stack machine, or a machine with arbitrarily many registers – Many compilers actually move the code through more than one IF

An Overview of Compilation

• Optimization takes an intermediate-code program and produces another one that does the same thing faster, or in less space – The term is a misnomer; we just improve code – The optimization phase is optional

• Code generation phase produces assembly language or (sometime) relocatable machine language

An Overview of Compilation

• Certain machine-specific optimizations (use of special instructions or addressing modes, etc.) may be performed during or after target code generation • Symbol table: all phases rely on a symbol table that keeps track of all the identifiers in the program and what the compiler knows about them – This symbol table may be retained (in some form) for use by a debugger, even after compilation has completed

An Overview of Compilation

• Lexical and Syntax Analysis – GCD Program (in C) int main() { int i = getint(), j = getint(); while (i != j) { if (i > j) i = i - j; else j = j - i; } putint(i); }

An Overview of Compilation

• Lexical and Syntax Analysis – GCD Program Tokens • Scanning (lexical analysis) and parsing recognize the structure of the program, groups characters into tokens, the smallest meaningful units of the program int int while if else } putint }

main i ( ( j

( = i i =

) getint != > j

{ ( j j -

(

i

)

;

) ) ) i

, { i ;

j

=

getint

(

)

=

i

-

j

;

;

An Overview of Compilation

• Lexical and Syntax Analysis – Context-Free Grammar and Parsing • Parsing organizes tokens into a parse tree that represents higher-level constructs in terms of their constituents • Potentially recursive rules known as context-free grammar define the ways in which these constituents combine

An Overview of Compilation

• Context-Free Grammar and Parsing – Example (while loop in C) iteration-statement → while ( expression ) statement statement, in turn, is often a list enclosed in braces: statement → compound-statement compound-statement → { block-item-list opt } where block-item-list opt → block-item-list or block-item-list opt → ϵ and block-item-list → block-item block-item-list → block-item-list block-item block-item → declaration block-item → statement

An Overview of Compilation

• Context-Free Grammar and Parsing – GCD Program Parse Tree

B A next slide

An Overview of Compilation

• Context-Free Grammar and Parsing (continued)

An Overview of Compilation

• Context-Free Grammar and Parsing (continued) A

B

An Overview of Compilation

• Syntax Tree – GCD Program Parse Tree

Chapter 2 :: Programming Language Syntax

Programming Language Pragmatics, Fourth Edition Michael L. Scott

Copyright © 2016 Elsevier

Regular Expressions

• A regular expression is one of the following: – A character – The empty string, denoted by  – Two regular expressions concatenated – Two regular expressions separated by | (i.e., or) – A regular expression followed by the Kleene star * (concatenation of zero or more strings)

Regular Expressions

• Numerical constants accepted by a simple hand-held calculator:

Context-Free Grammars

• The notation for context-free grammars (CFG) is sometimes called Backus-Naur Form (BNF) • A CFG consists of – A set of terminals T – A set of non-terminals N – A start symbol S (a non-terminal) – A set of productions

Context-Free Grammars

• Expression grammar with precedence and associativity

Context-Free Grammars

• In this grammar, generate the string "slope * x + intercept"

Context-Free Grammars

• Parse tree for expression grammar for "slope * x + intercept"

Context-Free Grammars

• Alternate (Incorrect) Parse tree for "slope * x + intercept" • Our grammar is ambiguous

Context-Free Grammars

• A better version because it is unambiguous and captures precedence

Context-Free Grammars

• Parse tree for expression grammar (with left associativity) for 3 + 4 * 5

Scanning

• Recall scanner is responsible for – tokenizing source – removing comments – (often) dealing with pragmas (i.e., significant comments) – saving text of identifiers, numbers, strings – saving source locations (file, line, column) for error messages

Scanning • Suppose we are building an ad-hoc (handwritten) scanner for Pascal: – We read the characters one at a time with look-ahead

• If it is one of the one-character tokens { ( ) [ ] < > , ; = + - etc } we announce that token • If it is a ., we look at the next character – If that is a dot, we announce .

– Otherwise, we announce . and reuse the look-ahead

Scanning

• If it is a <, we look at the next character – if that is a = we announce <= – otherwise, we announce < and reuse the lookahead, etc

• If it is a letter, we keep reading letters and digits and maybe underscores until we can't anymore – then we check to see if it is a reserve word

Scanning

• If it is a digit, we keep reading until we find a non-digit – if that is not a . we announce an integer – otherwise, we keep looking for a real number – if the character after the . is not a digit we announce an integer and reuse the . and the look-ahead

Scanning

• Pictorial representation of a scanner for calculator tokens, in the form of a finite automaton

Scanning

• This is a deterministic finite automaton (DFA) – Lex, scangen, etc. build these things automatically from a set of regular expressions – Specifically, they construct a machine that accepts the language identifier | int const | real const | comment | symbol | ...

Scanning

• We run the machine over and over to get one token after another – Nearly universal rule: • always take the longest possible token from the input thus foobar is foobar and never f or foo or foob • more to the point, 3.14159 is a real const and never 3, ., and 14159

• Regular expressions "generate" a regular language; DFAs "recognize" it

Scanning

• Scanners tend to be built three ways – ad-hoc – semi-mechanical pure DFA (usually realized as nested case statements) – table-driven DFA

• Ad-hoc generally yields the fastest, most compact code by doing lots of specialpurpose things, though good automaticallygenerated scanners come very close

Scanning

• Writing a pure DFA as a set of nested case statements is a surprisingly useful programming technique – though it's often easier to use perl, awk, sed – for details see Figure 2.11

• Table-driven DFA is what lex and scangen produce – lex (flex) in the form of C code – scangen in the form of numeric tables and a separate driver (for details see Figure 2.12)

Scanning • Note that the rule about longest-possible tokens means you return only when the next character can't be used to continue the current token – the next character will generally need to be saved for the next token

• In some cases, you may need to peek at more than one character of look-ahead in order to know whether to proceed – In Pascal, for example, when you have a 3 and you a see a dot • do you proceed (in hopes of getting 3.14)? or • do you stop (in fear of getting 3..5)?

Scanning

• In messier cases, you may not be able to get by with any fixed amount of look-ahead.In Fortr an, for example, we have DO 5 I = 1,25 loop DO 5 I = 1.25 assignment • Here, we need to remember we were in a potentially final state, and save enough information that we can back up to it, if we get stuck later

Parsing

• Terminology: – context-free grammar (CFG) – symbols • terminals (tokens) • non-terminals

– production – derivations (left-most and right-most - canonical) – parse trees – sentential form

Parsing

• By analogy to RE and DFAs, a context-free grammar (CFG) is a generator for a context-free language (CFL) – a parser is a language recognizer

• There is an infinite number of grammars for every context-free language – not all grammars are created equal, however

Parsing

• It turns out that for any CFG we can create a parser that runs in O(n^3) time • There are two well-known parsing algorithms that permit this – Early's algorithm – Cooke-Younger-Kasami (CYK) algorithm

• O(n^3) time is clearly unacceptable for a parser in a compiler - too slow

Parsing

• Fortunately, there are large classes of grammars for which we can build parsers that run in linear time – The two most important classes are called LL and LR

• LL stands for 'Left-to-right, Leftmost derivation'.

• LR stands for 'Left-to-right, Rightmost derivation’

Parsing

• LL parsers are also called 'top-down', or 'predictive' parsers & LR parsers are also called 'bottom-up', or 'shift-reduce' parsers • There are several important sub-classes of LR parsers – SLR – LALR

• We won't be going into detail on the differences between them

Parsing

• Every LL(1) grammar is also LR(1), though right recursion in production tends to require very deep stacks and complicates semantic analysis • Every CFL that can be parsed deterministically has an SLR(1) grammar (which is LR(1)) • Every deterministic CFL with the prefix property (no valid string is a prefix of another valid string) has an LR(0) grammar

Parsing

• You commonly see LL or LR (or whatever) written with a number in parentheses after it – This number indicates how many tokens of look-ahead are required in order to parse – Almost all real compilers use one token of look-ahead

• The expression grammar (with precedence and associativity) you saw before is LR(1), but not LL(1)

LL Parsing •

Here is an LL(1) grammar (Fig 2.15):

1.

2.

3.

4.

5.

6.

7.

8.

9.

program stmt_list

→ stmt list $$$ → stmt stmt_list | ε stmt → id := expr | read id | write expr expr → term term_tail term_tail → add op term term_tail | ε

LL Parsing •

LL(1) grammar (continued)

10. term → factor fact_tailt 11. fact_tail → mult_op fact fact_tail 12. | ε 13. factor → ( expr ) 14. | id 15. | number 16. add_op → + 17. | 18. mult_op → * 19. | /

LL Parsing

• Like the bottom-up grammar, this one captures associativity and precedence, but most people don't find it as pretty – for one thing, the operands of a given operator aren't in a RHS together!

– however, the simplicity of the parsing algorithm makes up for this weakness

• How do we parse a string with this grammar?

– by building the parse tree incrementally

LL Parsing

• Example (average program) read A read B sum := A + B write sum write sum / 2 • We start at the top and predict needed productions on the basis of the current left-most non-terminal in the tree and the current input token

LL Parsing

• Parse tree for the average program (Figure 2.18)

LL Parsing

• Table-driven LL parsing: you have a big loop in which you repeatedly look up an action in a two-dimensional table based on current leftmost non-terminal and current input token. The actions are (1) match a terminal (2) predict a production (3) announce a syntax error

LL Parsing

• LL(1) parse table for parsing for calculator language

LL Parsing

• To keep track of the left-most non-terminal, you push the as-yet-unseen portions of productions onto a stack – for details see Figure 2.21

• The key thing to keep in mind is that the stack contains all the stuff you expect to see between now and the end of the program – what you predict you will see

LL Parsing

• Problems trying to make a grammar LL(1) – left recursion • example:

id_list → id | id_list , id equivalently

id_list → id id_list_tail id_list_tail → , id id_list_tail | epsilon • we can get rid of all left recursion mechanically in any grammar

LL Parsing

• Problems trying to make a grammar LL(1) – common prefixes: another thing that LL parsers can't handle • solved by "left-factoring” • example: stmt → id := expr | id ( arg_list ) equivalently stmt → id id_stmt_tail id_stmt_tail → := expr | ( arg_list) • we can eliminate left-factor mechanically

LL Parsing

• Note that eliminating left recursion and common prefixes does NOT make a grammar LL – there are infinitely many non-LL LANGUAGES, and the mechanical transformations work on them just fine – the few that arise in practice, however, can generally be handled with kludges

LL Parsing

• Problems trying to make a grammar LL(1) – the"dangling else" problem prevents grammars from being LL(1) (or in fact LL(k) for any k) – the following natural grammar fragment is ambiguous (Pascal) stmt → if cond then_clause else_clause | other_stuff then_clause → then stmt else_clause → else stmt | epsilon

LL Parsing

• The less natural grammar fragment can be parsed bottom-up but not top-down stmt → balanced_stmt | unbalanced_stmt balanced_stmt → if cond then balanced_stmt else balanced_stmt | other_stuff unbalanced_stmt → if cond then stmt | if cond then balanced_stmt else unbalanced_stmt

LL Parsing

• The usual approach, whether top-down OR bottom-up, is to use the ambiguous grammar together with a disambiguating rule that says – else goes with the closest then or – more generally, the first of two possible productions is the one to predict (or reduce)

LL Parsing • Better yet, languages (since Pascal) generally employ explicit end-markers, which eliminate this problem • In Modula-2, for example, one says: if A = B then if C = D then E := F end else G := H end

• Ada says 'end if'; other languages say 'fi'

LL Parsing • One problem with end markers is that they tend to bunch up. In Pascal you say if A = B then … else if A = C then … else if A = D then … else if A = E then … else ...;

• With end markers this becomes if A = B then … else if A = C then … else if A = D then … else if A = E then … else ...; end; end; end; end;

LL Parsing

• The algorithm to build predict sets is tedious (for a "real" sized grammar), but relatively simple • It consists of three stages: – (1) compute FIRST sets for symbols – (2) compute FOLLOW sets for non-terminals (this requires computing FIRST sets for some strings) – (3) compute predict sets or table for all productions

LL Parsing • It is conventional in general discussions of grammars to use – lower case letters near the beginning of the alphabet for terminals – lower case letters near the end of the alphabet for strings of terminals – upper case letters near the beginning of the alphabet for non-terminals – upper case letters near the end of the alphabet for arbitrary symbols – greek letters for arbitrary strings of symbols

LL Parsing • Algorithm First/Follow/Predict: – FIRST(α) == {a : α →* a β} ∪ (if α =>* ε THEN {ε} ELSE NULL)

– FOLLOW(A) == {a : S →+ α A a β} ∪ (if S →* α A THEN {ε} ELSE NULL) – Predict (A → X1 ... Xm) == (FIRST (X1 ...

Xm) - {ε}) ∪ (if X1, ..., Xm →* ε then FOLLOW (A) ELSE NULL)

• Details following…

LL Parsing

LL Parsing

LL Parsing

• If any token belongs to the predict set of more than one production with the same LHS, then the grammar is not LL(1) • A conflict can arise because – the same token can begin more than one RHS – it can begin one RHS and can also appear after the LHS in some valid program, and one possible RHS is ε

LR Parsing

• LR parsers are almost always table-driven: – like a table-driven LL parser, an LR parser uses a big loop in which it repeatedly inspects a twodimensional table to find out what action to take – unlike the LL parser, however, the LR driver has non-trivial state (like a DFA), and the table is indexed by current input token and current state – the stack contains a record of what has been seen SO FAR (NOT what is expected)

LR Parsing

• A scanner is a DFA – it can be specified with a state diagram

• An LL or LR parser is a PDA – Early's & CYK algorithms do NOT use PDAs – a PDA can be specified with a state diagram and a stack • the state diagram looks just like a DFA state diagram, except the arcs are labeled with <input symbol, top-ofstack symbol> pairs, and in addition to moving to a new state the PDA has the option of pushing or popping a finite number of symbols onto/off the stack

LR Parsing

• An LL(1) PDA has only one state!

– well, actually two; it needs a second one to accept with, but that's all (it's pretty simple) – all the arcs are self loops; the only difference between them is the choice of whether to push or pop – the final state is reached by a transition that sees EOF on the input and the stack

LR Parsing • An SLR/LALR/LR PDA has multiple states – it is a "recognizer," not a "predictor" – it builds a parse tree from the bottom up – the states keep track of which productions we might be in the middle

• The parsing of the Characteristic Finite State Machine (CFSM) is based on – Shift

– Reduce

LR Parsing

• To illustrate LR parsing, consider the grammar (Figure 2.25, Page 90): 1. program → stmt list $$$ 2. stmt_list → stmt_list stmt 3. stmt_list → stmt 4. stmt → id := expr 5. stmt → read id 6. stmt → write expr 7. expr → term 8. expr → expr add op term

LR Parsing

•

LR grammar (continued):

9. term 10. term 11. factor 12. factor 13. factor 14. add_op 15. add_op 16. mult_op 17. mult_op

→ factor → term mult_op factor → ( expr ) → id → number → + → → * → /

LR Parsing

• This grammar is SLR(1), a particularly nice class of bottom-up grammar – it isn't exactly what we saw originally – we've eliminated the epsilon production to simplify the presentation

• For details on the table driven SLR(1) parsing please note the following slides

LR Parsing

LR Parsing

LR Parsing

LR Parsing

LR Parsing

• Figure 2.30: SLR parsing is based on – Shift – Reduce and also – Shift & Reduce (for optimization)

Chapter 3:: Names, Scopes, and Bindings

Programming Language Pragmatics, Fourth Edition Michael L. Scott

Name, Scope, and Binding

• A name is exactly what you think it is – Most names are identifiers – symbols (like '+') can also be names

• A binding is an association between two things, such as a name and the thing it names • The scope of a binding is the part of the program (textually)in which the binding is active

Binding

• Binding Time is the point at which a binding is created or, more generally, the point at which any implementation decision is made – language design time • program structure, possible type

– language implementation time • I/O, arithmetic overflow, type equality (if unspecified in manual)

Binding • Implementation decisions (continued ): –program writing time •algorithms, names

–compile time •plan for data layout

–link time •layout of whole program in memory

–load time •choice of physical addresses

Binding • Implementation decisions (continued): –run time •value/variable bindings, sizes of strings •subsumes –program start-up time –module entry time –elaboration time (point a which a declaration is first "seen") –procedure entry time –block entry time –statement execution time

Binding

• The terms STATIC and DYNAMIC are generally used to refer to things bound before run time and at run time, respectively – “static” is a coarse term; so is "dynamic"

• It is difficult to overstate the importance of Binding Times in programming languages

Binding

• In general, early binding times are associated with greater efficiency • Later binding times are associated with greater flexibility • Compiled languages tend to have early binding times • Interpreted languages tend to have later binding times • Today we talk about the binding of identifiers to the variables they name

Binding

• Scope Rules - control bindings – Fundamental to all programming languages is the ability to name data, i.e., to refer to data using symbolic identifiers rather than addresses – Not all data is named! For example, dynamic storage in C or Pascal is referenced by pointers, not names

Lifetime and Storage Management

• Key events – creation of objects – creation of bindings – references to variables (which use bindings) – (temporary) deactivation of bindings – reactivation of bindings – destruction of bindings – destruction of objects

Lifetime and Storage Management • The period of time from creation to destruction is called the LIFETIME of a binding – If object outlives binding it's garbage – If binding outlives object it's a dangling reference • The textual region of the program in which the binding is active is its scope • In addition to talking about the scope of a binding, we sometimes use the word scope as a noun all by itself, without an indirect object

Lifetime and Storage Management • Storage Allocation mechanisms –Static –Stack –Heap

• Static allocation for –code –globals –static or own variables –explicit constants (including strings, sets, etc) –scalars may be stored in the instructions Copyright © 2009 Elsevier

Lifetime and Storage Management • Central stack for –parameters –local variables –temporaries

• Why a stack?

–allocate space for recursive routines (not necessary in FORTRAN – no recursion) –reuse space (in all programming languages)

Copyright © 2009 Elsevier

Lifetime and Storage Management

• Contents of a stack frame (cf., Figure 3.1) – arguments and returns – local variables – temporaries – bookkeeping (saved registers, line number static link, etc.)

• Local variables and arguments are assigned fixed OFFSETS from the stack pointer or frame pointer at compile time

Lifetime and Storage Management

Lifetime and Storage Management • Maintenance of stack is responsibility of calling sequence and subroutine prologue and epilogue –space is saved by putting as much in the prologue and epilogue as possible –time may be saved by putting more in the body of the subroutine •putting stuff in the caller instead or •combining what's known in both places (interprocedural optimization)

Lifetime and Storage Management

• Heap for dynamic allocation

Scope Rules

• A scope is a program section of maximal size in which no bindings change, or at least in which no re-declarations are permitted (see below) • In most languages with subroutines, we OPEN a new scope on subroutine entry: – create bindings for new local variables, – deactivate bindings for global variables that are redeclared (these variable are said to have a "hole" in their scope) – make references to variables

Scope Rules • On subroutine exit: – destroy bindings for local variables – reactivate bindings for global variables that were deactivated

• Algol 68: – ELABORATION = process of creating bindings when entering a scope

• Ada (re-popularized the term elaboration): – storage may be allocated, tasks started, even exceptions propagated as a result of the elaboration of declarations

Scope Rules • With STATIC (LEXICAL) SCOPE RULES, a scope is defined in terms of the physical (lexical) structure of the program – The determination of scopes can be made by the compiler – All bindings for identifiers can be resolved by examining the program – Typically, we choose the most recent, active binding made at compile time – Most compiled languages, C and Pascal included, employ static scope rules

Scope Rules

• The classical example of static scope rules is the most closely nested rule used in block structured languages such as Algol 60 and Pascal – An identifier is known in the scope in which it is declared and in each enclosed scope, unless it is re-declared in an enclosed scope – To resolve a reference to an identifier, we examine the local scope and statically enclosing scopes until a binding is found

Scope Rules

• We will see classes - a relative of modules later on, when discussing abstraction and objectoriented languages – These have even more sophisticated (static) scope rules

• Euclid is an example of a language with lexically-nested scopes in which all scopes are closed – rules were designed to avoid ALIASES, which complicate optimization and correctness arguments

Scope Rules

• Note that the bindings created in a subroutine are destroyed at subroutine exit – The modules of Modula, Ada, etc., give you closed scopes without the limited lifetime – Bindings to variables declared in a module are inactive outside the module, not destroyed – The same sort of effect can be achieved in many languages with own (Algol term) or static (C term) variables (see Figure 3.5)

Scope Rules

• Access to non-local variables STATIC LINKS – Each frame points to the frame of the (correct instance of) the routine inside which it was declared – In the absence of formal subroutines, correct means closest to the top of the stack – You access a variable in a scope k levels out by following k static links and then using the known offset within the frame thus found

Scope Rules

Scope Rules

• The key idea in static scope rules is that bindings are defined by the physical (lexical) structure of the program.

• With dynamic scope rules, bindings depend on the current state of program execution – They cannot always be resolved by examining the program because they are dependent on calling sequences – To resolve a reference, we use the most recent, active binding made at run time

Scope Rules

• Dynamic scope rules are usually encountered in interpreted languages – early LISP dialects assumed dynamic scope rules.

• Such languages do not normally have type checking at compile time because type determination isn't always possible when dynamic scope rules are in effect

Scope Rules Example: Static vs. Dynamic program scopes (input, output ); var a : integer; procedure first; begin a := 1; end; procedure second; var a : integer; begin first; end; begin a := 2; second; write(a); end.

Scope Rules Example: Static vs. Dynamic

• If static scope rules are in effect (as would be the case in Pascal), the program prints a 1 • If dynamic scope rules are in effect, the program prints a 2 • Why the difference? At issue is whether the assignment to the variable a in procedure first changes the variable a declared in the main program or the variable a declared in procedure second

Scope Rules Example: Static vs. Dynamic

• Static scope rules require that the reference resolve to the most recent, compile-time binding, namely the global variable a • Dynamic scope rules, on the other hand, require that we choose the most recent, active binding at run time – Perhaps the most common use of dynamic scope rules is to provide implicit parameters to subroutines – This is generally considered bad programming practice nowadays • Alternative mechanisms exist – static variables that can be modified by auxiliary routines – default and optional parameters

Scope Rules Example: Static vs. Dynamic

• At run time we create a binding for a when we enter the main program.

• Then we create another binding for a when we enter procedure second – This is the most recent, active binding when procedure first is executed – Thus, we modify the variable local to procedure second, not the global variable – However, we write the global variable because the variable a local to procedure second is no longer active

The Meaning of Names within a Scope

• Aliasing – What are aliases good for? (consider uses of FORTRAN equivalence) • space saving - modern data allocation methods are better • multiple representations - unions are better • linked data structures - legit

– Also, aliases arise in parameter passing as an unfortunate side effect • Euclid scope rules are designed to prevent this

The Meaning of Names within a Scope

• Overloading – some overloading happens in almost all languages • integer + vs. real + • read and write in Pascal • function return in Pascal

– some languages get into overloading in a big way • Ada • C++

The Meaning of Names within a Scope

• It's worth distinguishing between some closely related concepts – overloaded functions - two different things with the same name; in C++ • overload norm int norm (int a){return a>0 ? a : -a;) complex norm (complex c ) { // ...

– polymorphic functions -- one thing that works in more than one way • in Modula-2: function min (A : array of integer); …

• in Smalltalk

The Meaning of Names within a Scope

• It's worth distinguishing between some closely related concepts (2) – generic functions (modules, etc.) - a syntactic template that can be instantiated in more than one way at compile time • via macro processors in C++ • built-in in C++ • in Clu • in Ada

Binding of Referencing Environments

• Accessing variables with dynamic scope: – (1) keep a stack (association list) of all active variables • When you need to find a variable, hunt down from top of stack

• This is equivalent to searching the activation records on the dynamic chain

Binding of Referencing Environments

• Accessing variables with dynamic scope: – (2) keep a central table with one slot for every variable name • If names cannot be created at run time, the table layout (and the location of every slot) can be fixed at compile time • Otherwise, you'll need a hash function or something to do lookup • Every subroutine changes the table entries for its locals at entry and exit.

Binding of Referencing Environments

• (1) gives you slow access but fast calls • (2) gives you slow calls but fast access • In effect, variable lookup in a dynamicallyscoped language corresponds to symbol table lookup in a statically-scoped language • Because static scope rules tend to be more complicated, however, the data structure and lookup algorithm also have to be more complicated

Binding of Referencing Environments

• REFERENCING ENVIRONMENT of a statement at run time is the set of active bindings • A referencing environment corresponds to a collection of scopes that are examined (in order) to find a binding

Binding of Referencing Environments

• SCOPE RULES determine that collection and its order • BINDING RULES determine which instance of a scope should be used to resolve references when calling a procedure that was passed as a parameter – they govern the binding of referencing environments to formal procedures

Separate Compilation

• Separately-compiled files in C provide a sort of poor man's modules: – Rules for how variables work with separate compilation are messy – Language has been jury-rigged to match the behavior of the linker – static on a function or variable outside a function means it is usable only in the current source file • This static is a different notion from the static variables inside a function

Separate Compilation

• Separately-compiled files in C (continued) – extern on a variable or function means that it is declared in another source file – Functions headers without bodies are extern by default – extern declarations are interpreted as forward declarations if a later declaration overrides them – Variables or functions (with bodies) that don't say static or extern are either global or common (a Fortran term)

Conclusions

• The morals of the story: – language features can be surprisingly subtle – designing languages to make life easier for the compiler writer can be a GOOD THING – most of the languages that are easy to understand are easy to compile, and vice versa

Conclusions

• A language that is easy to compile often leads to – a language that is easy to understand – more good compilers on more machines (compare Pascal and Ada!) – better (faster) code – fewer compiler bugs – smaller, cheaper, faster compilers – better diagnostics

Chapter 4 :: Semantic Analysis

Programming Language Pragmatics, Fourth Edition Michael L. Scott

Copyright © 2016 Elsevier

Role of Semantic Analysis

• Following parsing, the next two phases of the "typical" compiler are – semantic analysis – (intermediate) code generation

• The principal job of the semantic analyzer is to enforce static semantic rules – constructs a syntax tree (usually first) – information gathered is needed by the code generator

Role of Semantic Analysis

• There is considerable variety in the extent to which parsing, semantic analysis, and intermediate code generation are interleaved • A common approach interleaves construction of a syntax tree with parsing (no explicit parse tree), and then follows with separate, sequential phases for semantic analysis and code generation

Role of Semantic Analysis

• The PL/0 compiler has no optimization to speak of (there's a tiny little trivial phase, which operates on the syntax tree) • Its code generator produces MIPs assembler, rather than a machineindependent intermediate form

Attribute Grammars

• Both semantic analysis and (intermediate) code generation can be described in terms of annotation, or "decoration" of a parse or syntax tree • ATTRIBUTE GRAMMARS provide a formal framework for decorating such a tree • The notes below discuss attribute grammars and their ad-hoc cousins, ACTION ROUTINES

Attribute Grammars

• We'll start with decoration of parse trees, then consider syntax trees • Consider the following LR (bottom-up) grammar for arithmetic expressions made of constants, with precedence and associativity:

Attribute Grammars E → E + T E → E – T E → T T → T * F T → T / F T → F F → - F • This says nothing about what the program MEANS

Attribute Grammars

• We can turn this into an attribute grammar as follows (similar to Figure 4.1): E → E + T E → E – T E → T T → T * F T → T / F T → F F → - F F → (E) F → const

E1.val = E2.val + T.val E1.val = E2.val - T.val E.val = T.val T1.val = T2.val * F.val T1.val = T2.val / F.val T.val = F.val F1.val = - F2.val F.val = E.val F.val = C.val

Attribute Grammars

• The attribute grammar serves to define the semantics of the input program

• Attribute rules are best thought of as definitions, not assignments • They are not necessarily meant to be evaluated at any particular time, or in any particular order, though they do define their left-hand side in terms of the right-hand side

Evaluating Attributes

• The process of evaluating attributes is called annotation, or DECORATION, of the parse tree [see Figure 4.2 for (1+3)*2] – When a parse tree under this grammar is fully decorated, the value of the expression will be in the val attribute of the root

• The code fragments for the rules are called SEMANTIC FUNCTIONS – Strictly speaking, they should be cast as functions, e.g., E1.val = sum (E2.val, T.val), cf., Figure 4.1

Evaluating Attributes

Evaluating Attributes

• This is a very simple attribute grammar: – Each symbol has at most one attribute • the punctuation marks have no attributes

• These attributes are all so-called SYNTHESIZED attributes: – They are calculated only from the attributes of things below them in the parse tree

Evaluating Attributes

• In general, we are allowed both synthesized and INHERITED attributes: – Inherited attributes may depend on things above or to the side of them in the parse tree – Tokens have only synthesized attributes, initialized by the scanner (name of an identifier, value of a constant, etc.).

– Inherited attributes of the start symbol constitute run-time parameters of the compiler

Evaluating Attributes

• The grammar above is called SATTRIBUTED because it uses only synthesized attributes

• Its ATTRIBUTE FLOW (attribute dependence graph) is purely bottom-up – It is SLR(1), but not LL(1)

• An equivalent LL(1) grammar requires inherited attributes:

Evaluating Attributes – Example

• Attribute grammar in Figure 4.3:

Evaluating Attributes– Example

• Attribute grammar in Figure 4.3 (continued):

Evaluating Attributes– Example

Evaluating Attributes– Example

• Attribute grammar in Figure 4.3: – This attribute grammar is a good bit messier than the first one, but it is still L-ATTRIBUTED, which means that the attributes can be evaluated in a single left-to-right pass over the input – In fact, they can be evaluated during an LL parse – Each synthetic attribute of a LHS symbol (by definition of synthetic) depends only on attributes of its RHS symbols

Evaluating Attributes – Example

• Attribute grammar in Figure 4.3: – Each inherited attribute of a RHS symbol (by definition of L-attributed) depends only on • inherited attributes of the LHS symbol, or • synthetic or inherited attributes of symbols to its left in the RHS

– L-attributed grammars are the most general class of attribute grammars that can be evaluated during an LL parse

Evaluating Attributes

• There are certain tasks, such as generation of code for short-circuit Boolean expression evaluation, that are easiest to express with non-L-attributed attribute grammars • Because of the potential cost of complex traversal schemes, however, most real-world compilers insist that the grammar be Lattributed

Evaluating Attributes – Syntax Trees

Evaluating Attributes – Syntax Trees

Evaluating Attributes – Syntax Trees

Evaluating Attributes – Syntax Trees

Figure 4.7 Construction of a syntax tree for (1 + 3) * 2 via decoration of a bottom-up parse tree, using the grammar of Figure 4.5. This figure reads from bottom to top. In diagram (a), the values of the constants 1 and 3 have been placed in new syntax tree leaves. Pointer

s to these leaves propagate up into the attributes of E and T. In (b), the pointer s to these leaves become child pointer s of a new internal + node. In (c) the pointer to this node propagates up into the attributes of T, and a new leaf is created for 2 . Finally, in (d), the pointer s from T and F become child pointer s of a new internal × node, and a pointer to this node propagates up into the attributes of E .

Evaluating Attributes – Syntax Trees

Figure 4.8 Construction of a syntax tree via decoration of a top-down parse tree, using the grammar of Figure 4.6. In the top diagram, (a), the value of the constant 1 has been placed in a new syntax tree leaf. A pointer to this leaf then propagates to the st attribute of TT. In (b), a second leaf has been created to hold the constant 3 . Pointer s to the two leaves then become child pointer s of a new internal + node, a pointer to which propagates from the st attribute of the bottom-most TT, where it was created, all the way up and over to the st attribute of the top-most FT. In (c), a third leaf has been created for the constant 2 . Pointer s to this leaf and to the + node then become the children of a new ×node, a pointer to which propagates from the st of the lower FT, where it was created, all the way to the root of the tree

Action Routines

• We can tie this discussion back into the earlier issue of separated phases v. on-thefly semantic analysis and/or code generation

• If semantic analysis and/or code generation are interleaved with parsing, then the TRANSLATION SCHEME we use to evaluate attributes MUST be L-attributed

Action Routines

• If we break semantic analysis and code generation out into separate phase(s), then the code that builds the parse/syntax tree must still use a left-to-right (L-attributed) translation scheme • However, the later phases are free to use a fancier translation scheme if they want

Action Routines

• There are automatic tools that generate translation schemes for context-free grammars or tree grammars (which describe the possible structure of a syntax tree) – These tools are heavily used in syntax-based editors and incremental compilers – Most ordinary compilers, however, use ad-hoc techniques

Action Routines

• An ad-hoc translation scheme that is interleaved with parsing takes the form of a set of ACTION ROUTINES: – An action routine is a semantic function that we tell the compiler to execute at a particular point in the parse

• If semantic analysis and code generation are interleaved with parsing, then action routines can be used to perform semantic checks and generate code

Action Routines

• If semantic analysis and code generation are broken out as separate phases, then action routines can be used to build a syntax tree – A parse tree could be built completely automatically – We wouldn't need action routines for that purpose

Action Routines

• Later compilation phases can then consist of ad-hoc tree traversal(s), or can use an automatic tool to generate a translation scheme – The PL/0 compiler uses ad-hoc traversals that are almost (but not quite) left-to-right

• For our LL(1) attribute grammar, we could put in explicit action routines as follows:

Action Routines - Example

• Action routines (Figure 4.9)

Space Management for Attributes • Entries in the attributes stack are pushed and popped automatically

Decorating a Syntax Tree • Syntax tree for a simple program to print an average of an integer and a real

Decorating a Syntax Tree

• Tree grammar representing structure of syntax tree in Figure 4.12

Decorating a Syntax Tree

• Sample of complete tree grammar representing structure of syntax tree in Figure 4.12

Chapter 5:: Target Machine Architecture

Programming Language Pragmatics, Fourth Edition Michael L. Scott

Copyright © 2016 Elsevier

Assembly-Level View

• As mentioned early in this course, a compiler is simply a translator – It translates programs written in one language into programs written in another language • This other language can be almost anything • Most of the time, however, it's the machine language for some available computer

Assembly-Level View

• As a review, we will go over some of the material most relevant to language implementation, so that we can better understand – what the compiler has to do to your program – why certain things are fast and others slow – why certain things are easy to compile and others aren't

Assembly-Level View

• There are many different programming languages and there are many different machine languages – Machine languages show considerably less diversity than programming languages – Traditionally, each machine language corresponds to a different computer ARCHITECTURE – The IMPLEMENTATION is how the architecture is realized in hardware

Assembly-Level View

• Formally, an architecture is the interface to the hardware – what it looks like to a user writing programs on the bare machine.

• In the last 20 years, the line between these has blurred to the point of disappearing – compilers have to know a LOT about the implementation to do a decent job

Assembly-Level View

• Changes in hardware technology (e.g., how many transistors can you fit on one chip?) have made new implementation techniques possible – the architecture was also modified – Example: RISC (reduced instruction set computer) revolution ~20 years ago

• In the discussion below, we will focus on modern RISC architectures, with a limited amount of coverage of their predecessors, the CISC architectures

Workstation Macro-Architecture

• Most modern computers consist of a collection of DEVICES that talk to each other over a BUS • From the point of view of language implementation: – the most important device is the PROCESSOR(S) – the second most important is main memory – other devices include: disks, keyboards, screens, networks, general-purpose serial/parallel ports, etc.

Workstation Macro-Architecture • Almost all modern computers use the (von Neumann) stored program concept: – a program is simply a collection of bits in memory that the computer interprets as instructions, rather than as integers, floating point numbers, or some other sort of data

• What a processor does is repeatedly – fetch an instruction from memory – decode it - figure out what it says to do

– fetch any needed operands from registers or memory – execute the operation, and – store any result(s) back into registers or memory

Workstation Macro-Architecture

• This set of operations is referred to as the fetchexecute cycle – The computer runs this cycle at a furious pace, never stopping, regardless of the meaning of the instructions • You can point a processor's instruction fetch logic at a long string of floating point numbers and it will blithely begin to execute them; it will do something, though that something probably won't make much sense • Operating systems are designed so that when the computer has nothing useful to do it is pointed at an infinite loop that it can execute furiously, but harmlessly

Workstation Macro-Architecture • The crucial components of a typical processor include a collection of FUNCTIONAL UNITS: – hardware to decode instructions and drive the other functional units – hardware to fetch instructions and data from memory and to store them back again if they are modified – one or more arithmetic/logic units (ALUs) to do actual computations

– registers to hold the most heavily-used parts of the state of the computation – hardware to move data among the various functional units and registers

Memory Hierarchy

• Memory is too big to fit on one chip with a processor – Because memory is off-chip (in fact, on the other side of the bus), getting at it is much slower than getting at things on-chip – Most computers therefore employ a MEMORY HIERARCHY, in which things that are used more often are kept close at hand

Memory Hierarchy

Memory Hierarchy

• Some of these levels are visible to the programmer; others are not • For our purposes here, the levels that matter are registers and main memory • Registers are special locations that can hold (a very small amount of) data that can be accessed very quickly – A typical RISC machine has a few (often two) sets of registers that are used to hold integer and floating point operands

Memory Hierarchy

• It also has several special-purpose registers, including the – program counter (PC) • holds the address of the next instruction to be executed • usually incremented during fetch-execute cycle

– processor status register • holds a variety of bits of little interest in this course (privilege level, interrupt priority level, trap enable bits)

Data Representation

• Memory is usually (but not always) byteaddressable, meaning that each 8-bit piece has a unique address – Data longer than 8 bits occupy multiple bytes – Typically • an integer occupies 16, 32, or (recently) 64 bits • a floating point number occupies 32, 64, or (recently) 128 bits

Data Representation

• It is important to note that, unlike data in highlevel programming languages, memory is untyped (bits are just bits) • Operations are typed, in the sense that different operations interpret the bits in memory in different ways • Typical DATA FORMATS include – instruction – integer (various lengths) – floating point (various lengths)

Data Representation

• Other concepts we will not detail but covered in other courses – Big-endian vs. little-endian (for details see Figure 5.2) – Integer arithmetic • 2's complement arithmetic

– Floating-point arithmetic • IEEE standard, 1985

Data Representation

Instruction-Set Architecture

• The set of instructions executed by a modern processor may include: – data movement (load, store, push, pop, movem, swap - registers) – arithmetic and logical (negate, extend, add, subtract, multiply, divide, and, or, shift) – control transfer (jump, call, trap - jump into the operating system, return - from call or trap, conditional branch)

Instruction-Set Architecture Addressing Modes

• Instructions can specify many different ways to obtain their data (Addressing Modes) – data in instruction – data in register – address of data in instruction – address of data in register – address of data computed from two or more values contained in the instruction and/or registers

Instruction-Set Architecture Addressing Modes

• On a RISC machine, arithmetic/logic instructions use only the first two of these ADDRESSING MODES – load and store instructions use the others

• On a CISC machine, all addressing modes are generally available to all instructions – CISC machines typically have a richer set of addressing modes, including some that perform • multiple indirections, and/or • arithmetic computations on values in memory in order to calculate an effective address

Architecture Implementation

• As technology advances, there are occasionally times when some threshold is crossed that suddenly makes it possible to design machines in a very different way – One example of such a paradigm shift occurred in the mid 1980s with the development of RISC (reduced instruction set computer) architectures

Architecture Implementation

• During the 1950s and the early 1960s, the instruction set of a typical computer was implemented by soldering together large numbers of discrete components that performed the required operations • To build a faster computer, one generally designed extra, more powerful instructions, which required extra hardware • This has the unfortunate effect of requiring assembly language programmers to learn a new language

Architecture Implementation

• IBM hit upon an implementation technique called MICROPROGRAMMING: – same instruction set across a whole line of computers, from cheap to fast machines – basic idea of microprogramming • build a microengine in hardware that executed a interpreter program in firmware – interpreter implemented IBM 360 instruction set

• more expensive machines had fancier microengines – more of the 360 functionality in hardware – top-of-the-line machines had everything in hardware

Architecture Implementation

• Microprogramming makes it easy to extend the instruction set • people ran studies to identify instructions that often occurred in sequence (e.g., the sequence that jumps to a subroutine and updates bookkeeping information in the stack) • then provided new instructions that performed the function of the sequence • By clever programming in the firmware, it was generally possible to make the new instruction faster than the old sequence, and programs got faster.

Architecture Implementation

• The microcomputer revolution of the late 1970s (another paradigm shift) occurred when it became possible to fit a microengine onto a single chip (microprocessor): – personal computers were born – by the mid 1980s, VLSI technology reached the point where it was possible to eliminate the microengine and still implement a processor on a single chip

Architecture Implementation

• With a hardware-only processor on one chip, it then became possible to apply certain performance-enhancing tricks to the implementation, but only if the instruction set was very simple and predictable – This was the RISC revolution – Its philosophy was to give up "nice", fancy features in order to make common operations fast

Architecture Implementation

• RISC machines: – a common misconception is that small instruction sets are the distinguishing characteristic of a RISC machine – better characterization: RISC machines are machines in which at least one new instruction can (in the absence of conflicts) be started every cycle (hardware clock tick) • all possible mechanisms have been exploited to minimize the duration of a cycle, and to maximize the number of functional units that operate in parallel during a given cycle

Architecture Implementation

• Reduced cycle time comes from making all instructions simple and regular – Simple instructions never have to run slowly because of extra logic necessary to implement the complicated instructions – Maximal parallelism comes from giving the instructions a very regular, predictable format • the interactions between instructions are clear and the processor can begin working on the next instruction before the previous one has finished

Compiling for Modern Processors

• PIPELINING is probably the most important performance enhancing trick – It works kind of like this: TIME →

fetch

decode fetch

execute store

instr

instr

data

data

fetch

decode fetch

execute store

instr

instr

data

data

fetch decode fetch execute

store

instr

data

instr

data

Compiling for Modern Processors

• The processor has to be careful not to execute an instruction that depends on a previous instruction that hasn't finished yet – The compiler can improve the performance of the processor by generating code in which the number of dependencies that would stall the pipeline is minimized – This is called INSTRUCTION SCHEDULING; it's one of the most important machine-specific optimizations for modern compilers

Compiling for Modern Processors

• Loads and load delays are influenced by – Dependences • Flow dependence • Anti-dependence • Output dependence

• Branches – since control can go both ways, branches create delays

Compiling for Modern Processors • Usual goal: minimize pipeline stalls • Delay slots – loads and branches take longer than ordinary instructions – loads have to go to memory, which is slow – branches disrupt the pipeline – processor have interlock hardware – early RISC machines often provided delay slots for the second (maybe third) cycle of a load or store instruction, during which something else can occur – the instruction in a branch delay slot gets executed whether the branch occurs or not

Compiling for Modern Processors • Delay slots (continued) – the instruction in a load delay slot can't use the loaded value – as pipelines have grown deeper, people have generally realized that delay slots are more trouble than they're worth – most current processor implementations interlock all loads, so you don't have to worry about the correctness issues of load delay – some machines still have branch delay slots (so they can run code written in the late '80s) • later implementations usually provide a nullifying alternative that skips the instruction in the slot if static branch prediction is wrong

Compiling for Modern Processors

• Unfortunately, even this start a new instruction every cycle characterization of RISC machines is inadequate – In all honesty, there is no good clear definition of what RISC means

• Most recent RISC machines (and also the most recent x86 machines) are so-called SUPERSCALAR implementations that can start more than one instruction each cycle

Compiling for Modern Processors

• If it's a CISC machine, the number of instructions per second depends crucially on the mix of instructions produced by the compiler – the MHz number gives an upper bound (again assuming a single set of functional units) – if it's a "multi-issue“ (superscalar) processor like the PowerPC G3 or Intel machines since the Pentium Pro, the upper bound is higher than the MHz number

Compiling for Modern Processors

• As technology improves, complexity is beginning to creep back into RISC designs • Right now we see "RISC" machines with on-chip – vector units – memory management units – large caches

Compiling for Modern Processors

• We also see "CISC" machines (the Pentium family) with RISC-like subsets (single-cycle hard-coded instructions) • In the future, we might see – large amounts of main memory – multiple processors – network interfaces (now in prototypes) – additional functions • digital signal processing

Compiling for Modern Processors

• In addition, the 80x86 instruction set will be with us for a long time, due to the huge installed base of IBM-compatible PCs – After a failed attempt to introduce its own RISC architecture (the i860), Intel worked with HP on the RISC-like IA64, architecture – Currently x86-64 (AMD’s design) is the 64-bit extension of the x86 architecture that dominates the market

Compiling for Modern Processors

• In a sense, code for RISC machines resembles microcode – Complexity that used to be hidden in firmware must now be embedded in the compiler – Some of the worst of the complexity (e.g. branch delay slots) can be hidden by the assembler (as it is on MIPS machines) • it is definitely true that it is harder to produce good (fast) code for RISC machines than it is for CISC machines

Compiling for Modern Processors

• Example: the Pentium chip runs a little bit faster than a 486 if you use the same old binaries – If you recompile with a compiler that knows to use a RISC-like subset of the instruction set, with appropriate instruction scheduling, the Pentium can run much faster than a 486

Compiling for Modern Processors

• Multiple functional units – superscalar machines can issue (start) more than one instruction per cycle, if those instructions don't need the same functional units – for example, there might be two instruction fetch units, two instruction decode units, an integer unit, a floating point adder, and a floating point multiplier

Compiling for Modern Processors

• Because memory is so much slower than registers, (several hundred times slower at present) keeping the right things in registers is extremely important – RISC machines often have at least two different classes of registers (so they don't have to support all operations on all registers) which the compiler has to keep track of

Compiling for Modern Processors

• Some (SPARC) have a complicated collection of overlapping REGISTER WINDOWS

• Finally, good register allocation sometimes conflicts with good instruction scheduling – code that makes ideal use of functional units may require more registers than code that makes poorer use of functional units – good compilers spend a *great* deal of effort • make sure that the data they need most is in register

Compiling for Modern Processors

• Note that instruction scheduling and register allocation often conflict • Limited instruction formats/more primitive instructions – Many operations that are provided by a single instruction on a CISC machine take multiple instructions on a RISC machine – For example, some RISC machines don't provide a 32-bit multiply; you have to build it out of 4-bit (or whatever) multiplies

Compiling For Modern Machines

• To make all instructions the same length – data values and parts of addresses are often scaled and packed into odd pieces of the instruction – loading from a 32-bit address contained in the instruction stream takes two instructions, because one instruction isn't big enough to hold the whole address and the code for load • first instruction loads part of the address into a register

• second instruction adds the rest of the address into the register and performs the load

Summary

• Some major RISC architectures: – ARM (Android, iPhone, Windows Phone, etc.)

– MIPS (PlayStation, Nintendo, etc.)

– SPARC (Oracle, Fujitsu) – Power/Power PC (IBM, Motorola. Apple) – RISC-V (open source 32-bit)

• Currently there is growing demand for 64bit addressing (Intel, AMD)

Chapter 6:: Control Flow

Programming Language Pragmatics, Fourth Edition Michael L. Scott

Copyright © 2016 Elsevier

Control Flow • Basic paradigms for control flow: –Sequencing –Selection –Iteration –Procedural Abstraction –Recursion –Concurrency –Exception Handling and Speculation –Nondeterminacy

Expression Evaluation

• Infix, prefix operators • Precedence, associativity (see Figure 6.1) – C has 15 levels - too many to remember – Pascal has 3 levels - too few for good semantics – Fortran has 8 – Ada has 6 • Ada puts and & or at same level

– Lesson: when unsure, use parentheses!

Expression Evaluation

Figure 6.1 Operator precedence levels in Fortran, Pascal, C, and Ada. The operator s at the top of the figure group most tightly.

Expression Evaluation

• Ordering of operand evaluation (generally none) • Application of arithmetic identities – distinguish between commutativity, and (assumed to be safe) – associativity (known to be dangerous) (a + b) + c works if a~=maxint and b~=minint and c<0 a + (b + c) does not

– inviolability of parentheses

Expression Evaluation

• Short-circuiting – Consider (a < b) && (b < c): • If a >= b there is no point evaluating whether b < c because (a < b) && (b < c) is automatically false – Other similar situations

if (b != 0 && a/b == c) ... if (*p && p->foo) ... if (f || messy()) …

– Can be avoided to allow for side effects in the condition functions

Expression Evaluation • Variables as values vs. variables as references –value-oriented languages •C, Pascal, Ada

–reference-oriented languages •most functional languages (Lisp, Scheme, ML) •Clu, Smalltalk

–Algol-68 kinda halfway in-between –Java deliberately in-between •built-in types are values •user-defined types are objects - references

Expression Evaluation

• Expression-oriented vs. statement-oriented languages – expression-oriented: • functional languages (Lisp, Scheme, ML)

• Algol-68

– statement-oriented: • most imperative languages

– C kinda halfway in-between (distinguishes) • allows expression to appear instead of statement

Expression Evaluation • Orthogonality – Features that can be used in any combination • Meaning is consistent if (if b != 0 then a/b == c else false) then ... if (if f then true else messy()) then ...

• Initialization – Pascal has no initialization facility (assign)

• Aggregates – Compile-time constant values of user-defined composite types

Expression Evaluation

• Assignment – statement (or expression) executed for its side effect – assignment operators (+=, -=, etc) • handy • avoid redundant work (or need for optimization) • perform side effects exactly once

– C --, ++ • postfix form

Expression Evaluation

• Side Effects – often discussed in the context of functions – a side effect is some permanent state change caused by execution of function • some noticeable effect of call other than return value • in a more general sense, assignment statements provide the ultimate example of side effects – they change the value of a variable

Expression Evaluation

• SIDE EFFECTS ARE FUNDAMENTAL TO THE WHOLE VON NEUMANN MODEL OF COMPUTING

• In (pure) functional, logic, and dataflow languages, there are no such changes – These languages are called SINGLEASSIGNMENT languages

Expression Evaluation

• Several languages outlaw side effects for functions – easier to prove things about programs – closer to mathematical intuition – easier to optimize – (often) easier to understand

• But side effects can be nice – consider rand()

Expression Evaluation • Side effects are a particular problem if they affect state used in other parts of the expression in which a function call appears – It's nice not to specify an order, because it makes it easier to optimize

– Fortran says it's OK to have side effects • they aren't allowed to change other parts of the expression containing the function call

• Unfortunately, compilers can't check this completely, and most don't at all

Sequencing

• Sequencing – specifies a linear ordering on statements • one statement follows another

– very imperative, Von-Neumann

Selection

• Selection – sequential if statements if ... then ... else if ... then ... elsif ... else (cond

(C1) (E1) (C2) (E2)

...

(Cn) (En) (T) (Et)

)

Selection

• Selection – Fortran computed gotos – jump code • for selection and logically-controlled loops

• no point in computing a Boolean value into a register, then testing it • instead of passing register containing Boolean out of expression as a synthesized attribute, pass inherited attributes INTO expression indicating where to jump to if true, and where to jump to if false

Selection

• Jump is especially useful in the presence of short-circuiting • Example (section 6.4.1 of book): if ((A > B) and (C > D)) or (E <> F) then then_clause else

else_clause

Selection

• Code generated w/o short-circuiting (Pascal) r1 := A

r2 := B r1 := r1 > r2 r2 := C r3 := D r2 := r2 > r3 r1 := r1 & r2 r2 := E r3 := F r2 := r2 $<>$ r3 r1 := r1 $|$ r2 if r1 = 0 goto L2 L1:

then_clause -- label not actually used goto L3

L2: L3:

else_clause

-- load

Selection

• Code generated w/ short-circuiting (C) r1 := A r2 := B if r1 <= r2 goto L4 r1 := C r2 := D if r1 > r2 goto L1 L4:

r1 := E

L1:

r2 := F if r1 = r2 goto L2 then_clause goto L3 else_clause

L2: L3:

Iteration

• Enumeration-controlled – Pascal or Fortran-style for loops • scope of control variable

• changes to bounds within loop • changes to loop variable within loop • value after the loop – Can iterate over elements of any well-defined set

Recursion

• Recursion – equally powerful to iteration – mechanical transformations back and forth – often more intuitive (sometimes less) – naïve implementation less efficient • no special syntax required

• fundamental to functional languages like Scheme

Recursion

• Tail recursion – No computation follows recursive call int gcd (int a, int b) { /* assume a, b > 0 */ if (a == b) return a;

else if (a > b) return gcd (a - b,b); else return gcd (a, b – a); }

Chapter 7:: Data Types

Programming Language Pragmatics, Fourth Edition Michael L. Scott

Copyright © 2016 Elsevier

Data Types

• We all have developed an intuitive notion of what types are; what's behind the intuition?

– collection of values from a "domain" (the denotational approach) – internal structure of a bunch of data, described down to the level of a small set of fundamental types (the structural approach) – collection of well-defined operations that can be applied to objects of that type (the abstraction approach)

Data Types

• What are types good for?

– implicit context – checking - make sure that certain meaningless operations do not occur • type checking cannot prevent all meaningless operations • It catches enough of them to be useful

• Polymorphism results when the compiler finds that it doesn't need to know certain things

Data Types

• STRONG TYPING has become a popular buzz-word – like structured programming – informally, it means that the language prevents you from applying an operation to data on which it is not appropriate

• STATIC TYPING means strongly typed and that the compiler can do all the checking at compile time

Type Systems

• Examples – Common Lisp is strongly typed, but not statically typed – Ada is statically typed – Pascal is almost statically typed – Java is strongly typed, with a non-trivial mix of things that can be checked statically and things that have to be checked dynamically – C has become more strongly typed with each new version, though loopholes still remain

Type Systems

• Common terms: – discrete types – countable • integer • boolean • char • enumeration • subrange – Scalar types - one-dimensional • discrete • real

Type Systems

• Composite types: – records (unions) – arrays • strings

– sets – pointers – lists – files

Type Systems

• ORTHOGONALITY is a useful goal in the design of a language, particularly its type system – A collection of features is orthogonal if there are no restrictions on the ways in which the features can be combined (analogy to vectors)

Type Systems

• For example – Pascal is more orthogonal than Fortran, (because it allows arrays of anything, for instance), but it does not permit variant records as arbitrary fields of other records (for instance)

• Orthogonality is nice primarily because it makes a language easy to understand, easy to use, and easy to reason about

Type Checking

• A TYPE SYSTEM has rules for – type equivalence (when are the types of two values the same?)

– type compatibility (when can a value of type A be used in a context that expects type B?) – type inference (what is the type of an expression, given the types of the operands?)

Type Checking

• Type compatibility / type equivalence – Compatibility is the more useful concept, because it tells you what you can DO – The terms are often (incorrectly, but we do it too) used interchangeably.

Type Checking

• Certainly format does not matter: struct { int a, b; }

is the same as struct { int a, b; }

and we certainly want them to be the same as struct { int a; int b; }

Type Checking

• Two major approaches: structural equivalence and name equivalence – Name equivalence is based on declarations – Structural equivalence is based on some notion of meaning behind those declarations but can equate types the programmer doesn’t intend, considered lower-level – Name equivalence is more fashionable these days

Type Checking

• There are at least two common variants on name equivalence – The differences between all these approaches boils down to where you draw the line between important and unimportant differences between type descriptions – In all three schemes described in the book, we begin by putting every type description in a standard form that takes care of "obviously unimportant" distinctions like those above

Type Checking

• Structural equivalence depends on simple comparison of type descriptions substitute out all names – expand all the way to built-in types

• Original types are equivalent if the expanded type descriptions are the same

Type Checking

• Coercion – When an expression of one type is used in a context where a different type is expected, one normally gets a type error – But what about var a : integer; b, c : real; ... c := a + b;

Type Checking

• Coercion – Many languages allow things like this, and COERCE an expression to be of the proper type – Coercion can be based just on types of operands, or can take into account expected type from surrounding context as well – Fortran has lots of coercion, all based on operand type

Type Checking • C has lots of coercion, too, but with simpler rules: – all floats in expressions become doubles – short, int, and char become int in expressions – if necessary, precision is removed when assigning into LHS

Type Checking

• In effect, coercion rules are a relaxation of type checking – Recent thought is that this is probably a bad idea – Ada allows little conversion – Fortran and Pascal a little more – C++, Perl go hog-wild with coercions – They're one of the hardest parts of the language to understand

Type Checking

• Make sure you understand the difference between – type conversions (explicit) – type coercions (implicit) – sometimes the word 'cast' is used for conversions (C is guilty here)

Generic Subroutines and Modules

• Generic modules or classes are particularly valuable for creating containers: data abstractions that hold a collection of objects • Generic subroutines (methods) are needed in generic modules (classes), and may also be useful in their own right

Chapter 8 :: Composite Types

Programming Language Pragmatics, Fourth Edition Michael L. Scott

Copyright © 2016 Elsevier

Records (Structures) and Variants (Unions)

• Records – usually laid out contiguously – possible holes for alignment reasons – smart compilers may rearrange fields to minimize holes (C compilers promise not to) – implementation problems are caused by records containing dynamic arrays • we won't be going into that in any detail

Records (Structures) and Variants (Unions)

• Unions (variant records) – overlay space – cause problems for type checking

• Lack of tag means you don't know what is there • Ability to change tag and then access fields hardly better – can make fields "uninitialized" when tag is changed (requires extensive run-time support) – can require assignment of entire variant, as in Ada

Records (Structures) and Variants (Unions)

• Memory layout and its impact (structures)

Figure 8.1 Likely layout in memory for objects of type element on a 32-bit machine. Alignment restrictions lead to the shaded “holes.”

Records (Structures) and Variants (Unions)

• Memory layout and its impact (structures)

Figure 8.3 Likely memory layout for packed element records. The atomic_number and atomic_weight fields are nonaligned, and can only be read or written (on most machines) via multi-instruction sequences.

Records (Structures) and Variants (Unions)

• Memory layout and its impact (structures)

Figure 8.4 Rearranging record fields to minimize holes. By sor ting fields according to the size of their alignment constraint, a compiler can minimize the space devoted to holes, while keeping the fields aligned.

Records (Structures) and Variants (Unions)

• Memory layout and its impact (unions)

Figure 8.16 (CD) Likely memory layouts for element variants. The value of the naturally_ occurring field (shown here with a double border) is intended to indicate which of the interpretations of the remaining space is valid. Field source is assumed to point to a string that has been independently allocated.

Arrays

• Arrays are the most common and important composite data types • Unlike records, which group related fields of disparate types, arrays are usually homogeneous • Semantically, they can be thought of as a mapping from an index type to a component or element type • A slice or section is a rectangular portion of an array (See figure 8.5)

Arrays

Figure 8.5 Array slices(sections) in Fortran90. Much like the values in the header of an enumeration-controlled loop (Section6.5.1), a: b: c in a subscript indicates positions a, , a+2c, ...through b. If a or b is omitted, the corresponding bound of the array is assumed. If c is a+c

omitted, 1 is assumed. It is even possible to use negative values of c in order to select positions in reverse order. The slashes in the second subscript of the lower right example delimit an explicit list of positions.

Arrays • Dimensions, Bounds, and Allocation – global lifetime, static shape — If the shape of an array is known at compile time, and if the array can exist throughout the execution of the program, then the compiler can allocate space for the array in static global memory – local lifetime, static shape — If the shape of the array is known at compile time, but the array should not exist throughout the execution of the program, then space can be allocated in the subroutine’s stack frame at run time.

– local lifetime, shape bound at elaboration time

Arrays

Figure 8.7 Elaboration-time allocation of arrays in Ada or C99.

Arrays

• Contiguous elements (see Figure 8.8) – column major - only in Fortran – row major • used by everybody else • makes array [a..b, c..d] the same as array [a..b] of array [c..d]

Arrays

Figure 8.8 Row- and column-major memory layout for two-dimensional arrays. In row-major order, the elements of a row are contiguous in memory; in column-major order, the elements of a column are contiguous. The second cache line of each array is shaded, on the assumption that each element is an eight-byte floating-point number, that cache lines are 32 bytes long (a common size), and that the array begins at a cache line boundary. If the array is indexed from A[0,0] to A[9,9], then in the row-major case elements A[0,4] through A[0,7] share a cache line; in the column-major case elements A[4,0] through A[7,0] share a cache line.

Arrays

• Two layout strategies for arrays (Figure 8.9): – Contiguous elements – Row pointers

• Row pointers – an option in C – allows rows to be put anywhere - nice for big arrays on machines with segmentation problems – avoids multiplication – nice for matrices whose rows are of different lengths • e.g. an array of strings

– requires extra space for the pointers

Arrays

Figure 8.9 Contiguous array allocation v. row pointers in C. The declaration on the left is a tr ue two-dimensional array. The slashed boxes are NUL bytes; the shaded areas are holes. The declaration on the right is a ragged array of pointers to arrays of character s. In both cases, we have omitted bounds in the declaration that can be deduced from the size of the initializer (aggregate). Both data structures permit individual characters to be accessed using double subscripts, but the memory layout (and corresponding address arithmetic) is quite different.

Arrays

• Example 8.25 Indexing a contiguous array: Suppose A : array [L1..U1] of array [L2..U2] of array [L3..U3] of elem; D1 = U1-L1+1

D2 = U2-L2+1 D3 = U3-L3+1

Let

S3 = size of elem S2 = D3 * S3 S1 = D2 * S2

Arrays

Figure 8.10 Virtual location of an array with nonzero lower bounds. By computing the constant portions of an array index at compile time, we effectively index into an array whose starting address is offset in memory, but whose lower bounds are all zero.

Arrays

• Example 8.25 (continued) We could compute all that at run time, but we can make do with fewer subtractions: == (i * S1) + (j * S2) + (k * S3) + address of A - [(L1 * S1) + (L2 * S2) + (L3 * S3)] The stuff in square brackets is compile-time constant that depends only on the type of A

Strings

• In some languages, strings are really just arrays of characters • In others, they are often special-cased, to give them flexibility (like polymorphism or dynamic sizing) that is not available for arrays in general – It's easier to provide these things for strings than for arrays in general because strings are onedimensional and (more importantly) non-circular

Sets

• We learned about a lot of possible implementations – Bitsets are what usually get built into programming languages – Things like intersection, union, membership, etc. can be implemented efficiently with bitwise logical instructions – Some languages place limits on the sizes of sets to make it easier for the implementor • There is really no excuse for this

Pointers And Recursive Types

• Pointers serve two purposes: – efficient (and sometimes intuitive) access to elaborated objects (as in C) – dynamic creation of linked data structures, in conjunction with a heap storage manager

• Several languages (e.g. Pascal, Ada 83) restrict pointers to accessing things in the heap • Pointers are used with a value model of variables – They aren't needed with a reference model

Pointers And Recursive Types

Figure 8.12 Implementation of a tree in Lisp. A diagonal slash through a box indicates a null pointer. The C and A tags serve to distinguish the two kinds of memory blocks: cons cells and blocks containing atoms.

Pointers And Recursive Types

Figure 8.13 Typical implementation of a tree in a language with explicit pointers. As in Figure 8.12, a diagonal slash through a box indicates a null pointer.

Pointers And Recursive Types

• C pointers and arrays int *a == int a[] int **a == int *a[]

• BUT equivalences don't always hold – Specifically, a declaration allocates an array if it specifies a size for the first dimension – otherwise it allocates a pointer int **a, int *a[] pointer to pointer to int int *a[n], n-element array of row pointers int a[n][m], 2-d array

Pointers And Recursive Types

• Compiler has to be able to tell the size of the things to which you point – So the following aren't valid: int a[][] int (*a)[]

bad bad

– C declaration rule: read right as far as you can (subject to parentheses), then left, then out a level and repeat int *a[n], n-element array of pointers to integer int (*a)[n], pointer to n-element array of integers

Pointers And Recursive Types

• Problems with dangling pointers are due to – explicit deallocation of heap objects • only in languages that have explicit deallocation

– implicit deallocation of elaborated objects • Two implementation mechanisms to catch dangling pointers – Tombstones – Locks and Keys

Pointers And Recursive Types

Figure 8.17 (CD) Tombstones. A valid pointer refers to a tombstone that in turn refers to an object. A dangling reference refers to an “expired” tombstone.

Pointers And Recursive Types

Figure 8.18 (CD) Locks and Keys. A valid pointer contains a key that matches the lock on an object in the heap. A dangling reference is unlikely to match.

Pointers And Recursive Types

• Problems with garbage collection – many languages leave it up to the programmer to design without garbage creation - this is VERY hard – others arrange for automatic garbage collection

– reference counting • does not work for circular structures • works great for strings • should also work to collect unneeded tombstones

Pointers And Recursive Types

• Garbage collection with reference counts

Figure 8.14 Reference counts and circular lists. The list shown here cannot be found via any program variable, but because it is circular, every cell contains a nonzero count.

Pointers And Recursive Types

• Mark-and-sweep – commonplace in Lisp dialects – complicated in languages with rich type structure, but possible if language is strongly typed – achieved successfully in Java, C#, Scala, Go – complete solution impossible in languages that are not strongly typed – conservative approximation possible in almost any language (Xerox Portable Common Runtime approach)

Pointers And Recursive Types

Figure 8.15 Heap exploration via pointer reversal.

Lists

• A list is defined recursively as either the empty list or a pair consisting of an object (which may be either a list or an atom) and another (shorter) list – Lists are ideally suited to programming in functional and logic languages • In Lisp, in fact, a program is a list, and can extend itself at run time by constructing a list and executing it – Lists can also be used in imperative programs

Files and Input/Output

• Input/output (I/O) facilities allow a program to communicate with the outside world – interactive I/O and I/O with files

• Interactive I/O generally implies communication with human users or physical devices • Files generally refer to off-line storage implemented by the operating system • Files may be further categorized into – temporary – persistent

Chapter 9 :: Subroutines and Control Abstraction

Programming Language Pragmatics, Fourth Edition Michael L. Scott

Copyright © 2016 Elsevier

Review Of Stack Layout

• Allocation strategies – Static • Code

• Globals • Own variables • Explicit constants (including strings, sets, other aggregates)

• Small scalars may be stored in the instructions themselves

Review Of Stack Layout

Review Of Stack Layout

• Allocation strategies (2) – Stack • parameters

• local variables • temporaries

• bookkeeping information

– Heap • dynamic allocation

Review Of Stack Layout • Contents of a stack frame –bookkeeping •return PC (dynamic link) •saved registers •line number •saved display entries •static link

–arguments and returns –local variables –temporaries

Calling Sequences

• Maintenance of stack is responsibility of calling sequence and subroutine prolog and epilog – discussed in Chapter 3 – space is saved by putting as much in the prolog and epilog as possible – time may be saved by putting stuff in the caller instead, where more information may be known • e.g., there may be fewer registers IN USE at the point of call than are used SOMEWHERE in the callee

Calling Sequences

• Common strategy is to divide registers into caller-saves and callee-saves sets – caller uses the "callee-saves" registers first – "caller-saves" registers if necessary

• Local variables and arguments are assigned fixed OFFSETS from the stack pointer or frame pointer at compile time – some storage layouts use a separate arguments pointer – the VAX architecture encouraged this

Calling Sequences

Calling Sequences (LLVM on ARM) • Caller – saves into the “local variable and temporaries” area any caller-saves registers whose values are still needed – puts up to 4 small arguments into registers r0-r3 – puts the rest of the arguments into the argument build area at the top of the current frame – does b1 or b1x, which puts return address into register lr, jumps to target address, and (optionally) changes instruction set coding

Calling Sequences (LLVM on ARM) • In prolog, Callee – pushes necessary registers onto stack – initializes frame pointer by adding small constant to the sp placing result in r7 – subtracts from sp to make space for local variables, temporaries, and arg build area at top of stack

• In epilog, Callee – puts return value into r0-r3 or memory (as appropriate) – subtracts small constant from r7, puts result in sp (effectively deallocates most of frame) – pops saved registers from stack, pc takes place of lr from prologue (branches to caller as side effect)

Calling Sequences (LLVM on ARM)

• After call, Caller – moves return value to wherever it's needed – restores caller-saves registers lazily over time, as their values are needed

• All arguments have space in the stack, whether passed in registers or not • The subroutine just begins with some of the arguments already cached in registers, and 'stale' values in memory

Calling Sequences (LLVM on ARM)

• This is a normal state of affairs; optimizing compilers keep things in registers whenever possible, flushing to memory only when they run out of registers, or when code may attempt to access the data through a pointer or from an inner scope

Calling Sequences (LLVM on ARM)

• Many parts of the calling sequence, prologue, and/or epilogue can be omitted in common cases – particularly LEAF routines (those that don't call other routines) • leaving things out saves time • simple leaf routines don't use the stack - don't even use memory – and are exceptionally fast

Parameter Passing

• Parameter passing mechanisms have four basic implementations – value – value/result (copying) – reference (aliasing) – closure/name

• Many languages (e.g. Ada, C++, Swift) provide value and reference directly

Parameter Passing

• C/C++: functions – parameters passed by value (C) – parameters passed by reference can be simulated with pointers (C) void proc(int* x,int y){*x = *x+y } … proc(&a,b);

– or directly passed by reference (C++) int& x, int y) {x = x + y } proc(a,b); void proc(

Parameter Passing

• Ada goes for semantics: who can do what – In: callee reads only – Out: callee writes and can then read (formal not initialized); actual modified – In out: callee reads and writes; actual modified • Ada in/out is always implemented as – value/result for scalars, and either – value/result or reference for structured objects

Parameter Passing

• In a language with a reference model of variables (ML, Ruby), pass by reference (sharing) is the obvious approach • It's also the only option in Fortran • If you pass a constant, the compiler creates a temporary location to hold it • If you modify the temporary, who cares?

• Call-by name is an old Algol technique • Think of it as call by textual substitution (procedure with all name parameters works like macro) - what you pass are hidden procedures called THUNKS

Parameter Passing

Exception Handling

• What is an exception?

– a hardware-detected run-time error or unusual condition detected by software

• Examples – arithmetic overflow – end-of-file on input – wrong type for input data – user-defined conditions, not necessarily errors

Exception Handling

• What is an exception handler?

– code executed when exception occurs – may need a different handler for each type of exception

• Why design in exception handling facilities?

– allow user to explicitly handle errors in a uniform manner – allow user to handle errors without having to check these conditions explicitly in the program everywhere they might occur

Coroutines

• Coroutines are execution contexts that exist concurrently, but that execute one at a time, and that transfer control to each other explicitly, by name • Coroutines can be used to implement – iterators (Section 6.5.3) – threads (to be discussed in Chapter 12)

• Because they are concurrent (i.e., simultaneously started but not completed), coroutines cannot share a single stack

Coroutines

Chapter 10 :: Data Abstraction and Object Orientation

Programming Language Pragmatics, Fourth Edition Michael L. Scott

Copyright © 2016 Elsevier

Object-Oriented Programming

• Control or PROCESS abstraction is a very old idea (subroutines!), though few languages provide it in a truly general form (Scheme comes close) • Data abstraction is somewhat newer, though its roots can be found in Simula67 – An Abstract Data Type is one that is defined in terms of the operations that it supports (i.e., that can be performed upon it) rather than in terms of its structure or implementation

Object-Oriented Programming

• Why abstractions?

– easier to think about - hide what doesn't matter to programmer – protection - prevent access to things you shouldn't see – plug compatibility • replacement of pieces, often without recompilation, definitely without rewriting libraries • division of labor in software projects

Object-Oriented Programming • We talked about data abstraction some back in the unit on naming and scoping (Ch. 3) • Recall that we traced the historical development of abstraction mechanisms –Static set of variables –Locals –Statics –Modules –Module types –Objects

Basic Fortran Fortran, Algol 60, C Modula-2, Ada 83 Euclid Smalltalk, C++, Eiffel, Java Oberon, Modula-3, Ada 95

Object-Oriented Programming

• Statics allow a subroutine to retain values from one invocation to the next, while hiding the name in-between • Modules allow a collection of subroutines to share some statics, still with hiding – If you want to build an abstract data type, though, you have to make the module a manager

Object-Oriented Programming

• Module types allow the module to be the abstract data type - you can declare a bunch of them – This is generally more intuitive • It avoids explicit object parameters to many operations • One minor drawback: If you have an operation that needs to look at the innards of two different types, you'd define both types in the same manager module in Modula-2 • In C++ you need to make one of the classes (or some of its members) "friends" of the other class

Object-Oriented Programming

• Objects add inheritance and dynamic method binding • Simula 67 introduced these, but didn't have data hiding • The 3 key factors in OO programming – Encapsulation (data hiding) – Inheritance – Dynamic method binding

Encapsulation and Inheritance

• Visibility rules – Public and Private parts of an object declaration/definition – 2 reasons to put things in the declaration • so programmers can get at them • so the compiler can understand them

– At the very least the compiler needs to know the size of an object, even though the programmer isn't allowed to get at many or most of the fields (members) that contribute to that size • That's why private fields have to be in declaration

Encapsulation and Inheritance Classes (C++)

• C++ distinguishes among –public class members •accessible to anybody

–protected class members •accessible to members of this or derived classes

–private •accessible just to members of this class

• A C++ structure (struct) is simply a class whose members are public by default • C++ base classes can also be public, private, or protected

Encapsulation and Inheritance Classes (C++)

• Example: class circle : public shape { ... anybody can convert (assign) a circle* into a shape*

class circle : protected shape { ... only members and friends of circle or its derived classes can convert (assign) a circle* into a shape* class circle : private shape { ... only members and friends of circle can convert (assign) a circle* into a shape*

Encapsulation and Inheritance Classes (C++)

• Disadvantage of the module-as-manager approach: must include explicit create/initialize and destroy/finalize routines for every abstraction – Even w/o dynamic allocation inside module, users don't have necessary knowledge to do initialization – Ada 83 is a little better here: you can provide initializers for pieces of private types, but this is NOT a general approach – Object-oriented languages often give you constructors and maybe destructors • Destructors are important primarily in the absence of garbage collection

Encapsulation and Inheritance Classes (C++)

• A few C++ features you may not have learned: – classes as members foo::foo (args0) : member1 (args1), member2 (args2) { ... args1 and args2 need to be specified in terms of args0 • The reason these things end up in the header of foo is that they get executed before foo's constructor does, and the designers consider it good style to make that clear in the header of foo::foo

Encapsulation and Inheritance Classes (C++)

• A few C++ features (2): – initialization v. assignment foo::operator=(&foo) v. foo::foo(&foo) foo b; foo f = b; // calls constructor foo b, f; // calls no-argument constructor f = b; // calls operator=

Encapsulation and Inheritance Classes (C++)

• A few C++ features (3): – virtual functions (see the next dynamic method binding section for details): Key question: if child is derived from parent and I have a parent* p (or a parent& p) that points (refers) to an object that's actually a child, what member function do I get when I call p->f (p.f)?

• Normally I get p's f, because p's type is parent*.

• But if f is a virtual function, I get c's f.

Encapsulation and Inheritance Classes (C++)

• A few C++ features (4): –virtual functions (continued) •If a virtual function has a "0" body in the parent class, then the function is said to be a pure virtual function and the parent class is said to be abstract •You can't declare objects of an abstract class; you have to declare them to be of derived classes •Moreover any derived class must provide a body for the pure virtual function(s) •multiple inheritance in Standard C++ (see next)

–friends •functions •classes

Initialization and Finalization

• In Section 3.2, we defined the lifetime of an object to be the interval during which it occupies space and can hold data – Most object-oriented languages provide some sort of special mechanism to initialize an object automatically at the beginning of its lifetime • When written in the form of a subroutine, this mechanism is known as a constructor • A constructor does not allocate space

– A few languages provide a similar destructor mechanism to finalize an object automatically at the end of its lifetime

Initialization and Finalization

Issues • choosing a constructor • references and values – If variables are references, then every object must be created explicitly - appropriate constructor is called – If variables are values, then object creation can happen implicitly as a result of elaboration

• execution order – When an object of a derived class is created in C++, the constructors for any base classes will be executed before the constructor for the derived class

• garbage collection

Dynamic Method Binding

• Virtual functions in C++ are an example of dynamic method binding – you don't know at compile time what type the object referred to by a variable will be at run time

• Simula also has virtual functions (all of which are abstract) • In Smalltalk, Eiffel, and Java all member functions are virtual

Dynamic Method Binding

• Note that inheritance does not obviate the need for generics – You might think: “hey, I can define an abstract list class and then derive int_list, person_list, etc. from it”, but the problem is you won't be able to talk about the elements because you won't know their types – That's what generics are for: abstracting over types

• Java doesn't have generics, but it does have (checked) dynamic casts

Dynamic Method Binding

• Data members of classes are implemented just like structures (records) – With (single) inheritance, derived classes have extra fields at the end – A pointer to the parent and a pointer to the child contain the same address - the child just knows that the struct goes farther than the parent does

Dynamic Method Binding

• Non-virtual functions require no space at run time; the compiler just calls the appropriate version, based on type of variable – Member functions are passed an extra, hidden, initial parameter: this (called current in Eiffel and self in Smalltalk)

• C++ philosophy is to avoid run-time overhead whenever possible(Sort of the legacy from C) – Languages like Smalltalk have (much) more run-time support

Dynamic Method Binding

• Virtual functions are the only thing that requires any trickiness (Figure 10.3) – They are implemented by creating a dispatch table (vtable) for the class and putting a pointer to that table in the data of the object – Objects of a derived class have a different dispatch table • In the dispatch table, functions defined in the parent come first, though some of the pointers point to overridden versions • You could put the whole dispatch table in the object itself, saving a little time, but potentially wasting a LOT of space

Dynamic Method Binding

Dynamic Method Binding

Dynamic Method Binding

• Note that if you can query the type of an object, then you need to be able to get from the object to run-time type info – The standard implementation technique is to put a pointer to the type info at the beginning of the vtable – Of course you only have a vtable in C++ if your class has virtual functions • That's why you can't do a dynamic_cast on a pointer whose static type doesn't have virtual functions

Mix-In Inheritance

• Classes can inherit from only one “real” parent • Can “mix in” any number of interfaces, simulating multiple inheritance • Interfaces appear in Java, C#, Go, Ruby, etc.

– contain only abstract methods, no method bodies or fields

• Has become dominant approach, superseding true multiple inheritance

Mix-In Inheritance

True Multiple Inheritance

• In C++, you can say class professor : public teacher, public researcher { ... } Here you get all the members of teacher and all the members of researcher – If there's anything that's in both (same name and argument types), then calls to the member are ambiguous; the compiler disallows them

True Multiple Inheritance • You can of course create your own member in the merged class professor::print () { teacher::print (); researcher::print (); ... }

Or you could get both: professor::tprint () { teacher::print (); } professor::rprint () { researcher::print (); }

True Multiple Inheritance

• Virtual base classes: In the usual case if you inherit from two classes that are both derived from some other class B, your implementation includes two copies of B's data members • That's often fine, but other times you want a single copy of B – For that you make B a virtual base class

Object-Oriented Programming • Analogy to the real world is central to the OO paradigm - you think in terms of real-world objects that interact to get things done • Many OO languages are strictly sequential, but the model adapts well to parallelism as well • Strict interpretation of the term – uniform data abstraction - everything is an object – inheritance – dynamic method binding

Object-Oriented Programming

• Lots of conflicting uses of the term out there object-oriented style available in many languages – data abstraction crucial – inheritance required by most users of the term OO – centrality of dynamic method binding a matter of dispute

Object-Oriented Programming

• SMALLTALK is, historically, the canonical object-oriented language – It has all three of the characteristics listed above – It's based on the thesis work of Alan Kay at Utah in the late 1960‘s – It went through 5 generations at Xerox PARC, where Kay worked after graduating – Smalltalk-80 is the current standard

Object-Oriented Programming

• Other languages are described in what follows: • Modula-3 – single inheritance – all methods virtual – no constructors or destructors

• Java – interfaces, mix-in inheritance – all methods virtual

Object-Oriented Programming • Ada 95 –tagged types –single inheritance –no constructors or destructors –class-wide parameters: •methods static by default •can define a parameter or pointer that grabs the object-specific version of all methods –base class doesn't have to decide what will be virtual

–notion of child packages as an alternative to friends

Object-Oriented Programming

• Is C++ object-oriented?

– Uses all the right buzzwords – Has (multiple) inheritance and generics (templates) – Allows creation of user-defined classes that look just like built-in ones – Has all the low-level C stuff to escape the paradigm – Has friends – Has static type checking

Object-Oriented Programming

• In the same category of questions: – Is Prolog a logic language?

– Is Common Lisp functional?

• However, to be more precise: – Smalltalk is really pretty purely object-oriented – Prolog is primarily logic-based – Common Lisp is largely functional – C++ can be used in an object-oriented style

Chapter 11 :: Functional Languages

Programming Language Pragmatics Michael L. Scott

Copyright © 2016 Elsevier

Historical Origins

• The imperative and functional models grew out of work undertaken Alan Turing, Alonzo Church, Stephen Kleene, Emil Post, etc. ~1930s – different formalizations of the notion of an algorithm, or effective procedure, based on automata, symbolic manipulation, recursive function definitions, and combinatorics

• These results led Church to conjecture that any intuitively appealing model of computing would be equally powerful as well – this conjecture is known as Church’s thesis

Historical Origins

• Turing’s model of computing was the Turing machine a sort of pushdown automaton using an unbounded storage “tape” – the Turing machine computes in an imperative way, by changing the values in cells of its tape – like variables just as a high level imperative program computes by changing the values of variables

Historical Origins

• Church’s model of computing is called the lambda calculus – based on the notion of parameterized expressions (with each parameter introduced by an occurrence of the letter λ—hence the notation’s name.

– Lambda calculus was the inspiration for functional programming – one uses it to compute by substituting parameters into expressions, just as one computes in a high level functional program by passing arguments to functions

Historical Origins

• Mathematicians established a distinction between – constructive proof (one that shows how to obtain a mathematical object with some desired property) – nonconstructive proof (one that merely shows that such an object must exist, e.g., by contradiction)

• Logic programming is tied to the notion of constructive proofs, but at a more abstract level: – the logic programmer writes a set of axioms that allow the computer to discover a constructive proof for each particular set of inputs

Functional Programming Concepts

• Functional languages such as Lisp, Scheme, FP, ML, Miranda, and Haskell are an attempt to realize Church's lambda calculus in practical form as a programming language

• The key idea: do everything by composing functions – no mutable state – no side effects

Functional Programming Concepts

• Necessary features, many of which are missing in some imperative languages – 1st class and high-order functions – serious polymorphism – powerful list facilities – structured function returns – fully general aggregates – garbage collection

Functional Programming Concepts • So how do you get anything done in a functional language?

– Recursion (especially tail recursion) takes the place of iteration – In general, you can get the effect of a series of assignments x := 0 ... x := expr1 ... x := expr2 ... from f3(f2(f1(0))), where each f expects the value of x as an argument, f1 returns expr1, and f2 returns expr2

Functional Programming Concepts • Recursion even does a nifty job of replacing looping x := 0; i := 1; j := 100; while i < j do x := x + i*j; i := i + 1; j := j - 1 end while return x becomes f(0,1,100), where f(x,i,j) == if i < j then f (x+i*j, i+1, j-1) else x

Functional Programming Concepts

• Thinking about recursion as a direct, mechanical replacement for iteration, however, is the wrong way to look at things – One has to get used to thinking in a recursive style

• Even more important than recursion is the notion of higher-order functions – Take a function as argument, or return a function as a result – Great for building things

Functional Programming Concepts • Lisp also has (these are not necessary present in other functional languages) –homo-iconography –self-definition –read-evaluate-print

• Variants of LISP –Pure (original) Lisp –Interlisp, MacLisp, Emacs Lisp –Common Lisp –Scheme

Functional Programming Concepts

• Pure Lisp is purely functional; all other Lisps have imperative features • All early Lisps dynamically scoped – Not clear whether this was deliberate or if it happened by accident

• Scheme and Common Lisp statically scoped – Common Lisp provides dynamic scope as an option for explicitly-declared special functions – Common Lisp now THE standard Lisp • Very big; complicated (The Ada of functional programming)

Functional Programming Concepts

• Scheme is a particularly elegant Lisp • Other functional languages – ML – Miranda – Haskell – FP

• Haskell is the leading language for research in functional programming

A Bit of Scheme

• As mentioned earlier, Scheme is a particularly elegant Lisp – Interpreter runs a read-eval-print loop – Things typed into the interpreter are evaluated (recursively) once – Anything in parentheses is a function call (unless quoted) – Parentheses are NOT just grouping, as they are in Algol-family languages • Adding a level of parentheses changes meaning (+ 3 4) ⇒ 7 ((+ 3 4))) ⇒ error (the ' ⇒' arrow means 'evaluates to‘)

A Bit of Scheme • Scheme: – Boolean values #t and #f – Numbers – Lambda expressions – Quoting (+ 3 4) ⇒ 7

(quote (+ 3 4)) ⇒ (+ 3 4) '(+ 3 4) ⇒ (+ 3 4) – Mechanisms for creating new scopes (let ((square (lambda (x) (* x x))) (plus +)) (sqrt (plus (square a) (square b)))) let* letrec

A Bit of Scheme • Scheme: – Conditional expressions (if (< 2 3) 4 5) ⇒ 4 (cond ((< 3 2) 1)

((< 4 3) 2) (else 3)) ⇒ 3 – Imperative stuff • assignments • sequencing (begin) • iteration • I/O (read, display)

A Bit of Scheme • Scheme standard functions (this is not a complete list): –arithmetic –boolean operators –equivalence –list operators –symbol?

–number?

–complex?

–real?

–rational?

–integer?

A Bit of Scheme

Example program - Simulation of DFA

• We'll invoke the program by calling a function called 'simulate', passing it a DFA description and an input string – The automaton description is a list of three items: • start state • the transition function • the set of final states

– The transition function is a list of pairs • the first element of each pair is a pair, whose first element is a state and whose second element in an input symbol • if the current state and next input symbol match the first element of a pair, then the finite automaton enters the state given by the second element of the pair

A Bit of Scheme

Example program - Simulation of DFA

A Bit of Scheme

Example program - Simulation of DFA

A Bit of OCaml

• OCaml is a descendent of ML, and cousin to Haskell, F# – “O” stands for objective, referencing the object orientation introduced in the 1990s – Interpreter runs a read-eval-print loop like in Scheme – Things typed into the interpreter are evaluated (recursively) once – Parentheses are NOT function calls, but indicate tuples

A Bit of OCaml • Ocaml: – Boolean values – Numbers – Chars –Strings –More complex types created by lists, arrays, records, objects, etc.

–(+ - * /) for ints, (+. -. *. /.) for floats – let keyword for creating new names let average = fun x y -> (x +. y) /. 2.;;

A Bit of OCaml • Ocaml: –Variant Types type 'a tree = Empty | Node of 'a * 'a tree * 'a tree;;

–Pattern matching let atomic_number (s, n, w) = n;; let mercury = ("Hg", 80, 200.592);; atomic_number mercury;; ⇒ 80

A Bit of OCaml • OCaml: – Different assignments for references ‘:=’ and array elements ‘<-’ let insertion_sort a =

for i = 1 to Array.length a - 1 do let t = a.(i) in let j = ref i in while !j > 0 && t < a.(!j - 1) do a.(!j) <- a.(!j - 1); j := !j - 1 done; a.(!j) <- t done;;

A Bit of OCaml

Example program - Simulation of DFA

• We'll invoke the program by calling a function called 'simulate', passing it a DFA description and an input string – The automaton description is a record with three fields: • start state • the transition function • the list of final states

– The transition function is a list of triples • the first two elements are a state and an input symbol •if these match the current state and next input, then the automaton enters a state given by the third element

A Bit of OCaml

Example program - Simulation of DFA

A Bit of OCaml

Example program - Simulation of DFA

Evaluation Order Revisited

• Applicative order – what you're used to in imperative languages – usually faster

• Normal order – like call-by-name: don't evaluate arg until you need it – sometimes faster – terminates if anything will (Church-Rosser theorem)

Evaluation Order Revisited

• In Scheme – functions use applicative order defined with lambda – special forms (aka macros) use normal order defined with syntax-rules

• A strict language requires all arguments to be well-defined, so applicative order can be used • A non-strict language does not require all arguments to be well-defined; it requires normal-order evaluation

Evaluation Order Revisited

• Lazy evaluation gives the best of both worlds • But not good in the presence of side effects.

– delay and force in Scheme – delay creates a "promise"

High-Order Functions

• Higher-order functions – Take a function as argument, or return a function as a result – Great for building things – Currying (after Haskell Curry, the same guy Haskell is named after) • For details see Lambda calculus on CD • ML, Miranda, OCaml, and Haskell have especially nice syntax for curried functions

Functional Programming in Perspective

• Advantages of functional languages – lack of side effects makes programs easier to understand – lack of explicit evaluation order (in some languages) offers possibility of parallel evaluation (e.g. MultiLisp) – lack of side effects and explicit evaluation order simplifies some things for a compiler (provided you don't blow it in other ways) – programs are often surprisingly short – language can be extremely small and yet powerful

Functional Programming in Perspective • Problems –difficult (but not impossible!) to implement efficiently on von Neumann machines •lots of copying of data through parameters •(apparent) need to create a whole new array in order to change one element •heavy use of pointers (space/time and locality problem) •frequent procedure calls •heavy space use for recursion •requires garbage collection •requires a different mode of thinking by the programmer •difficult to integrate I/O into purely functional model

Chapter 12 :: Logic Languages

Programming Language Pragmatics, Fourth Edition Michael L. Scott

Copyright © 2016 Elsevier

Logic Programming

• Based on predicate calculus • Predicates - building-blocks P(a1,a2,...,aK) – limit(f, infinity, 0) – enrolled(you, CS xxx) – These are interesting because we attach meaning to them, but within the logical system they are simply structural building blocks, with no meaning beyond that provided by explicitlystated interrelationships

Logic Programming Concepts

• Operators – conjunction, disjunction, negation, implication

• Universal and existential quantifiers • Statements – sometimes true, sometimes false, often unknown – axioms - assumed true – theorems - provably true – hypotheses (goals) - things we'd like to prove true

Logic Programming Concepts

• Example statements:

all f, l [ limit(f, x0, l) <=> (all e [ e > 0 => (exists d [ d > 0 and all x [ ((|x-x0| < d) => (|f(x)-l|) < e)]])])]

all f, g [f = O(g) <= (exist c, n0 [ all n [ n > n0 => f(n) < cg(n)]])]

Logic Programming Concepts • Most statements can be written many ways • That's great for people but a nuisance for computers – It turns out that if you make certain restrictions on the format of statements you can prove theorems mechanically – That's what logic programming systems do – Unfortunately, the restrictions that we will put on our statements will not allow us to handle most of the theorems you learned in math, but we will have a surprising amount of power left anyway

Logic Programming Concepts

• We insist that all statements be in the form of HORN CLAUSES consisting of a HEAD and a BODY – The head is a single term – The body is a list of terms – A term can be a constant, variable, or STRUCTURE consisting of a FUNCTOR and a parenthesized list of arguments

Logic Programming Concepts • A structure can play the role of a data structure or a predicate – A constant is either an ATOM or a NUMBER – An atom is either what looks like an identifier beginning with a lowercase letter, or a quoted character string – A number looks like an integer or real from some more ordinary language – A variable looks like an identifier beginning with an uppercase letter – There are no declarations – All types are discovered implicitly

Logic Programming Concepts

• The meaning of the statement is that the conjunction of the terms in the body implies the head – A clause with an empty body is called a FACT – A clause with an empty head is a QUERY, or top-level GOAL – A clause with both sides is a RULE

• The Prolog interpreter has a collection of facts and rules in its DATABASE – Facts are axioms - things the interpreter assumes to be true

Prolog • Prolog can be thought of declaratively or imperatively: – We’ll emphasize the declarative semantics for now, because that's what makes logic programming interesting – We'll get into the imperative semantics later

• Prolog allows you to state a bunch of axioms – Then you pose a query (goal) and the system tries to find a series of inference steps (and assignments of values to variables) that allow it to prove your query starting from the axioms

Prolog

• The meaning of the statement is that mother(mary, fred). % you can either think of this as % a predicate asserting that mary % is the mother of fred % or a data structure (tree) % in which the functor (atom) % mother is the root, % mary is the left child, and % fred is the right child fat(albert). rainy(rochester).

Prolog

• Rules are theorems that allow the interpreter to infer things • To be interesting, rules generally contain variables employed(X) :- employs(Y,X).

can be read: for all X, X is employed if there exists a Y such that Y employs X

• Note the direction of the implication: – The example does NOT say that X is employed ONLY IF there is a Y that employs X

Prolog

• The scope of a variable is the clause in which it appears – Variables whose first appearance is on the left hand side of the clause have implicit universal quantifiers – Variables whose first appearance is in the body of the clause have implicit existential quantifiers

• Similarly:

Prolog

grandmother(A, C) :- mother(A, B), mother(B, C). can be read: for all A, C [A is the grandmother of C if there exists a B such that A is the mother of B and B is the mother of C].

We probably want another rule that says grandmother(A, C) :- mother(A, B), father(B, C).

Prolog

• To run a Prolog program, one asks the interpreter a question – This is done by stating a theorem - asserting a predicate - which the interpreter tries to prove • If it can, it says yes • If it can't, it says no • If your predicate contained variables, the interpreter prints the values it had to give them to make the predicate true.

Prolog • The interpreter works by what is called BACKWARD CHAINING – It begins with the thing it is trying to prove and works backwards looking for things that would imply it, until it gets to facts

• It is also possible in theory to work forward from the facts trying to see if any of the things you can prove from them are what you were looking for - that can be very timeconsuming – Fancier logic languages use both kinds of chaining, with special smarts or hints from the user to bound the searches

Prolog • The predicate you ask for is the interpreter's original GOAL – In an attempt to SATISFY that goal, it looks for facts or rules with which the goal can be UNIFIED • Unification is a process by which compatible statements are merged

– Any variables that do not yet have values but which correspond to constants or to variables with values in the other clause get INSTANTIATED with that value – Anyplace where uninstantiated variables correspond, those variables are identified with each other, but remain without values

Prolog • The interpreter starts at the beginning of your database (this ordering is part of Prolog, NOT of logic programming in general) and looks for something with which to unify the current goal – If it finds a fact, great; it succeeds – If it finds a rule, it attempts to satisfy the terms in the body of the rule depth first – This process is motivated by the RESOLUTION PRINCIPLE, due to Robinson: • It says that if C1 and C2 are Horn clauses, where C2 represents a true statement and the head of C2 unifies with one of the terms in the body of C1, then we can replace the term in C1 with the body of C2 to obtain another statement that is true if and only if C1 is true

Prolog

• When it attempts resolution, the Prolog interpreter pushes the current goal onto a stack, makes the first term in the body the current goal, and goes back to the beginning of the database and starts looking again • If it gets through the first goal of a body successfully, the interpreter continues with the next one • If it gets all the way through the body, the goal is satisfied and it backs up a level and proceeds

Prolog

• If it fails to satisfy the terms in the body of a rule, the interpreter undoes the unification of the left hand side (this includes uninstantiating any variables that were given values as a result of the unification) and keeps looking through the database for something else with which to unify (This process is called BACKTRACKING) • If the interpreter gets to the end of database without succeeding, it backs out a level (that's how it might fail to satisfy something in a body) and continues from there

Prolog • We can visualize backtracking search as a tree in which the top-level goal is the root and the leaves are facts (see Figure 12.2 - next slide) – The children of the root are all the rules and facts with which the goal can unify – The interpreter does an OR across them: one of them must succeed in order for goal to succeed – The children of a node in the second level of the tree are the terms in the body of the rule – The interpreter does an AND across these: all of them must succeed in order for parent to succeed – The overall search tree then consists of alternating AND and OR levels

Prolog

Prolog

• PROLOG IS NOT PURELY DECLARATIVE – The ordering of the database and the left-toright pursuit of sub-goals gives a deterministic imperative semantics to searching and backtracking – Changing the order of statements in the database can give you different results • It can lead to infinite loops • It can certainly result in inefficiency

• Tree relationships:

Prolog parent(a,b). parent(a,d). parent(a,k). parent(k,l). parent(k,m). parent(b,e). parent(b,f). parent(f,g). parent(f,h). parent(f,i).

% a is the parent of b

ancestor(X,Y) :- parent(X,Y). ancestor(X,Y) :- parent(Z,Y), ancestor(X,Z).

Prolog

• Then the question ?- ancestor(U,h). generates the answers U = f; U = b; U = a; no • The question ?- ancestor(b,U).

generates all nodes in the subtree rooted in b << trace >>

Prolog • If we change the order of the two ancestor rules, we get different execution orders: ?- ancestor(U,h).

U = a; U = b; U = f; no • If we change the order of the subgoals in the compound rule, ancestor(X,Y) :- ancestor(X,Z), parent(Z,Y). we run into an infinite loop (see also Figure 12.2)

Prolog

• Arithmetic: The '=' operator determines whether its operands can be unified ?- A = 37.

A = 37 yes ?- 2 = 2. yes

Math operators are functors (structure names), not functions ?- (2+3) = 5 no

Prolog

• For math we use the built-in operator is ?- is(X, 1+2).

X = 3 yes ?- X is 1+2.

X = 3 yes % LHS of 'is' must be as-yet uninstantiated ?- 1+2 is 4-1. no % RHS of 'is' must already be instantiated ?- X is Y. <error>

Prolog

• Tic-tac-toe (see Figure 12.3 on next slide) – This program finds the next move, given a board configuration – It does not play a whole game (see the book for an extended version that does) – It depends on the ordering of rules • move(A) is the root rule • A is a result parameter

– No winning strategy • each player can force a draw

Prolog

Logic Programming Examples % express that three given squares lie in a %line ordered_line(1,2,3). ordered_line(4,5,6). ordered_line(7,8,9). ordered_line(1,4,7). ordered_line(2,5,8). ordered_line(3,6,9). ordered_line(1,5,9). ordered_line(3,5,7). line(A,B,C) :- ordered_line(A,B,C). line(A,B,C) :- ordered_line(A,C,B). line(A,B,C) :- ordered_line(B,A,C). line(A,B,C) :- ordered_line(B,C,A). line(A,B,C) :- ordered_line(C,A,B). line(A,B,C) :- ordered_line(C,B,A) % we assume a not so perfect opponent

Prolog % the following rules work well move(A) :- good(A), empty(A).

full(A) :- x(A). full(A) :- o(A). empty(A):- not full(A) % strategy (key is ordering following five rules) good(A) :- win(A). good(A) :- block_win(A). good(A) :- split(A). good(A) :- block_split(A). good(A) :- build(A).

Prolog % first choice is to win(1) win(A) :- x(B), x(C), line(A,B,C). % block opponent from winning (2) block_win(A) :- o(B), o(C), line(A,B,C).

% opponent cannot block us from winning next % see Figure 11.6 before for this case(3) split(A) :- x(B), x(C), different(B,C), line(A,B,D), line(A,C,E), empty(D), empty(E). same(A,A). different(A,B) :- not same(A,B).

Prolog % prevent opponent from creating a split (4) block_split(A) :- o(B), o(C), different(B,C), line(A,B,D), line(A,C,E), empty(D), empty(E). % pick a square toward three in a row (5) build(A) :- x(B), line(A,B,C), empty(C). % if none of the five, final defaults (in this % order) good(5). good(1). good(3). good(7). good(9). good(2). good(4). good(6). good(8).

Chapter 13 :: Concurrency

Programming Language Pragmatics, Fourth Edition Michael L. Scott

Copyright © 2016 Elsevier

Background and Motivation

• A PROCESS or THREAD is a potentiallyactive execution context • Classic von Neumann (stored program) model of computing has single thread of control • Parallel programs have more than one • A process can be thought of as an abstraction of a physical PROCESSOR

Background and Motivation • Processes/Threads can come from –multiple CPUs –kernel-level multiplexing of single physical machine –language or library level multiplexing of kernel-level abstraction

• They can run –in true parallel –unpredictably interleaved –run-until-block

• Most work focuses on the first two cases, which are equally difficult to deal with

Background and Motivation • Two main classes of programming notation – synchronized access to shared memory – message passing between processes that don't share memory

• Both approaches can be implemented on hardware designed for the other, though shared memory on message-passing hardware tends to be slow

Background and Motivation

• Race conditions – A race condition occurs when actions in two processes are not synchronized and program behavior depends on the order in which the actions happen – Race conditions are not all bad; sometimes any of the possible program outcomes are ok (e.g. workers taking things off a task queue)

Background and Motivation • Race conditions (we want to avoid race conditions): – Suppose processors A and B share memory, and both try to increment variable X at more or less the same time – Very few processors support arithmetic operations on memory, so each processor executes – LOAD X – INC – STORE X

– Suppose X is initialized to 0. If both processors execute these instructions simultaneously, what are the possible outcomes?

• could go up by one or by two

Background and Motivation • Synchronization – SYNCHRONIZATION is the act of ensuring that events in different processes happen in a desired order – Synchronization can be used to eliminate race conditions – In our example we need to synchronize the increment operations to enforce MUTUAL EXCLUSION on access to X – Most synchronization can be regarded as either • Mutual exclusion (making sure that only one process is executing a CRITICAL SECTION [touching a variable, for example] at a time), or as • CONDITION SYNCHRONIZATION, which means making sure that a given process does not proceed until some condition holds (e.g. that a variable contains a given value)

Background and Motivation • One might be tempted to think of mutual exclusion as a form of condition synchronization (the condition being that nobody else is in the critical section), but it isn't – The distinction is basically existential vs. universal quantification • Mutual exclusion requires multi-process consensus

• We do NOT in general want to over-synchronize – That eliminates parallelism, which we generally want to encourage for performance

• Basically, we want to eliminate "bad" race conditions, i.e., the ones that cause the program to give incorrect results

Background and Motivation • Historical development of shared memory ideas – To implement synchronization you have to have something that is ATOMIC • that means it happens all at once, as an indivisible action • In most machines, reads and writes of individual memory locations are atomic (note that this is not trivial; memory and/or busses must be designed to arbitrate and serialize concurrent accesses) • In early machines, reads and writes of individual memory locations were all that was atomic

– To simplify the implementation of mutual exclusion, hardware designers began in the late 60's to build socalled read-modify-write, or fetch-and-phi, instructions into their machines

Concurrent Programming Fundamentals

• SCHEDULERS give us the ability to "put a thread/process to sleep" and run something else on its process/processor – start with coroutines – make uniprocessor run-until-block threads – add preemption – add multiple processors

Concurrent Programming Fundamentals

Concurrent Programming Fundamentals

Concurrent Programming Fundamentals •

Coroutines – –

•

Multiple execution contexts, only one of which is active Transfer (other): • save all callee-saves registers on stack, including ra and fp • *current := sp • current := other • sp := *current • pop all callee-saves registers (including ra, but NOT sp!)

Run-until block threads on a single process – –

–

– –

Need to get rid of explicit argument to transfer Ready list data structure: threads that are runnable but not running procedure reschedule: t : cb := dequeue(ready_list) transfer(t) • return (into different coroutine!)

Other and current are pointers to CONTEXT BLOCKs • Contains sp; may contain other stuff as well (priority, I/O status, etc.)

No need to change PC; always changes at the same place Create new coroutine in a state that looks like it's blocked in transfer. (Or maybe let it execute and then "detach". That's basically early reply)

Concurrent Programming Fundamentals – To do this safely, we need to save 'current' somewhere - two ways to do this: • Suppose we're just relinquishing the processor for the sake of fairness (as in MacOS or Windows 3.1): procedure yield: enqueue (ready_list, current) reschedule • Now suppose we're implementing synchronization: sleep_on(q) enqueue(q, current) reschedule

– Some other thread/process will move us to the ready list when we can continue

Concurrent Programming Fundamentals

Concurrent Programming Fundamentals • Preemption – Use timer interrupts (in OS) or signals (in library package) to trigger involuntary yields – Requires that we protect the scheduler data structures: procedure yield: disable_signals enqueue(ready_list, current) Reschedule re-enable_signals – Note that reschedule takes us to a different thread, possibly in code other than yield Invariant: EVERY CALL to reschedule must be made with signals disabled, and must re-enable them upon its return disable_signals if not <desired condition> sleep_on <condition queue> re-enable signals

Concurrent Programming Fundamentals • Multiprocessors – Disabling signals doesn't suffice: procedure yield: disable_signals acquire(scheduler_lock) enqueue(ready_list, current) reschedule release(scheduler_lock) re-enable_signals

// spin lock

disable_signals acquire(scheduler_lock) // spin lock if not <desired condition> sleep_on <condition queue> release(scheduler_lock) re-enable signals

Implementing Synchronization • Condition synchronization with atomic reads and writes is easy – You just cast each condition in the form of "location X contains value Y" and you keep reading X in a loop until you see what you want

• Mutual exclusion is harder – Much early research was devoted to figuring out how to build it from simple atomic reads and writes – Dekker is generally credited with finding the first correct solution for two processes in the early 1960s – Dijkstra published a version that works for N processes in 1965 – Peterson published a much simpler two-process solution in 1981

Implementing Synchronization • Repeatedly reading a shared location until it reaches a certain value is known as SPINNING or BUSY-WAITING • A busy-wait mutual exclusion mechanism is known as a SPIN LOCK – The problem with spin locks is that they waste processor cycles – Synchronization mechanisms are needed that interact with a thread/process scheduler to put a process to sleep and run something else instead of spinning – Note, however, that spin locks are still valuable for certain things, and are widely used • In particular, it is better to spin than to sleep when the expected spin time is less than the rescheduling overhead

Implementing Synchronization • SEMAPHORES were the first proposed SCHEDULER-BASED synchronization mechanism, and remain widely used • CONDITIONAL CRITICAL REGIONS and MONITORS came later • Monitors have the highest-level semantics, but a few sticky semantic problem - they are also widely used • Synchronization in Java is sort of a hybrid of monitors and CCRs (Java 3 will have true monitors.)

• Shared-memory synch in Ada 95 is yet another hybrid

Implementing Synchronization

• A semaphore is a special counter • It has an initial value and two operations, P and V, for changing that value • A semaphore keeps track of the difference between the number of P and V operations that have occurred • A P operation is delayed (the process is descheduled) until #P-#V <= C, the initial value of the semaphore

Implementing Synchronization

Note: a possible implementation is shown on the next slide

Implementing Synchronization

Implementing Synchronization

Implementing Synchronization • It is generally assumed that semaphores are fair, in the sense that processes complete P operations in the same order they start them • Problems with semaphores – They're pretty low-level.

• When using them for mutual exclusion, for example (the most common usage), it's easy to forget a P or a V, especially when they don't occur in strictly matched pairs (because you do a V inside an if statement, for example, as in the use of the spin lock in the implementation of P)

– Their use is scattered all over the place.

• If you want to change how processes synchronize access to a data structure, you have to find all the places in the code where they touch that structure, which is difficult and error-prone

Language-Level Mechanisms • Monitors were an attempt to address the two weaknesses of semaphores listed above • They were suggested by Edsger W. Dijkstra, developed more thoroughly by Brinch Hansen, and formalized nicely by Tony Hoare (a real cooperative effort!) in the early 1970s • Several parallel programming languages have incorporated monitors as their fundamental synchronization mechanism – none incorporate the precise semantics of Hoare's formalization

Language-Level Mechanisms • A monitor is a shared object with operations, internal state, and a number of condition queues.

Only one operation of a given monitor may be active at a given point in time • A process that calls a busy monitor is delayed until the monitor is free – On behalf of its calling process, any operation may suspend itself by waiting on a condition – An operation may also signal a condition, in which case one of the waiting processes is resumed, usually the one that waited first

Language-Level Mechanisms • The precise semantics of mutual exclusion in monitors are the subject of considerable dispute. Hoare's original proposal remains the clearest and most carefully described – It specifies two bookkeeping queues for each monitor: an entry queue, and an urgent queue – When a process executes a signal operation from within a monitor, it waits in the monitor's urgent queue and the first process on the appropriate condition queue obtains control of the monitor – When a process leaves a monitor it unblocks the first process on the urgent queue or, if the urgent queue is empty, it unblocks the first process on the entry queue instead

Language-Level Mechanisms •

Building a correct monitor requires that one think about the "monitor invariant“. The monitor invariant is a predicate that captures the notion "the state of the monitor is consistent."

– It needs to be true initially, and at monitor exit – It also needs to be true at every wait statement – In Hoare's formulation, needs to be true at every signal operation as well, since some other process may immediately run

• Hoare's definition of monitors in terms of semaphores makes clear that semaphores can do anything monitors can • The inverse is also true; it is trivial to build a semaphores from monitors (Exercise)

Chapter 14 :: Scripting Languages

Programming Language Pragmatics, Fourth Edition Michael L. Scott

Copyright © 2016 Elsevier

What Is A Scripting Language • Modern scripting languages have two principal sets of ancestors.

– command interpreters or “shells” of traditional batch and “terminal” (command-line) computing • IBM’s JCL, MS-DOS command interpreter, Unix sh and csh – various tools for text processing and report generation • IBM’s RPG, and Unix’s sed and awk.

• From these evolved – Rexx, IBM’s “Restructured Extended Executor,” which dates from 1979 – Perl, originally devised by Larry Wall in the late 1980s, and now one of the most widely used general purpose scripting language.

– Other general purpose scripting languages include Tcl (“tickle”), Python, Ruby, VBScript (for Windows) and AppleScript (for the Mac)

What Is A Scripting Language

• Common Characteristics – Both batch and interactive use – Economy of expression – Lack of declarations; simple scoping rules.

– Flexible dynamic typing – Easy access to other programs – Sophisticated pattern matching and string manipulation – High level data types

Problem Domains • Some general purpose languages—Scheme and Visual Basic in particular—are widely used for scripting • Conversely, some scripting languages, including Perl, Python, and Ruby, are intended by their designers for general purpose use, with features intended to support “programming in the large” – modules, separate compilation, reflection, program development environments

• For the most part, however, scripting languages tend to see their principal use in well defined problem domains

Problem Domains • Shell Languages – They have features designed for interactive use – Provide a wealth of mechanisms to manipulate file names, arguments, and commands, and to glue together other programs • Most of these features are retained by more general scripting languages – We consider a few of them - full details can be found in the bash man page, or in various online tutorials: • Filename and Variable Expansion • Tests, Queries, and Conditions • Pipes and Redirection • Quoting and Expansion • Functions • The #! Convention

Problem Domains • Text Processing and Report Generation – Sed

Problem Domains • Text Processing and Report Generation – Awk

Problem Domains • Perl – Perl was originally developed by Larry Wall in 1987, while he was working at the NSA – The original version was an attempt to combine sed, awk, and sh – It was a Unix-only tool, meant primarily for text processing (the name stands for “practical extraction and report language”) • over the years Perl has grown into a large and complex language, – Perl is almost certainly the most popular and widely used scripting language – It is also fast enough for much general purpose use, and includes • separate compilation, modularization, and dynamic library mechanisms appropriate for large-scale projects – It has been ported to almost every known operating system

Problem Domains

• Mathematics and Statistics – APL

Problem Domains • “Glue” Languages and General Purpose Scripting –

As noted, Rexx is generally considered the first of the general purpose scripting languages, predating Perl and Tcl by almost a decade

–

Perl and Tcl are roughly contemporaneous: both were initially developed in the late 1980s • Perl was originally intended for glue and text processing applications

• Tcl was originally an extension language, but soon grew into glue applications –

Python was originally developed by Guido van Rossum at CWI in Amsterdam, the Netherlands, in the early 1990s • He continued his work at CNRI in Reston, Virginia, beginning in 1995 • In 2000 the Python team moved to BeOpen.com, and to Digital Creations

• Recent versions of the language are owned by the Python Software – All releases are Open Source.

– Ruby • As the popularity of scripting grew in the 1990s, users were motivated to develop additional languages, to provide additional features, address the needs of specific application domains or support a style of programming

Problem Domains

Problem Domains

Problem Domains • “Glue” Languages and General Purpose Scripting – Perl – Python – Ruby • Yukihiro Matsumoto wanted a language “more powerful than Perl, and more object-oriented than Python” • Success helped by Ruby on Rails web-development framework which was adopted by Apple, Twitter, and others • Ousterhout joined Sun Microsystems in 1994, where for three years he led a multiperson team devoted to Tcl development – In comparison to Perl, Tcl is somewhat more verbose • It makes less use of punctuation, and has fewer special cases

Problem Domains • Extension Languages – Most applications accept some sort of commands • these commands are entered textually or triggered by user interface events such as mouse clicks, menu selections, and keystrokes • Commands in a graphical drawing program might save or load a drawing; select, insert, delete, or modify its parts; choose a line style, weight, or color; zoom or rotate the display; or modify user preferences.

– An extension language serves to increase the usefulness of an application by allowing the user to create new commands, generally using the existing commands as primitives.

– Extension languages are increasingly seen as an essential feature of sophisticated tools • Adobe’s graphics suite (Illustrator, Photoshop, InDesign, etc.) can be extended (scripted) using JavaScript, Visual Basic (on Windows), or AppleScript • AOLserver, an open-source web server from America On-Line, can be scripted using Tcl. Disney and Industrial Light and Magic use Python to extend their internal (proprietary) tools

Problem Domains • Extension Languages – To admit extension, a tool must • incorporate, or communicate with, an interpreter for a scripting language • provide hooks that allow scripts to call the tool’s existing commands • allow the user to tie newly de.ned commands to user interface events – With care, these mechanisms can be made independent of any particular scripting language – One of the oldest existing extension mechanisms is that of the emacs text editor, used to write this book • An enormous number of extension packages have been created for emacs; many of them are installed by default in the standard distribution.

• The extension language for emacs is a dialect of Lisp called Emacs Lisp.

• An example script appears in Figure 13.9 – It assumes that the user has used the standard marking mechanism to select a region of text

Problem Domains

Scripting the World Wide Web • CGI Scripts – The original mechanism for server-side web scripting is the Common Gateway Interface (CGI) – A CGI script is an executable program residing in a special directory known to the web server program – When a client requests the URI corresponding to such a program, the server executes the program and sends its output back to the client • this output needs to be something that the browser will understand: typically HTML. – CGI scripts may be written in any language available • Perl is particularly popular: – its string-handling and “glue” mechanisms are suited to generating HTML – it was already widely available during the early years of the web

Scripting the World Wide Web

Scripting the World Wide Web • Embedded Server-Side Scripts – Though widely used, CGI scripts have several disadvantages: • The web server must launch each script as a separate program, with potentially significant overhead – Though, CGI script compiled to native code can be very fast once running • Scripts must generally be installed in a trusted directory by trusted system administrators – they cannot reside in arbitrary locations as ordinary pages do • The name of the script appears in the URI, typically prefixed with the name of the trusted directory, so static and dynamic pages look different to end users • Each script must generate not only dynamic content, but also the HTML tags that are needed to format and display it – This extra “boilerplate” makes scripts more difficult to write – Most web servers now provide a “module loading” mechanism that allows interpreters for one or more scripting languages

Scripting the World Wide Web

Scripting the World Wide Web • Client-Side Scripts – embedded server-side scripts are generally faster than CGI script, at least when startup cost predominates • communication across the Internet is still too slow for interactive pages – Because they run on the web designer’s site, CGI scripts and, to a lesser extent, embeddable server-side scripts can be written in many different languages • all the client ever sees is standard HTML. – Client-side scripts, by contrast, require an interpreter on the client’s machine • there is a powerful incentive for convergence in client-side scripting languages: most designers want their pages to be viewable by as wide an audience as possible •

interest are known to run Internet Explorer, pages intended for the general public almost

•

always use JavaScript for interactive features.

Scripting the World Wide Web • Client-Side Scripts – While Powershell is widely used within specific organizations, where all the clients of interest are known to run Internet Explorer, pages intended for the general public almost always use JavaScript for interactive features

• Embedded elements – An embedded element is a program designed to run inside some other program, like a Java Applet – Useful especially for embedding video when HTML couldn’t handle the task well – Adobe’s Flash Player is most widely used plug-in for embedding content in web pages

Scripting the World Wide Web

Innovative Features

• Earlier we listed several common characteristics of scripting languages – see text for more details: – both batch and interactive use – economy of expression – lack of declarations; simple scoping rules – flexible dynamic typing – easy access to other programs – sophisticated pattern matching and string manipulation – high level data types

Innovative Features • Most scripting languages (Scheme is the obvious exception) do not require variables to be declared • Perl and JavaScript permit optional declarations - sort of compiler-checked documentation • Perl can be run in a mode (use strict ’vars’) that requires declarations – With or without declarations, most scripting languages use dynamic typing

• The interpreter can perform type checking at run time, or coerce values when appropriate

Innovative Features • Nesting and scoping conventions vary quite a bit – Scheme, Python, JavaScript provide the classic combination of nested subroutines and static (lexical) scope – Tcl allows subroutines to nest, but uses dynamic scope – Named subroutines (methods) do not nest in PHP or Ruby • Perl and Ruby join Scheme, Python, JavaScript, in providing first class anonymous local subroutines – Nested blocks are statically scoped in Perl • In Ruby they are part of the named scope in which they appear – Scheme, Perl, Python provide for variables captured in closures – PHP and the major glue languages (Perl, Tcl, Python, Ruby) all have sophisticated namespace • mechanisms for information hiding and the selective import of names from separate modules

Innovative Features • String and Pattern Manipulation – Regular expressions are present in many scripting languages and related tools employ extended versions of the notation • extended regular expressions in sed (Figure 14.1) awk (Figures 14.2 and 14.3), Perl (Figures 14.4 and 14.5), Python (Figure 14.6), and Ruby (Figure 14.7) • grep, the stand-alone Unix is a pattern-matching tool

– Two main groups.

• The first group includes awk, egrep (the most widely used of several different versions of grep), the regex routines of the C standard library, and older versions of Tcl – These implement REs as defined in the POSIX standard

• Languages in the second group follow the lead of Perl, which provides a large set of extensions, sometimes referred to as “advanced REs”

Innovative Features • Data Types – As we have seen, scripting languages don’t generally require (or even permit) the declaration of types for variables – Most perform extensive run-time checks to make sure that values are never used in inappropriate ways – Some languages (e.g., Scheme, Python, and Ruby) are relatively strict about this checking • When the programmer who wants to convert from one type to another must say so explicitly

– Perl (and likewise Rexx and Tcl) takes the position that programmers should check for the errors they care about • in the absence of such checks the program should do something reasonable

Innovative Features • Object Orientation – Perl 5 has features that allow one to program in an objectoriented style – PHP and JavaScript have cleaner, more conventional-looking object-oriented features • both allow the programmer to use a more traditional imperative style

– Python and Ruby are explicitly and uniformly object-oriented – Perl uses a value model for variables; objects are always accessed via pointers – In PHP and JavaScript, a variable can hold either a value of a primitive type or a reference to an object of composite type.

• In contrast to Perl, however, these languages provide no way to speak of the reference itself, only the object to which it refers

Innovative Features • Object Orientation (2) – Python and Ruby use a uniform reference model – Classes are themselves objects in Python and Ruby – They are types in PHP, much as they are in C++, Java, or C# – Classes in Perl are an alternative way of looking at packages – JavaScript, remarkably, has objects but no classes • its inheritance is based on a concept known as prototypes

– While Perl’s mechanisms suffice to create object-oriented programs, dynamic lookup makes them slower than equivalent imperative programs – Both PHP and JavaScript are more explicitly object-oriented

Chapter 15 :: Building a Runnable Program

Programming Language Pragmatics, Fourth Edition Michael L. Scott

Copyright © 2016 Elsevier

Introduction • The phases of compilation are commonly grouped in – front end responsible for the analysis of source code – back end responsible for the synthesis of target code.

• Chapters 2 and 4 discussed the work of the front end, culminating in the construction of a syntax tree • The current chapter turns to the work of the backend, and specifically to code generation, assembly, and linking • We continue with code improvement in Chapter 17 • Now we look at how the compiler produces that code from a syntax tree, generating a runnable program

Back-End Compiler Structure

• A plausible seven-phase structure for a conventional compiler is shown in Figure 15.1: – First three phases (scanning, parsing, and semantic analysis) are language-dependent – Last two (target code generation and machinespecific code improvement) are machine dependent – The middle two (intermediate code generation and machine-independent code improvement) are dependent on neither the language nor the machine

Back-End Compiler Structure

• The scanner and parser drive a set of action routines that build a syntax tree • The semantic analyzer traverses the tree, performing all static semantic checks and initializing various attributes of use to the back end

Back-End Compiler Structure

Back-End Compiler Structure • Certain code improvements can be performed on syntax trees, but a less hierarchical representation of the program makes most code improvement easier • Our example compiler therefore includes an explicit phase for intermediate code generation – The code generator groups the nodes into basic blocks – It then creates a control flow graph in which the nodes are basic blocks and the arcs represent interblock control flow • Within each basic block, operations are represented as instructions for an idealized machine with an unlimited number of registers - we will call these virtual registers • By allocating a new one for every computed value, the compiler can avoid creating artificial connections between otherwise independent computations too early in the compilation process

Back-End Compiler Structure • The machine-independent code improvement phase performs transformations on the control flow graph.

– local code improvement - it modifies the instruction sequence within each basic block to eliminate redundant loads, stores, and arithmetic computations – global code improvement - it also identifies and removes a variety of redundancies across the boundaries between basic blocks within a subroutine – an expression whose value is computed immediately before an if statement need not be recomputed after else – An expression that appears within the body of a loop need only be evaluated once if its value will not change in subsequent iterations

• Some global improvements change the number of basic blocks and/or the arcs among them

Back-End Compiler Structure

Back-End Compiler Structure • The next phase of compilation is target code generation – This phase strings the basic blocks together into a linear program, translating each block into the instruction set of the target machine and generating branch instructions (or “fall-throughs”) that correspond to the arcs of the control flow graph.

• The output of this phase differs from real assembly language primarily in its continued reliance on virtual registers

Back-End Compiler Structure

Back-End Compiler Structure • The final phase of our example compiler structure consists of register allocation and instruction scheduling - machinespecific code improvement • Register allocation requires that we map the unlimited virtual registers onto the bounded set of registers available in the target machine – If there aren’t enough architectural registers to go around, we may need to generate additional loads and stores to multiplex a given architectural register among two or more virtual registers – As described in Section 5.5, instruction scheduling consists of reordering the instructions of each basic block to fill the pipeline(s) of the target machine

Back-End Compiler Structure •

Phases and Passes – A pass of compilation is a phase or sequence of phases that is serialized with respect to the rest of compilation • it does not start until previous phases have completed • it finishes before any subsequent phases start.

• if desired, a pass may be written as a separate program, reading its input from a file and writing its output to a file.

-- Two-pass compilers are particularly common they may be divided between the front end and the back end (between semantic analysis and intermediate code generation) or • they may be divided between intermediate code generation and global code improvement • In the latter case, the first pass is still commonly referred to as the front end and the second pass as the back end

Intermediate Forms • An intermediate form (IF) provides the connection between the front end and the back end of the compiler, and continues to represent the program during the various back-end phases.

• IFs can be classified in terms of their level, or degree of machine dependence.

• High level IFs – IFs are often based on trees or directed acyclic graphs (DAGs) that directly capture the structure of modern programming languages – facilitates certain kinds of machine-independent code improvement, incremental program updates, direct interpretation, and other operations based strongly on the structure of the source – Because the permissible structure of a tree can be described formally by a set of productions (cf., Section 4.6), manipulations of tree-based forms can be written as attribute grammars – Stack-based languages are another common type of high level IF

Intermediate Forms • The most common medium level IFs consist of three-address instructions for a simple idealized machine, typically one with an unlimited number of registers – Since the typical instruction specifies two operands, an operator, and a destination, three-address instructions are called quadruples – In older compilers, one may sometimes find an intermediate form consisting of triples or indirect triples in which destinations are specified implicitly • the index of a triple in the instruction stream is the name of the result • an operand is generally named by specifying the index of the triple that produced it.

Intermediate Forms • Different compilers use different IFs – Many compilers use more than one IF internally, though in the common two-pass organization one of these is distinguished as “the” intermediate form • connection between the front end and the back end.

– the syntax trees passed from semantic analysis to intermediate code generation constitute a high level IF – control flow graphs containing pseudo-assembly language (passed in and out of machine-independent code improvement) are a medium level IF – the assembly language of the target machine serves as a low level IF

• Compilers that have back ends for different target architectures do as much work as possible on a high or medium level IF – the machine-independent parts of the code improver can be shared by different back ends

Intermediate Forms

Code Generation • The back end of Figure 15.1 is too complex to present in any detail in a single chapter – To limit the scope of our discussion, we will content ourselves in this chapter with producing correct but naive code – This choice will allow us to consider a significantly simpler back end.

– Starting with Figure 15.1, we drop the machine-independent code improver and then merge intermediate and target code generation into a single phase • generates linear assembly language - no code improvements for control flow, therefore, there is no need to represent that flow explicitly in a control flow graph

Code Generation • We also adopt a much simpler register allocation algorithm – operates directly on the syntax tree prior to code generation - eliminates need for virtual registers and the subsequent mapping onto architectural registers

• Finally, we drop instruction scheduling. The resulting compiler structure appears in Figure 15.5.

– Its code generation phase closely resembles the intermediate code generation of Figure 15.1.

• An Attribute Grammar for GCD Example is presented in Section 15.3.1

Code Generation • Register Allocation – Evaluation of the rules of the attribute grammar itself consists of two main tasks – In each subtree we first determine the registers that will be used to hold various quantities at run time; then we generate code.

– Our naive register allocation strategy uses the next_free_reg inherited attribute to manage registers r1 ... rk as an expression evaluation stack

• To calculate the value of (a + b) × (c - (d / e)) for example, we would generate the following:

Code Generation

Code Generation •

•

•

•

•

In a particularly complicated fragment of code it is possible to run out of architectural registers.

– In this case we must spill one or more registers to memory Our naive register allocator pushes a register onto the program’s subroutine call stack – In effect, architectural registers hold the top k elements of an expression evaluation stack of effectively unlimited size It should be emphasized that our register allocation algorithm, makes very poor use of machine resources If we were generating medium level intermediate code, we would employ virtual registers, rather than architectural ones – Mapping of virtual registers to architectural registers would occur much later in the compilation process.

Target code for the GCD program appears in Figure 14.7.

Address Space Organization • Assemblers, linkers, and loaders typically operate on a pair of related file formats – relocatable object code – executable object code

• Relocatable object code is acceptable as input to a linker – multiple files in this format can be combined to create an executable program

• Executable object code is acceptable as input to a loader: – it can be brought into memory and run

Address Space Organization • A relocatable object file includes the following descriptive information: – import table: Identifies instructions that refer to named locations whose addresses are unknown, but are presumed to lie in other files yet to be linked to this one – relocation table: Identifies instructions that refer to locations within the current file, but that must be modified at link time to reflect the offset of the current file within the final, executable program – export table: Lists the names and addresses of locations in the current file that may be referred to in other files

• Imported and exported names are known as external symbols

Address Space Organization • Running program segments – code – constants – initialized data – uninitialized data: may be allocated at load time or on demand in response to page faults • Usually zero filled, both to provide repeatable symptoms for programs that erroneously read data they have not yet written

– stack: may be allocated in some fixed amount at load time • more commonly, is given a small initial size, and then • extended automatically by the operating system in response to (faulting) accesses beyond the current segment end.

Address Space Organization • Running program segments (2): – heap: may also be allocated in some fixed amount at load time.

• more commonly, is given a small initial size, and is then • extended in response to explicit requests from heap-management library routines – files: In many systems, library routines allow a program to map a file into memory • The map routine interacts with the operating system to create a new segment for the file, and returns the address of the beginning of the segment • the contents of the segment are usually fetched from disk on demand, in response to page faults – dynamic libraries: Modern operating systems typically arrange for most programs to share a single copy of the code for popular libraries

Assembly • Some compilers translate source files directly into object files acceptable to the linker • More commonly, they generate assembly language that must subsequently be processed by an assembler to create an object file – symbolic (textual) notation for code.

– within a compiler it would still be symbolic, most likely consisting of records and linked lists

• To translate this symbolic representation into executable code, we must – replace opcodes and operands with their machine language encodings – replace uses of symbolic names with actual addresses

Assembly • When passing assembly language from the compiler to the assembler, it makes sense to use some internal (records and linked lists) representation • At the same time, we must provide a textual front end to accommodate the occasional need for human input:

Assembly • An alternative organization has the compiler generate object code directly – This organization gives the compiler a bit more flexibility: operations normally performed by an assembler (e.g., assignment of addresses to variables) can be performed earlier if desired.

– Because there is no separate assembly pass, the overall translation to object code may be slightly faster

Assembly • Emitting Instructions – The most basic task of the assembler is to translate symbolic representations of instructions into binary form – In some assemblers this is easy • there is a one-one correspondence between mnemonic operations and instruction op-codes

– Many assemblers extend the instruction set in minor ways to make the assembly language easier for human beings to read – Most MIPS assemblers, for example, provide a large number of pseudoinstructions that translate into different real instructions depending on their arguments, or that correspond to multi-instruction sequences

Assembly • Assemblers respond to a variety of directives (MIPS): – segment switching • .text directive indicates that subsequent instructions and data should be placed in the code (text) segment.

• .data directive indicates that subsequent instructions and data should be placed in the initialized data segment.

• .space n directive indicates that n bytes of space should be reserved in the uninitialized data segment • .byte, .half, .word, .float, and .double directives each take a sequence of arguments • related .ascii directive takes a single character string as argument, which it places in consecutive bytes – symbol identification • .global name directive indicates that name should be entered into the table of exported symbols.

– alignment • .align n directive causes the subsequent output to be aligned at an address evenly divisible by 2n

Assembly • RISC assemblers implement a virtual machine instruction set is “nicer” than that of the real hardware – In addition to pseudoinstructions, the virtual machine may have non-delayed branches – If desired, the compiler or assembly language programmer can ignore the existence of branch delays – The assembler will move nearby instructions to fill delay slots if possible, or generate nops if necessary.

– To support systems programmers, the assembler must also make it possible to specify that delay slots have already been filled

Assembly • Assemblers commonly work in several phases – if the input is textual, an initial phase scans and parses the input, and builds an internal representation – there are two additional phases.

• first phase identifies all internal and external (imported) symbols, assigning locations to the internal ones – complicated by the length of some instructions (on a CISC machine) or – complicated by number of real instructions produced by a pseudoinstruction (on a RISC machine)

• final phase produces object code

Assembly • CISC assemblers distinguish between absolute and relocatable words in an object file • Absolute words are known at assembly time; they need not be changed by the linker – constants and register-register instructions • A relocatable word must be modified by adding to it the address within the final program of the code or data segment of the current object file – A CISC jump instruction might consist of a one-byte jmp opcode followed by a four-byte target address – For a local target, the address bytes in the object file would contain the symbol’s offset within the file – The linker finalizes the address by adding the offset of the file’s code segment within the final program

Linking • Most language implementations - certainly all that are intended for the construction of large programs - support separate compilation – fragments of the program can be compiled and assembled more-orless independently

• After compilation, these fragments (known as compilation units) are “glued together” by a linker – programmer explicitly divides the program into modules or files separately compiled – integrated environments may abandon the notion of a file in favor of a database of subroutines separately compiled

• Linker joins together compilation units

Linking • A static linker does its work prior to program execution, producing an executable object file • A dynamic linker does its work after the program has been brought into memory for execution • Each of the compilation units of a program to be linked must be a relocatable object file – some files will have been produced by compiling fragments of the application being constructed – others will be general purpose library packages needed by the application

• Since most programs make use of libraries, even a “onefile” application typically needs to be linked

Linking • Linking involves two subtasks: relocation and the resolution of external references • Some authors refer to relocation as loading, and call the entire “joining together” process “link-loading.”

• In this book we use “loading” to refer to the process of bringing an executable object file into memory for execution – on very simple machines loading entails relocation – the operating system uses virtual memory to giving the impression that it starts at some standard address (zero) – often loading also entails a certain amount of linking

Linking

Dynamic Linking • On a multi-user system, it is common for several instances of a program (an editor or web browser, for example) to be executing simultaneously – It would be highly wasteful to allocate space in memory for a separate, identical copy of the code of such a program for every running instance

• Many operating systems therefore keep track of the programs that are running, and set up memory mapping tables so that all instances of the same program share the same read-only copy of the program’s code segment – Each instance receives its own writable copy of the data segment – Code segment sharing can save enormous amounts of space – It does not work, however, for instances of programs that are similar but not identical

Chapter 15:: Run-time Program Management

Programming Language Pragmatics, Fourth Edition Michael L. Scott

Copyright © 2016 Elsevier

Introduction

Run-time system: Refers to the set of libraries on which the language implementation depends for correct operation

• Some parts of the run-time, obtain all the information they need from subroutine arguments

• Others require more extensive knowledge of the compiler or the generated program.

Introduction

Run-time system • Simple cases This knowledge is a set of conventions that the compiler and runtime both respect.

• More complex cases The compiler generates program-specific metadata that the runtime must inspect to do its job.

Introduction

Many examples of compiler/runtime integration have been discussed: • Garbage Collection (Section 8.5.3) • Variable Numbers of Arguments (Section 9.3.3) • Exception Handling (Section 9.4) • Event Handling (Section 9.6) • Coroutine & Thread Implementation (9.5 & 13.2.4) • Remote Procedure Call (Section 13.5.4) • Transactional Memory (Section 13.4.4) • Dynamic Linking (Section 15.7)

Introduction

The length and complexity of the list above generally means that the compiler and the run-time system must be developed together.

• Some languages have very small run-time systems: most user level code required to execute a given source program is either generated directly by the compiler or contained in languageindependent libraries.

• Other languages have extensive run-time systems. e.g. C# is heavily dependent on a run-time system defined by the Common Language Infrastructure standard which depends on data generated by the compiler

Virtual Machines

A virtual machine (VM) provides a complete programming environment • Its application programming interface (API) provides all requirements for execution of programs that run above it • Is term used for environments whose level of abstraction is comparable to that of a computer implemented in hardware

Virtual Machines

Virtual machines tend to be characterized as either: • System VM Faithfully emulates all the hardware facilities needed to run a standard OS, including both privileged and unprivileged instructions, memory-mapped I/O, virtual memory, and interrupt facilities. (sometimes referred to as virtual machine monitors (VMMs))

• Process VM Provides the environment needed by a single user-level process: the unprivileged subset of the instruction set and a library-level interface to I/O and other services.

Virtual Machines

The Java Virtual Machine (JVM) • Started as a development of the language Java in 1990–91 at Sun Microsystems 1st public release of Java occurred in 1995 • Code in the JVM was entirely interpreted • JIT compiler added in 1998, with release of Java 2

Virtual Machines

JVM Architecture Summary • Interface provided by JVM designed as target for a Java compiler • Provides direct support for all the built-in and reference types defined by the Java language

• Enforces both definite assignment (Section 6.1.3) & type safety.

• Includes built-in support for many of Java’s language features & standard library packages, including exceptions, threads, garbage collection, reflection, dynamic loading, & security • Usually Java byte code (JBC) is produced from Java source- however, compilers targeting the JVM exist for many languages, including Ruby, JavaScript, Python, Scheme, C, Ada, Cobol, & others

Virtual Machines JVM Storage Management Storage allocation mechanisms in the JVM mirror those of the Java language: • Global constant pool

• Set of registers • Stack for each thread • Method area to hold executable byte code • Heap for dynamically allocated objects

Virtual Machines

A trivial “Hello, world” program (Example 16.2):

Virtual Machines

Virtual Machines

JVM Class Files Class file is stored as a stream of bytes Typically, real file provided by the operating system, could just as easily be a record in a database Multiple class files may be combined into a Java archive (.jar ) file

Virtual Machines

JVM Byte Code (JBC) Stack oriented, operands and results of arithmetic and logic instructions are kept in the operand stack of the current method frame, rather than in registers Instruction set, version 2 categories: - load/store - arithmetic - type conversion - object management - operand stack management - control transfer - method calls - exceptions & monitors

Virtual Machines

JVM byte code for a list insert (Example 16.3):

Virtual Machines Figure 16.2 Java source and byte code for a list insertion method (Example 16.3):

Virtual Machines

Common Language Infrastructure (CLI) • Began at Microsoft Corporation in the late 1990s • Need for interoperability among programming languages running on Windows platforms • For .NET, a specification for CLI virtual machine was standardized by ECMA in 2001

Virtual Machines

Architecture and Comparison to the JVM Similarities between CLI and JVM: - Both systems define a multithreaded, stack-based virtual machine, with built-in support for garbage collection, exceptions, virtual method dispatch, and mix-in inheritance.

- Both represent programs using a platform-independent, selfdescriptive, byte code notation.

- For languages like C#, the CLI provides all the safety of the JVM, including definite assignment, strong typing, and protection against overflow or underflow of the operand stack.

Virtual Machines

Architecture and Comparison to the JVM Contrasts between CLI and JVM:

- Richer type system for CLI - Richer calling mechanisms in CLI - Unsafe code made explicit in CLI - Miscellaneous CLI support

Virtual Machines

The Common Type System Built-in Types

- Integers in 8, 16, 32, and 64 bit lengths, both signed and unsigned

- “Native” integers supported by hardware, again both signed and unsigned - IEEE floating-point both single and double precision Object references and “managed” pointers

Virtual Machines

The Common Type System Constructed Types - Dynamically allocated instances of class, interface, array, and delegate types - Methods – function types - Properties – getters and setters for objects - Events – lists of delegates, associated with an object - Value types – records (structures), unions, & enumerations - Boxed value types – values embedded in dynamically allocated object - Function pointers – references to static functions - Typed references – pointers bundled, with type descriptor - Unmanaged pointers – as in C, can point to just about anything, & support pointer arithmetic

Virtual Machines

The Common Language Specification • Defines subset of CTS that most languages can accommodate

• Omits several of the types provided by the CTS • Imposes restrictions on the use of other types; establishes naming conventions, limits the use of overloading, and defines the operators and conversions that programs can assume are supported on built-in types

• None of these restrictions applies to program components that operate only within a given language

Virtual Machines

Metadata and Assemblies

• Portable Executable (PE) assemblies are the rough equivalent of Java .jar files

• Contain the code for a collection of CLI classes based on the Common Object File Format (COFF), originally developed for AT&T’s System V Unix.

The Common Intermediate Language (CIL) • Version 4 of ECMA standard defines approximately 250 instructions • CIL bears a strong resemblance to JBC • Any differences stem from the assumption that CIL will always be JITcompiled

Late Binding of Machine Code

Just-in-Time (JIT) and Dynamic Compilation • JIT system compiles programs immediately prior to execution, can add significant delay to program start-up time • Cost of JIT compilation is typically lessened by the existence of an earlier source-to-byte-code compilere.g. Java byte code (JBC)

Late Binding of Machine Code

Dynamic Compilation • In some cases JIT compilation must be delayed, either because: - Source or byte code was not created or discovered until run time - Perform optimizations that depend on information gathered during execution

• Common Lisp, the language is typically compiled, but a program can extend itself at run time

Late Binding of Machine Code

Binary Translation • Recompilation of object code • Allows already-compiled programs to be run on a machine with a different instruction set architecture e.g. Apple’s Rosetta system, which allows programs compiled for older PowerPC-based Macintosh computers to run on newer x86-based Macs

Late Binding of Machine Code

Binary Translation • Principal challenge is loss of information in the original source-to-object code translation • Object code typically lacks both type information and clearlydelineated subroutines and control-flow constructs of source code and byte code • Yet most of this information appears in compiler’s symbol table

Late Binding of Machine Code

Binary Translation • Typical binary translator reads an object file & reconstructs control flow graph (cf. Section 15.1.1) - Task complicated by lack of explicit information about basic blocks.

- While branches (the ends of basic blocks) are easy to identify, beginnings are more difficult: since branch targets are sometimes computed at run time or looked up in dispatch tables or virtual function tables

Late Binding of Machine Code

Binary Rewriting Technique to modify existing executable code can be used for: - Profiling (insert instrumentation of some kind) - Simulate new architectures - Evaluate the coverage of test suites- implement model checking for parallel programs, a process that exposes race conditions - “Audit” the quality of a compiler’s optimizationsinsert dynamic semantic checks into a program - “Sandbox” untrusted code, permits an safe execution in same address space as application.

Late Binding of Machine Code

Mobile Code and Sandboxing • Portability is one of the principal motivations for late binding of machine code.

• Code compiled for one machine architecture or operating system cannot generally be run on another due to dependencies.

• For code to be mobile, it must be executed in some sort of sandbox.

• Sandbox mechanisms lie at the boundary between language implementation and operating systems.

Inspection/Introspection

Reflection • Load already-compiled program, to use reflection tools for querying symbol table information created by compiler • E.g. reflection useful when printing diagnostics

Inspection/Introspection

Reflection • C#’s reflection API is similar to that of Java: System.Type is analogous to java.lang.Class - System. Reflection is analogous to java.lang.reflect -

• All of the major scripting languages (Perl, PHP, Tcl, Python, Ruby, JavaScript) provide extensive reflection mechanisms

• Principal difference between reflection in Java or C# and in scripting languages (Lisp) is the later is dynamically typed

Inspection/Introspection

Symbolic Debugging • Built into most programming language interpreters, virtual machines, and integrated program development environments

• Also available as stand-alone tools e.g. GNU's gdb • Adjective symbolic refers to a debugger’s understanding of high level language syntax

Inspection/Introspection

Symbolic Debugging Debugger allows the user to perform two main kinds of operations: - Breakpoint: specifies that execution should stop if it reaches a particular location in the source code.

- Watchpoint: specifies that execution should stop if a particular variable is read or written.

Inspection/Introspection

Symbolic Debugging • Both data and control operations also depend on the ability to manipulate a program from outside: - To stop and start it, and to read and write its data.

• Debugger control can be implemented: - Interpreters - Dynamic binary rewriting- support from operating system(compiled programs only)

• Some processors provide hardware support

Chapter 17:: Code Improvement

Programming Language Pragmatics, Fourth Edition Michael L. Scott

Copyright © 2016 Elsevier

Introduction • We discussed the generation, assembly, and linking of target code in the back end of a compiler – We get correct but highly suboptimal code • redundant computations • inefficient use of the registers, multiple functional units, and cache

• This chapter takes a look at code improvement: the phases of compilation devoted to generating good code – we interpret “good” to mean fast – occasionally we also consider program transformations to decrease memory requirements

Introduction • In a very simple compiler, we can use a peephole optimizer to peruse already-generated target code for obviously suboptimal sequences of adjacent instructions • At a slightly higher level, we can generate near-optimal code for basic blocks – a basic block is a maximal-length sequence of instructions that will always execute in its entirety (assuming it executes at all)

– in the absence of hardware exceptions, control never enters a basic block except at the beginning, and never exits except at the end

Introduction • Code improvement at the level of basic blocks is known as local optimization – elimination of redundant operations (unnecessary loads, common subexpression calculations) – effective instruction scheduling and register allocation

• At higher levels of aggressiveness, compilers employ techniques that analyze entire subroutines for further speed improvements • These techniques are known as global optimization – multi-basic-block versions of redundancy elimination – instruction scheduling, and register allocation – code modifications designed to improve the performance of loops

Introduction • Both global redundancy elimination and loop improvement typically employ a control flow graph representation of the program – Use a family of algorithms known as data flow analysis (flow of information between basic blocks)

• Recent compilers perform various forms of interprocedural code improvement • Interprocedural improvement is difficult – subroutines may be called from many different places • hard to identify available registers, common subexpressions, etc.

– subroutines are separately compiled

Phases of Code Improvement • We will concentrate in our discussion on the forms of code improvement that tend to achieve the largest increases in execution speed, and are most widely used – Compiler phases to implement these improvements is shown in Figure 17.1

Phases of Code Improvement

Phases of Code Improvement • The machine-independent part of the back end begins with intermediate code generation – identifies fragments of the syntax tree that correspond to basic blocks – creates a control flow graph in which each node contains a sequence of three-address instructions for an idealized machine (unlimited supply of virtual registers)

• The machine-specific part of the back end begins with target code generation – strings the basic blocks together into a linear program • translates each block into the instruction set of the target machine and generating branch instructions that correspond to the arcs of the control flow graph

Phases of Code Improvement • The machine-independent part of the back end begins with intermediate code generation – identifies fragments of the syntax tree that correspond to basic blocks – creates a control flow graph in which each node contains a sequence of three-address instructions for an idealized machine (unlimited supply of virtual registers)

• The machine-specific part of the back end begins with target code generation – strings the basic blocks together into a linear program • translates each block into the instruction set of the target machine and generating branch instructions that correspond to the arcs of the control flow graph

Phases of Code Improvement • Machine-independent code improvement has three separate phases 1. Local redundancy elimination: identifies and eliminates redundant loads, stores, and computations within each basic block 2. Global redundancy elimination: identifies similar redundancies across the boundaries between basic blocks (but within the bounds of a single subroutine) 3. Loop improvement: effects several improvements specific to loops • •

these are particularly important, since most programs spend most of their time in loops.

Global redundancy elimination and loop improvement may actually be subdivided into several separate phases

Phases of Code Improvement • Machine-specific code improvement has four separate phases – Preliminary and final instruction scheduling are essentially identical (Phases 1 & 3) – Register allocation (Phase 2) and instruction scheduling tend to interfere with one another • •

the instruction schedules minimize pipeline stalls which tend to increase the demand for architectural registers (register pressure) we schedule instructions first, then allocate architectural registers, then schedule instructions again – If it turns out that there aren’t enough architectural registers, the register allocator will generate additional load and store instructions to spill registers temporarily to memory – the second round of instruction scheduling attempts to fill any delays induced by the extra loads

Peephole Optimization • A relatively simple way to significantly improve the quality of naive code is to run a peephole optimizer over the target code – works by sliding a several instruction window (a peephole) over the target code, looking for suboptimal patterns of instructions – the patterns to look for are heuristic • patterns to match common suboptimal idioms produced by a particular front end • patterns to exploit special instructions available on a given machine

• A few examples are presented in what follows

Peephole Optimization • Elimination of redundant loads and stores – The peephole optimizer can often recognize that the value produced by a load instruction is already available in a register r2 := r1 + 5 i := r2 r3 := i r3 := r3 × 3

becomes r2 := r1 + 5 i := r2 r3 := r2 × 3

Peephole Optimization

• Constant folding • A naive code generator may produce code that performs calculations at run time that could actually be performed at compile time – A peephole optimizer can often recognize such code

r2 := 3 × 2 becomes r2 := 6

Peephole Optimization • Constant propagation – Sometimes we can tell that a variable will have a constant value at a particular point in a program – We can then replace occurrences of the variable with occurrences of the constant r2 := 4 r3 := r1 + r2 r2 := . . . becomes r2 := 4 r3 := r1 + 4 r2 := . . . and then r3 := r1 + 4 r2 := . . .

Peephole Optimization • Common subexpression elimination – When the same calculation occurs twice within the peephole of the optimizer, we can often eliminate the second calculation: r2 := r1 × 5 r2 := r2 + r3 r3 := r1 × 5 becomes r4 := r1 × 5 r2 := r4 + r3 r3 := r4 – Often, as shown here, an extra register will be needed to hold the common value

Peephole Optimization • It is natural to think of common subexpressions as something that could be eliminated at the source code level, and programmers are sometimes tempted to do so • The following, for example, x = a + b + c; y = a + b + d; could be replaced with t = a + b; x = t + c; y = t + d;

Peephole Optimization • Copy propagation – Even when we cannot tell that the contents of register b will be constant, we may sometimes be able to tell that register b will contain the same value as register • replace uses of b with uses of a, so long as neither a nor b is modified

r2 := r1 r3 := r1 + r2 r2 := 5 becomes r2 := r1 r3 := r1 + r1 r2 := 5 and then r3 := r1 + r1 r2 := 5

Peephole Optimization • Strength reduction – Numeric identities can sometimes be used to replace a comparatively expensive instruction with a cheaper one • In particular, multiplication or division by powers of two can be replaced with adds or shifts:

r1 := r2 × 2 becomes r1 := r2 + r2 or r1 := r2 << 1 r1 := r2 / 2 becomes r1 := r2 >> 1

Peephole Optimization • Elimination of useless instructions – Instructions like the following can be dropped entirely:

r1 := r1 + 0 r1 := r1 × 1 • Filling of load and branch delays – Several examples of delay-filling transformations were presented in Chapter 5

• Exploitation of the instruction set – Particularly on CISC machines, sequences of simple instructions can often be replaced by a smaller number of more complex instructions

Redundancy Elimination in Basic Blocks • Throughout the remainder of this chapter we will trace the improvement of code for a specific subroutine: calculates into an array the binomial coefficients

Redundancy Elimination in Basic Blocks

• In C, the code looks like this:

Redundancy Elimination in Basic Blocks • We employ a medium level intermediate form (IF) for control flow – Every calculated value is placed in a separate register – To emphasize virtual registers (of which there is an unlimited supply), we name them v1, v2, . . .

– We use r1, r2, . . . to represent architectural registers in Section 17.8.

Redundancy Elimination in Basic Blocks • To improve the code within basic blocks, we need to – minimize loads and stores – identify redundant calculations

• There are two techniques usually employed 1. translate the syntax tree for a basic block into an expression DAG (directed acyclic graph) in which redundant loads and computations are merged into individual nodes with multiple parents 2. similar functionality can also be obtained without an explicitly graphical program representation, through a technique known as local value numbering

• We describe the last technique below

Redundancy Elimination in Basic Blocks • Value numbering assigns the same name (a “number”) to any two or more symbolically equivalent computations (“values”), so that redundant instances will be recognizable by their common name • Our names are virtual registers, which we merge whenever they are guaranteed to hold a common value • While performing local value numbering, we will also implement – – – – –

local constant folding constant propagation, copy propagation common subexpression elimination strength reduction useless instruction elimination

Redundancy Elimination in Basic Blocks

Global Redundancy and Data Flow Analysis

• We now concentrate on the elimination of redundant loads and computations across the boundaries between basic blocks • We translate the code of our basic blocks into static single assignment (SSA) form, which will allow us to perform global value numbering • Once value numbers have been assigned, we shall be able to perform – global common subexpression elimination – constant propagation – copy propagation

Global Redundancy and Data Flow Analysis

• In a compiler both the translation to SSA form and the various global optimizations would be driven by data flow analysis.

– We detail the problems of identifying • •

common subexpressions useless store instructions

– We will also give data flow equations for the calculation of reaching definitions, used to move invariant computations out of loops

• Global redundancy elimination can be structured in such a way that it catches local redundancies as well, eliminating the need for a separate local pass

Global Redundancy and Data Flow Analysis

• Value numbering, as introduced earlier, assigns a distinct virtual register name to every symbolically distinct value that is loaded or computed in a given body of code – It allows us to recognize when certain loads or computations are redundant.

• The first step in global value numbering is to distinguish among the values that may be written to a variable in different basic blocks – We accomplish this step using static single assignment (SSA) form

Global Redundancy and Data Flow Analysis

• For example, if the instruction v2 := x is guaranteed to read the value of x written by the instruction x3 := v1, then we replace v2 := x with v2 := x3 • If we cannot tell which version of x will be read, we use a hypothetical function φ to choose among the possible alternatives – we won’t actually have to compute φ-functions at run time •

the only purpose is to help us identify possible code improvements

– we will drop them (and the subscripts) prior to target code generation

Global Redundancy and Data Flow Analysis

Global Redundancy and Data Flow Analysis • With flow-dependent values determined by φfunctions, we are now in a position to perform global value numbering – As in local value numbering, the goal is to merge any virtual registers that are guaranteed to hold symbolically equivalent expressions – In the local case, we were able to perform a linear pass over the code – We kept a dictionary that mapped loaded and computed expressions to the names of virtual registers that contained them

Global Redundancy and Data Flow Analysis

• This approach does not suffice in the global case, because the code may have cycles – The general solution can be formulated using data flow – It can also be obtained with a simpler algorithm that begins by unifying all expressions with the same top-level operator • •

In the end, repeatedly separates expressions whose operands are distinct It is quite similar to the DFA minimization algorithm of Chapter 2

• We perform this analysis for our running example informally

Global Redundancy and Data Flow Analysis

Global Redundancy and Data Flow Analysis

• Many instances of data flow analysis can be cast in the following framework: 1. four sets for each basic block B, called InB, OutB, GenB, and KillB; 2. values for the Gen and Kill sets; 3. an equation relating the sets for any given block B; 4. an equation relating the Out set of a given block to the In sets of its successors, or relating the In set of the block to the Out sets of its predecessors; and (often) 5. certain initial conditions

Global Redundancy and Data Flow Analysis

• The goal of the analysis is to find a fixed point of the equations: a consistent set of In and Out sets (usually the smallest or the largest) that satisfy both the equations and the initial conditions – Some problems have a single fixed point – Others may have more than one •

we usually want either the least or the greatest fixed point (smallest or largest sets)

Global Redundancy and Data Flow Analysis

• In the case of global common subexpression elimination, InB is the set of expressions (virtual registers) guaranteed to be available at the beginning of block B – These available expressions will all have been set by predecessor blocks – OutB is the set of expressions guaranteed to be available at the end of B – KillB is the set of expressions killed in B: invalidated by assignment to one of the variables used to calculate the expression, and not subsequently recalculated in B – GenB is the set of expressions calculated in B and not subsequently killed in B

Global Redundancy and Data Flow Analysis

• The data flow equations for available expression analysis are:

• Our initial condition is In1 = : no expressions are available at the beginning of execution

Global Redundancy and Data Flow Analysis

• Available expression analysis is known as a forward data flow problem, because information flows forward across branches: the In set of a block depends on the Out sets of its predecessors – We will see an example of a backward data flow problem later

• We calculate the desired fixed point of our equations in an inductive (iterative) fashion, much as we computed first and follow sets in Chapter 2 • Our equation for InB uses intersection to insist that an expression be available on all paths into B – In our iterative algorithm, this means that InB can only shrink with subsequent iterations

Global Redundancy and Data Flow Analysis

Global Redundancy and Data Flow Analysis

• We turn our attention to live variable analysis very important in any subroutine in which global common subexpression analysis has eliminated load instructions • Live variable analysis is a backward flow problem • It determines which instructions produce values that will be needed in the future, allowing us to eliminate dead (useless) instructions – in our example we consider only values written to memory and with the elimination of dead stores – applied to values in virtual registers as well, live variable analysis can help to identify other dead instructions

Global Redundancy and Data Flow Analysis

• For this instance of data flow analysis – InB is the set of variables live at the beginning of block B – OutB is the set of variables live at the end of the block – GenB is the set of variables read in B without first being written in B – KillB is the set of variables written in B without having been read first

• The data flow equations are:

Global Redundancy and Data Flow Analysis

Loop Improvement I • We consider two classes of loop improvements: – those that move invariant computations out of the body of a loop and into its header, and – those that reduce the amount of time spent maintaining induction variables

• Later we consider transformations that improve instruction scheduling by restructuring a loop body to include portions of more than one iteration of the original loop – manipulate multiply nested loops to improve cache performance or increase opportunities for parallelization

Loop Improvement I • A loop invariant is an instruction (i.e., a load or calculation) in a loop whose result is guaranteed to be the same in every iteration – If a loop is executed n times and we are able to move an invariant instruction out of the body and into the header (saving its result in a register for use within the body), then we will eliminate n -1 calculations from the program •

a potentially significant savings

• In order to tell whether an instruction is invariant, we need to identify the bodies of loops, and we need to track the locations at which operand values are defined

Loop Improvement I •

Tracking the locations at which an operand may have been defined amounts to the problem of reaching definitions –

•

Formally, we say an instruction that assigns a value v into a location (variable or register) l reaches a point p in the code if v may still be in l at p

Like the conversion to static single assignment form, considered informally earlier, the problem of reaching definitions can be structured as a set of forward, any-path data flow equations –

–

We let GenB be the set of final assignments in block B (those that are not overwritten later in B) For each assignment in B we also place in KillB all other assignments (in any block) to the same location

Loop Improvement I • Then we have

• Given InB (the set of reaching definitions at the beginning of the block), we can determine the reaching definitions of all values used within B by a simple linear perusal of the code

Loop Improvement I

Loop Improvement I • An induction variable (or register) is one that takes on a simple progression of values in successive iterations of a loop.

– We confine our attention to arithmetic progressions – Induction variables appear as loop indices, subscript computations, or variables incremented or decremented explicitly within the body of the loop

• Induction variables are important for two reasons: – They commonly provide opportunities for strength reduction, replacing multiplication with addition – They are commonly redundant: instead of keeping several induction variables in registers, we can often keep a smaller number and calculate the remainder from those when needed

Loop Improvement I

Loop Improvement I

Instruction Scheduling •

We will perform two rounds of instruction scheduling separated by register allocation – –

Given our use of pseudoassembler, we won’t consider peephole optimization in any further detail On a pipelined machine, performance depends critically on the extent to which the compiler is able to keep the pipeline full •

–

A compiler can solve the subproblem of filling branch delays in a more or less straightforward fashion

If we examine the body of the loop in our combinations example, optimizations described thus far have transformed Block 2 from the 30 instruction sequence into the eight instruction sequence of Figure 17.11 •

Unfortunately, on a pipelined machine without instruction reordering, this code is still distinctly suboptimal

Instruction Scheduling • To schedule instructions to make better use of the pipeline, we first arrange them into a directed acyclic graph (DAG), in which each node represents an instruction, and each arc represents a dependence – Most arcs will represent flow dependences, in which one instruction uses a value produced by a previous instruction – A few will represent anti-dependences, in which a later instruction overwrites a value read by a previous instruction – In our example, these will correspond to updates of induction variables

Instruction Scheduling

Loop Improvement II Loop Unrolling and Software Pipelining • Loop unrolling is a transformation that embeds two or more iterations of a source-level loop in a single iteration of a new, longer loop, and allowing the scheduler to intermingle the instructions of the original iterations • If we unroll two iterations of our combinations example we obtain the code of Figure 17.14 – We have used separate names (here starting with the letter ‘t’) for registers written in the initial half of the loop – This convention minimizes anti- and output dependences, giving us more latitude in scheduling

Loop Improvement II

Loop Improvement II

Loop Improvement II • A software-pipelined version of our combinations subroutine appears in the bottom half of Figure 17.15 and as a control flow graph in Figure 17.16 – The idea is to build a loop whose body comprises portions of several consecutive iterations of the original loop, with no internal start-up or shut-down cost – In our example, each iteration of the software-pipelined loop contributes to three separate iterations of the original loop – Within each new iteration (shown between vertical bars) nothing needs to wait for the divide to complete – To avoid delays, we have altered the code in several ways

Loop Improvement II

Loop Improvement II Loop Reordering • The code improvement techniques that we have considered thus far have served two principal purposes – eliminate redundant or unnecessary instructions – minimize stalls on a pipelined machine.

• Two other goals have become increasingly important in recent years – it has become increasingly important to minimize cache misses (processor speed outstrips memory latency) – it has become important to identify sections of code that can execute concurrently (parallel machines)

• As with other optimizations, the largest benefits come from changing the behavior of loops

Loop Improvement II Loop Reordering • A loop-reordering compiler can improve this code by interchanging the nested loops: for j := 1 to n for i := 1 to n A[i, j] := 0

Figure 17.17 Tiling (blocking) of a matrix operation

Loop Improvement II Loop Reordering

for i := 1 to n for j := 1 to n A[i, j] := 0 • If A is laid out in row-major order, and if each cache line contains m elements of A, then this code will suffer n2/m cache misses • If A is laid out in column major order, and if the cache is too small to hold n lines of A, then the code will suffer n2 misses, fetching the entire array from memory m times

Loop Improvement II Loop Dependences • When reordering loops, we must to respect all data dependences (loop-carried dependences) i := 2 to n for j := 1 to n1 A[i, j] := A[i, j] A[i1, j+1] • Here the calculation of A[i, j] in iteration (i, j) depends on the value of A[i-1, j+1], which was calculated in iteration (i-1, j+1).

• This dependence is often represented by a diagram of the iteration space (see next slide):

Loop Improvement II • The i and j dimensions in this diagram represent loop indices, not array subscripts.

– The arcs represent the loop-carried flow dependence

• If we wish to interchange the i and j loops of this code (e.g., to improve cache locality), we find that we cannot do it, because of the dependence: we would end up trying to write A[i, j] before we had written A[i-1, j+1]

Loop Improvement II • By analyzing the loop dependence, we note that we can reverse the order of the j loop without violating the dependence: for i := 2 to n for j := n1 to 1 by 1 A[i, j] := A[i, j] A[i1, j+1]

• This change transforms the iteration space as shown here

Loop Improvement II • Another transformation that sometimes serves to eliminate a dependence is known as loop skewing – it reshapes a rectangular iteration space into a parallelogram, by adding the outer loop index to the inner one, and then subtracting from the appropriate subscripts:

for i := 2 to n for j := i+1 to i+n-1 A[i,j-i]:= A[i,j-i]-A[i1,j+1-i] • A moment’s consideration will reveal that this code accesses the exact same elements as before, in the exact same order

Loop Improvement II • Its iteration space, however, looks like here (the loops can be safely be interchanged)

Loop Improvement II Parallelization • Loop iterations (at least in non-recursive programs) constitute the principal source of operations that can execute in parallel – one needs to find independent loop iterations - with no loop-carried dependences

• Even in languages without special constructs, a compiler can often parallelize code by identifying— or creating—loops with as few loop-carried dependences as possible – These transformations are valuable tools in this endeavor

Loop Improvement II • Consider the problem of “zero-ing out” a twodimensional array (row-major order): for i := 0 to n-1 for j := 0 to n-1 A[i, j] := 0

• On a machine containing several general purpose processors, we parallelize the outer loop: –– on processor pid: for i := (n/p × pid) to (n/p ×(pid + 1)-1) for j := 1 to n A[i, j] := 0

Loop Improvement II • Other issues of importance in parallelizing compilers include communication and load balance • Locality in parallel programs reduces communication among processors and between the processors and memory – Optimizations similar to those employed to reduce the number of cache misses on a uniprocessor can be used to reduce communication trafic on a multiprocessor.

• Load balance refers to the division of labor among processors on a parallel machine – dividing a program among 16 processors, we shall obtain a speedup of close to 16 only if each processor takes the same amount of time to do its work – assigning 5% of the work to each of 15 processors and 25% of the work to the sixteenth, we are likely to see a speedup of no more than four

Register Allocation • In a simple compiler with no global optimizations, register allocation can be performed independently in every basic block – simple compilers usually apply a set of heuristics to identify such frequently accessed variables variables and allocate them to registers over the life of a subroutine.

– candidates for a dedicated register include loop indices, the implicit pointers of with statements in Pascal-family languages, and scalar local variables and parameters

• It has been known since the early 1970s that register allocation is equivalent to the NP-hard problem of graph coloring

Register Allocation • We first identify virtual registers that cannot share an architectural register, because they contain values that are live concurrently – we use reaching definitions data flow analysis – we then determine the live range of variables

• Given these live ranges, we construct a register interference graph – The nodes of this graph represent virtual registers.

Registers vi and vj are connected by an arc if they are simultaneously live – The interference graph corresponding to Figure 15.17 appears in Figure 15.18 (see next slides)

Register Allocation •

For the software-pipelined version of our earlier combinations subroutine (Figure 17.16), we chart the live ranges of the virtual registers as shown in Figure 17.18 – Note that the live range of v19 spans the backward branch at the end of Block 2; though typographically disconnected it is contiguous in time

Register Allocation

Register Allocation • The problem of mapping virtual registers onto the smallest possible number of architectural registers now amounts to finding a minimal coloring of this graph - an assignment of “colors” to nodes such that no arc connects two nodes of the same color • In our example, we can find one of several optimal solutions by inspection – note that the six registers in the center of the figure constitute a clique (a completely connected subgraph); each must be mapped to a separate architectural register – final code for the combinations subroutine appears in Figure 17.20

Register Allocation