Database Systems Design, Implementation, & Management, 12th Edition.pdf

1305627482_520056.pptx Slide 1 Chapter 1 Database Systems

Slide 2 Learning Objectives In this chapter, you will learn: The difference between data and information What a database is, the various types of databases, and why they are valuable assets for decision making The importance of database design How modern databases evolved from file systems 2

Slide 3 Learning Objectives In this chapter, you will learn: About flaws in file system data management The main components of the database system The main functions of a database management system (DBMS) 3

Slide 4 Data versus Information Data Information Raw facts Have not yet been processed to reveal their meaning to the end user Building blocks of information Data management Generation, storage, and retrieval of data Produced by processing raw data to reveal its meaning Requires context Bedrock of knowledge Should be accurate, relevant, and timely to enable good decision making 4

Slide 5 Introducing the Database Shared, integrated computer structure that stores a collection of: End-user data - Raw facts of interest to end user Metadata: Data about data, which the end-user data are integrated and managed Describe data characteristics and relationships Database management system (DBMS) Collection of programs Manages the database structure Controls access to data stored in the database 5

Slide 6 Role of the DBMS

Intermediary between the user and the database Enables data to be shared Presents the end user with an integrated view of the data Receives and translates application requests into operations required to fulfill the requests Hides database’s internal complexity from the application programs and users 6

Slide 7 Figure 1.3 - The DBMS Manages the Interaction between the End User and the Database 7

Slide 8 Advantages of the DBMS Better data integration and less data inconsistency Data inconsistency: Different versions of the same data appear in different places Increased end-user productivity Improved: Data sharing Data security Data access Decision making Data quality: Accuracy, validity, and timeliness of data 8

Slide 9 Types of Databases Single-user database: Supports one user at a time Desktop database: Runs on PC Multiuser database: Supports multiple users at the same time Workgroup databases: Supports a small number of users or a specific department Enterprise database: Supports many users across many departments 9

Slide 10 Types of Databases Centralized database: Data is located at a single site Distributed database: Data is distributed across different sites Cloud database: Created and maintained using cloud data services that provide defined performance measures for the database 10

Slide 11 Types of Databases General-purpose databases: Contains a wide variety of data used in multiple disciplines Discipline-specific databases: Contains data focused on specific subject areas Operational database: Designed to support a company’s day-to-day operations 11

Slide 12 Types of Databases

Analytical database: Stores historical data and business metrics used exclusively for tactical or strategic decision making Data warehouse: Stores data in a format optimized for decision support Online analytical processing (OLAP) Tools for retrieving, processing, and modeling data from the data warehouse Business intelligence: Captures and processes business data to generate information that support decision making 12

Slide 13 Types of Databases Unstructured data: It exists in their original state Structured data: It results from formatting Structure is applied based on type of processing to be performed Semistructured data: Processed to some extent Extensible Markup Language (XML) Represents data elements in textual format 13

Slide 14 Database Design Focuses on the design of the database structure that will be used to store and manage end-user data Well-designed database Facilitates data management Generates accurate and valuable information Poorly designed database causes difficult-to-trace errors 14

Slide 15 Evolution of File System Data Processing 15

Slide 16 Table 1.2 - Basic File Terminology 16

Slide 17 Figure 1.8 - A Simple File System 17

Slide 18 Problems with File System Data Processing 18

Slide 19 Structural and Data Dependence

Structural dependence: Access to a file is dependent on its own structure All file system programs are modified to conform to a new file structure Structural independence: File structure is changed without affecting the application’s ability to access the data 19

Slide 20 Structural and Data Dependence Data dependence Data access changes when data storage characteristics change Data independence Data storage characteristics is changed without affecting the program’s ability to access the data Practical significance of data dependence is difference between logical and physical format 20

Slide 21 Data Redundancy Unnecessarily storing same data at different places Islands of information: Scattered data locations Increases the probability of having different versions of the same data 21

Slide 22 Data Redundancy Implications Poor data security Data inconsistency Increased likelihood of data-entry errors when complex entries are made in different files Data anomaly: Develops when not all of the required changes in the redundant data are made successfully 22

Slide 23 Types of Data Anomaly 23

Slide 24 Database Systems Logically related data stored in a single logical data repository Physically distributed among multiple storage facilities DBMS eliminates most of file system’s problems Current generation DBMS software: Stores data structures, relationships between structures, and access paths Defines, stores, and manages all access paths and components 24

Slide 25 Figure 1.9 - Contrasting Database and File Systems 25

Slide 26 Figure 1.10 - The Database System Environment 26

Slide 27 DBMS Functions 27

Slide 28 DBMS Functions 28

Slide 29 DBMS Functions 29

Slide 30 Disadvantages of Database Systems 30

Slide 31 Table 1.3 - Database Career Opportunities 31 31

1305627482_520057.pptx Slide 1 Chapter 2 Data Models

Slide 2 Learning Objectives In this chapter, you will learn: About data modeling and why data models are important About the basic data-modeling building blocks What business rules are and how they influence database design 2

Slide 3 Learning Objectives In this chapter, you will learn: How the major data models evolved About emerging alternative data models and the need they fulfill How data models can be classified by their level of abstraction 3

Slide 4 Data Modeling and Data Models Data modeling: Iterative and progressive process of creating a specific data model for a determined problem domain Data models: Simple representations of complex real-world data structures Useful for supporting a specific problem domain Model - Abstraction of a real-world object or event 4

Slide 5 Importance of Data Models 5

Slide 6 Data Model Basic Building Blocks Entity: Unique and distinct object used to collect and store data Attribute: Characteristic of an entity Relationship: Describes an association among entities One-to-many (1:M) Many-to-many (M:N or M:M) One-to-one (1:1) Constraint: Set of rules to ensure data integrity 6

Slide 7 Business Rules

7

Slide 8 Sources of Business Rules 8

Slide 9 Reasons for Identifying and Documenting Business Rules Help standardize company’s view of data Communications tool between users and designers Allow designer to: Understand the nature, role, scope of data, and business processes Develop appropriate relationship participation rules and constraints Create an accurate data model 9

Slide 10 Translating Business Rules into Data Model Components Nouns translate into entities Verbs translate into relationships among entities Relationships are bidirectional Questions to identify the relationship type How many instances of B are related to one instance of A? How many instances of A are related to one instance of B? 10

Slide 11 Naming Conventions Entity names - Required to: Be descriptive of the objects in the business environment Use terminology that is familiar to the users Attribute name - Required to be descriptive of the data represented by the attribute Proper naming: Facilitates communication between parties Promotes self-documentation 11

Slide 12 Hierarchical and Network Models Hierarchical Models Network Models Developed to manage large amounts of data for complex manufacturing projects Represented by an upside-down tree which contains segments (equivalent of a file system’s record type) Depicts a set of one-to-many (1:M) relationships Created to represent complex data relationships effectively Improved database performance and imposed a database standard Allows a record to have more than one parent Depicts both one-to-many (1:M) and many-to-many (M:N) relationships 12

Slide 13

Standard Database Concepts 13

Slide 14 Standard Database Concepts 14

Slide 15 The Relational Model Produced an automatic transmission database that replaced standard transmission databases Based on a relation Relation or table: Matrix composed of intersecting tuple and attribute Tuple: Rows Attribute: Columns Describes a precise set of data manipulation constructs 15

Slide 16 Relational Database Management System (RDBMS) Performs basic functions provided by the hierarchical and network DBMS systems Makes the relational data model easier to understand and implement Hides the complexities of the relational model from the user 16

Slide 17 Figure 2.2 - A Relational Diagram 17

Slide 18 SQL-Based Relational Database Application End-user interface Allows end user to interact with the data Collection of tables stored in the database Each table is independent from another Rows in different tables are related based on common values in common attributes SQL engine Executes all queries 18

Slide 19 The Entity Relationship Model Graphical representation of entities and their relationships in a database structure Entity relationship diagram (ERD) Uses graphic representations to model database components Entity instance or entity occurrence Rows in the relational table Connectivity: Term used to label the relationship types 19

Slide 20 Figure 2.3 - The ER Model Notations 20

Slide 21 The Object-Oriented Data Model (OODM) or Semantic Data Model Object-oriented database management system(OODBMS) Based on OODM Object: Contains data and their relationships with operations that are performed on it Basic building block for autonomous structures Abstraction of real-world entity Attributes - Describe the properties of an object 21

Slide 22 The Object-Oriented Data Model (OODM) Class: Collection of similar objects with shared structure and behavior organized in a class hierarchy Class hierarchy: Resembles an upside-down tree in which each class has only one parent Inheritance: Object inherits methods and attributes of parent class Unified Modeling Language (UML) Describes sets of diagrams and symbols to graphically model a system 22

Slide 23 Figure 2.4 - A Comparison of OO, UML and ER Models 23

Slide 24 Object/Relational and XML Extended relational data model (ERDM) Supports OO features and complex data representation Object/Relational Database Management System (O/R DBMS) Based on ERDM, focuses on better data management Extensible Markup Language (XML) Manages unstructured data for efficient and effective exchange of all data types 24

Slide 25 Big Data Aims to: Find new and better ways to manage large amounts of web and sensor-generated data Provide high performance and scalability at a reasonable cost Characteristics Volume Velocity Variety 25

Slide 26 Big Data Challenges

26

Slide 27 Big Data New Technologies 27

Slide 28 NoSQL Databases Not based on the relational model Support distributed database architectures Provide high scalability, high availability, and fault tolerance Support large amounts of sparse data Geared toward performance rather than transaction consistency Store data in key-value stores 28

Slide 29 Figure 2.5 - A Simple Key-Value Representation 29

Slide 30 Figure 2.6 - The Evolution of Data Models 30

Slide 31 Hierarchical Model Advantages Disadvantages Promotes data sharing Parent/child relationship promotes conceptual simplicity and data integrity Database security is provided and enforced by DBMS Efficient with 1:M relationships Requires knowledge of physical data storage characteristics Navigational system requires knowledge of hierarchical path Changes in structure require changes in all application programs Implementation limitations No data definition Lack of standards 31

Slide 32 Network Model Advantages Disadvantages Conceptual simplicity Handles more relationship types Data access is flexible Data owner/member relationship promotes data integrity Conformance to standards Includes data definition language (DDL) and data manipulation language (DML)

System complexity limits efficiency Navigational system yields complex implementation, application development, and management Structural changes require changes in all application programs 32

Slide 33 Relational Model Advantages Disadvantages Structural independence is promoted using independent tables Tabular view improves conceptual simplicity Ad hoc query capability is based on SQL Isolates the end user from physical-level details Improves implementation and management simplicity Requires substantial hardware and system software overhead Conceptual simplicity gives untrained people the tools to use a good system poorly May promote information problems 33

Slide 34 Entity Relationship Model Advantages Disadvantages Visual modeling yields conceptual simplicity Visual representation makes it an effective communication tool Is integrated with the dominant relational model Limited constraint representation Limited relationship representation No data manipulation language Loss of information content occurs when attributes are removed from entities to avoid crowded displays 34

Slide 35 Object-Oriented Model Advantages Disadvantages Semantic content is added Visual representation includes semantic content Inheritance promotes data integrity Slow development of standards caused vendors to supply their own enhancements Compromised widely accepted standard Complex navigational system Learning curve is steep High system overhead slows transactions 35

Slide 36 NoSQL Advantages Disadvantages

High scalability, availability, and fault tolerance are provided Uses low-cost commodity hardware Supports Big Data Key-value model improves storage efficiency Complex programming is required There is no relationship support There is no transaction integrity support In terms of data consistency, it provides an eventually consistent model 36

Slide 37 Table 2.3 - Data Model Basic Terminology Comparison 37

Slide 38 Figure 2.7 - Data Abstraction Levels 38

Slide 39 The External Model End users’ view of the data environment ER diagrams are used to represent the external views External schema: Specific representation of an external view 39

Slide 40 Figure 2.8 - External Models For Tiny College 40

Slide 41 The Conceptual Model Represents a global view of the entire database by the entire organization Conceptual schema: Basis for the identification and high-level description of the main data objects Has a macro-level view of data environment Is software and hardware independent Logical design: Task of creating a conceptual data model 41

Slide 42 Figure 2.9 - Conceptual Model For Tiny College 42

Slide 43 The Internal Model

Representing database as seen by the DBMS mapping conceptual model to the DBMS Internal schema: Specific representation of an internal model Uses the database constructs supported by the chosen database Is software dependent and hardware independent Logical independence: Changing internal model without affecting the conceptual model 43

Slide 44 Figure 2.10 - Internal Model for Tiny College 44

Slide 45 The Physical Model Operates at lowest level of abstraction Describes the way data are saved on storage media such as disks or tapes Requires the definition of physical storage and data access methods Relational model aimed at logical level Does not require physical-level details Physical independence: Changes in physical model do not affect internal model 45

Slide 46 Table 2.4 - Levels of Data Abstraction 46

1305627482_520058.pptx Slide 1 Chapter 3 The Relational Database Model

Slide 2 Learning Objectives In this chapter, you will learn: That the relational database model offers a logical view of data About the relational model’s basic component: relations That relations are logical constructs composed of rows (tuples) and columns (attributes) That relations are implemented as tables in a relational DBMS 2

Slide 3 Learning Objectives In this chapter, you will learn: About relational database operators, the data dictionary, and the system catalog How data redundancy is handled in the relational database model Why indexing is important 3

Slide 4 A Logical View of Data Relational database model enables logical representation of the data and its relationships Logical simplicity yields simple and effective database design methodologies Facilitated by the creation of data relationships based on a logical construct called a relation 4

Slide 5 Table 3.1 - Characteristics of a Relational Table 5

Slide 6 Keys Consist of one or more attributes that determine other attributes Used to: Ensure that each row in a table is uniquely identifiable Establish relationships among tables and to ensure the integrity of the data Primary key (PK): Attribute or combination of attributes that uniquely identifies any given row 6

Slide 7

Determination State in which knowing the value of one attribute makes it possible to determine the value of another Basis for establishing the role of a key Based on the relationships among the attributes 7

Slide 8 Dependencies Functional dependence: Value of one or more attributes determines the value of one or more other attributes Determinant: Attribute whose value determines another Dependent: Attribute whose value is determined by the other attribute Full functional dependence: Entire collection of attributes in the determinant is necessary for the relationship 8

Slide 9 Types of Keys Composite key: Key that is composed of more than one attribute Key attribute: Attribute that is a part of a key Entity integrity: Condition in which each row in the table has its own unique identity All of the values in the primary key must be unique No key attribute in the primary key can contain a null 9

Slide 10 Types of Keys Null: Absence of any data value that could represent: An unknown attribute value A known, but missing, attribute value A inapplicable condition Referential integrity: Every reference to an entity instance by another entity instance is valid Secondary key: Key used strictly for data retrieval purposes 10

Slide 11 Figure 3.2 - An Example of a Simple Relational Database 11

Slide 12 Table 3.3 - Relational Database Keys 12

Slide 13 Integrity Rules 13

Slide 14 Figure 3.3 - An Illustration of Integrity Rules 14

Slide 15 Ways to Handle Nulls Flags: Special codes used to indicate the absence of some value NOT NULL constraint - Placed on a column to ensure that every row in the table has a value for that column UNIQUE constraint Restriction placed on a column to ensure that no duplicate values exist for that column 15

Slide 16 Relational Algebra Theoretical way of manipulating table contents using relational operators Relvar: Variable that holds a relation Heading contains the names of the attributes and the body contains the relation Relational operators have the property of closure Closure: Use of relational algebra operators on existing relations produces new relations 16

Slide 17 Relational Set Operators 17

Slide 18 Figure 3.4 - Select 18

Slide 19 Figure 3.5 - Project 19

Slide 20 Figure 3.6 - Union and Figure 3.7 - Intersect 20

Slide 21 Relational Set Operators Difference Yields all rows in one table that are not found in the other table Tables must be union-compatible to yield valid results Product Yields all possible pairs of rows from two tables

21

Slide 22 Figure 3.8 – Difference 22

Slide 23 Figure 3.9 - Product 23

Slide 24 Relational Set Operators Join Allows information to be intelligently combined from two or more tables Divide Uses one 2-column table as the dividend and one single-column table as the divisor Output is a single column that contains all values from the second column of the dividend that are associated with every row in the divisor 24

Slide 25 Types of Joins Natural join: Links tables by selecting only the rows with common values in their common attributes Join columns: Common columns Equijoin: Links tables on the basis of an equality condition that compares specified columns of each table Theta join: Extension of natural join, denoted by adding a theta subscript after the JOIN symbol 25

Slide 26 Types of Joins Inner join: Only returns matched records from the tables that are being joined Outer join: Matched pairs are retained and unmatched values in the other table are left null Left outer join: Yields all of the rows in the first table, including those that do not have a matching value in the second table Right outer join: Yields all of the rows in the second table, including those that do not have matching values in the first table 26

Slide 27 Figure 3.10 - Two Tables That Will Be Used in JOIN Illustrations 27

Slide 28

Figure 3.16 - Divide 28

Slide 29 Data Dictionary and the System Catalog Data dictionary: Description of all tables in the database created by the user and designer System catalog: System data dictionary that describes all objects within the database Homonyms and synonyms must be avoided to lessen confusion Homonym: Same name is used to label different attributes Synonym: Different names are used to describe the same attribute 29

Slide 30 Relationships within the Relational Database 1:M relationship - Norm for relational databases 1:1 relationship - One entity can be related to only one other entity and vice versa Many-to-many (M:N) relationship - Implemented by creating a new entity in 1:M relationships with the original entities Composite entity (Bridge or associative entity): Helps avoid problems inherent to M:N relationships, includes the primary keys of tables to be linked 30

Slide 31 Figure 3.21 - The 1:1 Relationship between PROFESSOR and DEPARTMENT 31

Slide 32 Figure 3.26 - Changing the M:N Relationship to Two 1:M Relationships 32

Slide 33 Figure 3.27 - The Expanded ER Model 33

Slide 34 Data Redundancy Revisited Relational database facilitates control of data redundancies through use of foreign keys To be controlled except the following circumstances Data redundancy must be increased to make the database serve crucial information purposes Exists to preserve the historical accuracy of the data 34

Slide 35 Figure 3.30 - The Relational Diagram for the Invoicing System

35

Slide 36 Indexes Orderly arrangement to logically access rows in a table Index key: Index’s reference point that leads to data location identified by the key Unique index: Index key can have only one pointer value associated with it Each index is associated with only one table 36

Slide 37 Table 3.8 – Dr. Codd’s 12 Relational Database Rules 37

Slide 38 Table 3.8 – Dr. Codd’s 12 Relational Database Rules 38

1305627482_520059.pptx Slide 1 Chapter 4 Entity Relationship (ER) Modeling

Slide 2 Learning Objectives In this chapter, you will learn: The main characteristics of entity relationship components How relationships between entities are defined, refined, and incorporated into the database design process How ERD components affect database design and implementation That real-world database design often requires the reconciliation of conflicting goals 2

Slide 3 Entity Relationship Model (ERM) Basis of an entity relationship diagram (ERD) ERD depicts the: Conceptual database as viewed by end user Database’s main components Entities Attributes Relationships Entity - Refers to the entity set and not to a single entity occurrence 3

Slide 4 Attributes Characteristics of entities Required attribute: Must have a value, cannot be left empty Optional attribute: Does not require a value, can be left empty Domain - Set of possible values for a given attribute Identifiers: One or more attributes that uniquely identify each entity instance 4

Slide 5 Figure 4.1 - The Attributes of the Student Entity: Chen and Crow’s Foot 5

Slide 6 Attributes Composite identifier: Primary key composed of more than one attribute Composite attribute: Attribute that can be subdivided to yield additional attributes Simple attribute: Attribute that cannot be subdivided Single-valued attribute: Attribute that has only a single value Multivalued attributes: Attributes that have many values 6

Slide 7 Figure 4.3 - A Multivalued Attribute in an Entity 7

Slide 8 Attributes Multivalued attributes: Attributes that have many values and require creating: Several new attributes, one for each component of the original multivalued attribute A new entity composed of the original multivalued attribute’s components Derived attribute: Attribute whose value is calculated from other attributes Derived using an algorithm 8

Slide 9 Figure 4.4 – Splitting the Multivalued Attributes into New Attributes 9

Slide 10 Figure 4.6 - Depiction of a Derived Attribute 10

Slide 11 Table 4.2 - Advantages and Disadvantages of Storing Derived Attributes 11

Slide 12 Relationships Association between entities that always operate in both directions Participants: Entities that participate in a relationship Connectivity: Describes the relationship classification Cardinality: Expresses the minimum and maximum number of entity occurrences associated with one occurrence of related entity 12

Slide 13 Figure 4.7 - Connectivity and Cardinality in an ERD 13

Slide 14 Existence Dependence 14

Slide 15 Relationship Strength 15

Slide 16 Figure 4.8 - A Weak (Non-Identifying) Relationship between COURSE and CLASS 16

Slide 17 Figure 4.9 - A Strong (Identifying) Relationship between COURSE and CLASS 17

Slide 18 Weak Entity Conditions Existence-dependent Has a primary key that is partially or totally derived from parent entity in the relationship Database designer determines whether an entity is weak based on business rules 18

Slide 19 Figure 4.10 - A Weak Entity in an ERD 19

Slide 20 Figure 4.11 - A Weak Entity in a Strong Relationship 20

Slide 21 Relationship Participation 21

Slide 22 Table 4.3 - Crow’s Foot Symbols 22

Slide 23

Figure 4.13 - CLASS is Optional to COURSE 23

Slide 24 Figure 4.14 - COURSE and CLASS in a Mandatory Relationship 24

Slide 25 Relationship Degree Indicates the number of entities or participants associated with a relationship Unary relationship: Association is maintained within a single entity Recursive relationship: Relationship exists between occurrences of the same entity set Binary relationship: Two entities are associated Ternary relationship: Three entities are associated 25

Slide 26 Figure 4.15 - Three Types of Relationship Degree 26

Slide 27 Figure 4.17 - An ER Representation of Recursive Relationships 27

Slide 28 Associative (Composite) Entities Used to represent an M:N relationship between two or more entities Is in a 1:M relationship with the parent entities Composed of the primary key attributes of each parent entity May also contain additional attributes that play no role in connective process 28

Slide 29 Figure 4.23 - Converting the M:N Relationship into Two 1:M Relationships 29

Slide 30 Figure 4.25 - A Composite Entity in an ERD 30

Slide 31

Developing an ER Diagram Create a detailed narrative of the organization’s description of operations Identify business rules based on the descriptions Identify main entities and relationships from the business rules Develop the initial ERD Identify the attributes and primary keys that adequately describe entities Revise and review ERD 31

Slide 32 Figure 4.26 - The First Tiny College ERD Segment 32

Slide 33 Figure 4.27 - The Second Tiny College ERD Segment 33

Slide 34 Figure 4.28 - The Third Tiny College ERD Segment 34

Slide 35 Figure 4.29 - The Fourth Tiny College ERD Segment 35

Slide 36 Figure 4.30 - The Fifth Tiny College ERD Segment 36

Slide 37 Figure 4.31 - The Sixth Tiny College ERD Segment 37

Slide 38 Figure 4.32 - The Seventh Tiny College ERD Segment 38

Slide 39 Figure 4.33 - The Eighth Tiny College ERD Segment 39

Slide 40 Figure 4.34 - The Ninth Tiny College ERD Segment 40

Slide 41 Table 4.4 - Components of the ERM 41

Slide 42 Database Design Challenges: Conflicting Goals 42

Slide 43 Figure 4.38 - Various Implementations of the 1:1 Recursive Relationship 43

1305627482_520060.pptx Slide 1 Chapter 5 Advanced Data Modeling

Slide 2 Learning Objectives In this chapter, you will learn: About the extended entity relationship (EER) model How entity clusters are used to represent multiple entities and relationships The characteristics of good primary keys and how to select them How to use flexible solutions for special data-modeling cases 2

Slide 3 Extended Entity Relationship Model (EERM) Result of adding more semantic constructs to the original entity relationship (ER) model EER diagram (EERD): Uses the EER model 3

Slide 4 Entity Supertypes and Subtypes Entity supertype: Generic entity type related to one or more entity subtypes Contains common characteristics Entity subtype: Contains unique characteristics of each entity subtype Criteria to determine the usage There must be different, identifiable kinds of the entity in the user’s environment The different kinds of instances should each have one or more attributes that are unique to that kind of instance 4

Slide 5 Specialization Hierarchy Depicts arrangement of higher-level entity supertypes and lower-level entity subtypes Relationships are described in terms of “is-a” relationships Subtype exists within the context of a supertype Every subtype has one supertype to which it is directly related Supertype can have many subtypes 5

Slide 6 Specialization Hierarchy Provides the means to: Support attribute inheritance Define a special supertype attribute known as the subtype discriminator Define disjoint/overlapping constraints and complete/partial constraints 6

Slide 7 Figure 5.2 - Specialization Hierarchy 7

Slide 8 Inheritance Enables an entity subtype to inherit attributes and relationships of the supertype All entity subtypes inherit their primary key attribute from their supertype At the implementation level, supertype and its subtype(s) maintain a 1:1 relationship Entity subtypes inherit all relationships in which supertype entity participates Lower-level subtypes inherit all attributes and relationships from its upper-level supertypes 8

Slide 9 Subtype Discriminator Attribute in the supertype entity that determines to which entity subtype the supertype occurrence is related Default comparison condition is the equality comparison 9

Slide 10 Disjoint and Overlapping Constraints Disjoint subtypes: Contain a unique subset of the supertype entity set Known as nonoverlapping subtypes Implementation is based on the value of the subtype discriminator attribute in the supertype Overlapping subtypes: Contain nonunique subsets of the supertype entity set Implementation requires the use of one discriminator attribute for each subtype 10

Slide 11 Figure 5.4 - Specialization Hierarchy with Overlapping Subtypes 11

Slide 12 Table 5.1 - Discriminator Attributes with Overlapping Subtypes 12

Slide 13 Completeness Constraint Specifies whether each supertype occurrence must also be a member of at least one subtype Types Partial completeness: Not every supertype occurrence is a member of a subtype Total completeness: Every supertype occurrence must be a member of any

13

Slide 14 Table 5.2 - Specialization Hierarchy Constraint Scenarios 14

Slide 15 Specialization and Generalization 15

Slide 16 Entity Cluster Virtual entity type used to represent multiple entities and relationships in ERD Avoid the display of attributes to eliminate complications that result when the inheritance rules change 16

Slide 17 Figure 5.5 - Tiny College ERD Using Entity Clusters 17

Slide 18 Primary Keys Single attribute or a combination of attributes, which uniquely identifies each entity instance Guarantees entity integrity Works with foreign keys to implement relationships 18

Slide 19 Natural Keys or Natural Identifier Real-world identifier used to uniquely identify real-world objects Familiar to end users and forms part of their day-to-day business vocabulary Also known as natural identifier Used as the primary key of the entity being modeled 19

Slide 20 Desirable Primary Key Characteristics 20

Slide 21

Use of Composite Primary Keys Identifiers of composite entities Each primary key combination is allowed once in M:N relationship Identifiers of weak entities Weak entity has a strong identifying relationship with the parent entity 21

Slide 22 Use of Composite Primary Keys When used as identifiers of weak entities, represent a real-world object that is: Existence-dependent on another real-world object Represented in the data model as two separate entities in a strong identifying relationship 22

Slide 23 Figure 5.6 - The M:N Relationship between STUDENT and CLASS 23

Slide 24 Surrogate Primary Keys Primary key used to simplify the identification of entity instances are useful when: There is no natural key Selected candidate key has embedded semantic contents or is too long Require ensuring that the candidate key of entity in question performs properly Use unique index and not null constraints 24

Slide 25 Table 5.4 - Data Used to Keep Track of Events 25

Slide 26 Design Case 1: Implementing 1:1 Relationships Foreign keys work with primary keys to properly implement relationships in relational model Rule Put primary key of the parent entity on the dependent entity as foreign key Options for selecting and placing the foreign key: Place a foreign key in both entities Place a foreign key in one of the entities 26

Slide 27 Table 5.5 - Selection of Foreign Key in a 1:1 Relationship 27

Slide 28

Figure 5.7 - The 1:1 Relationship between Department and Employee 28

Slide 29 Design Case 2: Maintaining History of Time-Variant Data Time-variant data: Data whose values change over time and for which a history of the data changes must be retained Requires creating a new entity in a 1:M relationship with the original entity New entity contains the new value, date of the change, and other pertinent attribute 29

Slide 30 Figure 5.8 - Maintaining Salary History 30

Slide 31 Figure 5.9 - Maintaining Manager History 31

Slide 32 Figure 5.10 - Maintaining Job History 32

Slide 33 Design Case 3: Fan Traps Design trap: Occurs when a relationship is improperly or incompletely identified Represented in a way not consistent with the real world Fan trap: Occurs when one entity is in two 1:M relationships to other entities Produces an association among other entities not expressed in the model 33

Slide 34 Figure 5.11 - Incorrect ERD with Fan Trap Problem 34

Slide 35 Figure 5.12 - Corrected ERD After Removal of the Fan Trap 35

Slide 36

Design Case 4: Redundant Relationships Occur when there are multiple relationship paths between related entities Need to remain consistent across the model Help simplify the design 36

Slide 37 Figure 5.13 - A Redundant Relationship 37

1305627482_520061.pptx Slide 1 Chapter 6 Normalization of Database Tables

Slide 2 Learning Objectives In this chapter, you will learn: What normalization is and what role it plays in the database design process About the normal forms 1NF, 2NF, 3NF, BCNF, and 4NF How normal forms can be transformed from lower normal forms to higher normal forms That normalization and ER modeling are used concurrently to produce a good database design That some situations require denormalization to generate information efficiently

Slide 3 Normalization Evaluating and correcting table structures to minimize data redundancies Reduces data anomalies Assigns attributes to tables based on determination Normal forms First normal form (1NF) Second normal form (2NF) Third normal form (3NF)

Slide 4 Normalization Structural point of view of normal forms Higher normal forms are better than lower normal forms Properly designed 3NF structures meet the requirement of fourth normal form (4NF) Denormalization: Produces a lower normal form Results in increased performance and greater data redundancy

Slide 5 Need for Normalization Used while designing a new database structure Analyzes the relationship among the attributes within each entity Determines if the structure can be improved Improves the existing data structure and creates an appropriate database design

Slide 6 Normalization Process Objective is to ensure that each table conforms to the concept of well-formed relations Each table represents a single subject No data item will be unnecessarily stored in more than one table All nonprime attributes in a table are dependent on the primary key Each table is void of insertion, update, and deletion anomalies

Slide 7 Normalization Process

Ensures that all tables are in at least 3NF Higher forms are not likely to be encountered in business environment Works one relation at a time Starts by: Identifying the dependencies of a relation (table) Progressively breaking the relation into new set of relations

Slide 8 Table 6.2 - Normal Forms

Slide 9 Table 6.3 - Functional Dependence Concepts

Slide 10 Types of Functional Dependencies Partial dependency: Functional dependence in which the determinant is only part of the primary key Assumption - One candidate key Straight forward Easy to identify Transitive dependency: An attribute functionally depends on another nonkey attribute

Slide 11 Conversion to First Normal Form Repeating group: Group of multiple entries of same type can exist for any single key attribute occurrence Existence proves the presence of data redundancies Enable reducing data redundancies Steps Eliminate the repeating groups Identify the primary key Identify all dependencies

Slide 12 Conversion to First Normal Form Dependency diagram: Depicts all dependencies found within given table structure Helps to get an overview of all relationships among table’s attributes Makes it less likely that an important dependency will be overlooked

Slide 13 Conversion to First Normal Form 1NF describes tabular format in which: All key attributes are defined There are no repeating groups in the table All attributes are dependent on the primary key All relational tables satisfy 1NF requirements Some tables contain partial dependencies Subject to data redundancies and various anomalies

Slide 14 Figure 6.3 - First Normal Form (1NF) Dependency Diagram

Slide 15 Conversion to Second Normal Form

Steps Make new tables to eliminate partial dependencies Reassign corresponding dependent attributes Table is in 2NF when it: Is in 1NF Includes no partial dependencies

Slide 16 Figure 6.4 - Second Normal Form (2NF) Conversion Results

Slide 17 Figure 6.5 - Third Normal Form (3NF) Conversion Results

Slide 18 Conversion to Third Normal Form Steps Make new tables to eliminate transitive dependencies Determinant: Any attribute whose value determines other values within a row Reassign corresponding dependent attributes Table is in 3NF when it: Is in 2NF Contains no transitive dependencies

Slide 19 Improving the Design Evaluate PK assignments and naming conventions Refine attribute atomicity Atomic attribute: Cannot be further subdivided Atomicity: Characteristic of an atomic attribute Identify new attributes and new relationships Refine primary keys as required for data granularity Granularity: Level of detail represented by the values stored in a table’s row Maintain historical accuracy and evaluate using derived attributes

Slide 20 Figure 6.6 - The Completed Database

Slide 21 Figure 6.6 - The Completed Database (cont’d.)

Slide 22 Surrogate Key Considerations Used by designers when the primary key is considered to be unsuitable System-defined attribute Created an managed via the DBMS Have a numeric value which is automatically incremented for each new row

Slide 23 The Boyce-Codd Normal Form (BCNF) Every determinant in the table should be a candidate key Candidate key - Same characteristics as primary key but not chosen to be the primary key Equivalent to 3NF when the table contains only one candidate key Violated only when the table contains more than one candidate key Considered to be a special case of 3NF

Slide 24 Figure 6.8 - A Table That is in 3NF and not in BCNF

Slide 25 Figure 6.9 – Decomposition to BCNF

Slide 26 Table 6.5 - Sample Data for a BCNF Conversion

Slide 27 Fourth Normal Form (4NF) Table is in 4NF when it: Is in 3NF Has no multivalued dependencies Rules All attributes must be dependent on the primary key, but they must be independent of each other No row may contain two or more multivalued facts about an entity

Slide 28 Figure 6.11 - Tables with Multivalued Dependencies

Slide 29 Figure 6.12 - A Set of Tables in 4NF

Slide 30 Figure 6.12 - A Set of Tables in 4NF (cont’d.)

Slide 31 Normalization and Database Design Normalization should be part of the design process Proposed entities must meet required the normal form before table structures are created Principles and normalization procedures to be understood to redesign and modify databases ERD is created through an iterative process Normalization focuses on the characteristics of specific entities

Slide 32 Figure 6.13 - Initial Contracting Company ERD

Slide 33 Figure 6.14 - Modified Contracting Company ERD

Slide 34

Figure 6.15 - Incorrect M:N Relationship Representation

Slide 35 Figure 6.16 - Final Contracting Company ERD

Slide 36 Figure 6.17 - The Implemented Database

Slide 37 Denormalization Design goals Creation of normalized relations Processing requirements and speed Number of database tables expands when tables are decomposed to conform to normalization requirements Joining a larger number of tables: Takes additional input/output (I/O) operations and processing logic Reduces system speed

Slide 38 Denormalization Defects in unnormalized tables Data updates are less efficient because tables are larger Indexing is more cumbersome No simple strategies for creating virtual tables known as views

Slide 39 Table 6.6 – Common Denormalization Examples

Slide 40 Table 6.7 - Data-Modeling Checklist

Slide 41 Table 6.7 - Data-Modeling Checklist (cont’d.)

Slide 42 Table 6.7 - Data-Modeling Checklist (cont’d.)

Slide 43 Table 6.7 - Data-Modeling Checklist (cont’d.)

1305627482_520062.pptx Slide 1 Chapter 7 Introduction to Structured Query Language (SQL)

Slide 2 Learning Objectives In this chapter, you will learn: The basic commands and functions of SQL How to use SQL for data administration (to create tables and indexes) How to use SQL for data manipulation (to add, modify, delete, and retrieve data) How to use SQL to query a database for useful information 2

Slide 3 Introduction to SQL Categories of SQL functions: Data definition language (DDL) Data manipulation language (DML) Nonprocedural language with basic command vocabulary set of less than 100 words Differences in SQL dialects are minor 3

Slide 4 Table 7.1 - SQL Data Definition Command 4

Slide 5 Table 7.2 - SQL Data Manipulation Commands 5

Slide 6 Figure 7.1 - The Database Model 6

Slide 7 Creating the Database Create database structure RDBMS creates physical files that will hold database Differs from one RDBMS to another Authentication is the process DBMS uses to verify that only registered users access the database Required for the creation tables User should log on to RDBMS using user ID and password created by database administrator 7

Slide 8 The Database Schema Logical group of database objects – such as tables and indexes - related to each other Command: CREATE SCHEMA AUTHORIZATION {creator}; Seldom used directly as command is usually optional 8

Slide 9 Table 7.4 - Common SQL Data Types 9

Slide 10 Creating Table Structures Use one line per column (attribute) definition Use spaces to line up attribute characteristics and constraints Table and attribute names are fully capitalized Features of table creating command sequence: NOT NULL specification ensures data entry UNIQUE specification avoids duplicated values Table definition enclosed in parentheses RDBMS automatically enforces referential integrity for foreign keys.

10

Slide 11 SQL Constraints 11

Slide 12 SQL Indexes When primary key is declared, DBMS automatically creates unique index The CREATE INDEX command can be used to create indexes on the basis of any selected attribute UNIQUE qualifier prevents a value that has been used before Composite indexes prevent data duplication To delete an index use the DROP INDEX command 12

Slide 13 Data Manipulation Commands 13

Slide 14 Data Manipulation Commands 14

Slide 15 Data Manipulation Commands 15

Slide 16 Inserting Table Rows with a SELECT Subquery Syntax INSERT INTO tablename SELECT columnlist FROM tablename Used to add multiple rows using another table as source SELECT command - Acts as a subquery and is executed first Subquery: Query embedded/nested inside another query 16

Slide 17 Selecting Rows Using Conditional Restrictions Can select partial table contents by placing restrictions on rows to be included Syntax enables to specify which rows to select: SELECT columnlist FROM tablelist [WHERE conditionlist]; WHERE clause adds conditional restrictions to the SELECT statement 17

Slide 18 Table 7.6 - Comparison Operators Adds conditional restrictions on selected character attributes and dates 18

Slide 19 Comparison Operators: Computed Columns and Column Aliases SQL accepts any valid expressions/formulas in the computed columns Alias: Alternate name given to a column or table in any SQL statement to improve the readability Computed column, an alias, and date arithmetic can be used in a single query 19

Slide 20 Arithmetic Operators The Rule of Precedence: Establish the order in which computations are completed Performed in this order: Operations within parentheses Power operations Multiplications and divisions Additions and subtractions 20

Slide 21 Table 7.7 - The Arithmetic Operators 21

Slide 22 Logical Operators: AND, OR and NOT OR and AND: Used to link multiple conditional expressions in a WHERE or HAVING clause OR requires only one of the conditional expressions to be true AND requires all of the conditional expressions to be true NOT is used to negate the result of a conditional expression Boolean algebra is dedicated to the use to logical operations 22

Slide 23 Figure 7.12 - Selected PRODUCT Table Attributes: The Logical OR 23

Slide 24 Figure 7.13 - Selected PRODUCT Table Attributes: The Logical AND 24

Slide 25 Figure 7.14 - Selected PRODUCT Table Attributes: The Logical AND and OR 25

Slide 26 Special Operators 26

Slide 27 Additional Data Definition Commands ALTER TABLE command: To make changes in the table structure Keywords use with the command ADD - Adds a column MODIFY - Changes column characteristics DROP - Deletes a column Used to: Add table constraints Remove table constraints 27

Slide 28 Changing a Column’s Data Type and Data Characteristics ALTER used to change data type and characteristics Some RDBMSs do not permit changes to data types unless column is empty Changes in characteristics are permitted if they do not alter the existing data type Syntax: Data Type: ALTER TABLE tablename MODIFY (columnname(datatype)); Data Characteristic: ALTER TABLE tablename MODIFY (columnname(characteristic)); 28

Slide 29 Adding and Dropping Columns Adding a column Use ALTER and ADD Do not include the NOT NULL clause for new column Dropping a column Use ALTER and DROP Some RDBMSs impose restrictions on the deletion of an attribute 29

Slide 30 Advanced Data Updates UPDATE command updates only data in existing rows If a relationship is established between entries and existing columns, the relationship can assign values to appropriate slots Arithmetic operators are useful in data updates In Oracle, ROLLBACK command undoes changes made by last two UPDATE statements 30

Slide 31 Copying Parts of Tables SQL permits copying contents of selected table columns Data need not be reentered manually into newly created table(s) Table structure is created Rows are added to new table using rows from another table 31

Slide 32 Adding Primary and Foreign Key Designations A created new table based on another table does not include old table’s integrity rule (no primary key) Can re-establish integrity rules using ALTER command Use ALTER TABLE command to ADD primary and foreign keys Composite primary keys and multiple foreign keys can be designated in a single SQL command 32

Slide 33 Deleting a Table from the Database DROP TABLE: Deletes table from database Syntax - DROP TABLE tablename; Can drop a table only if it is not the one side of any relationship RDBMS generates a foreign key integrity violation error message if the table is dropped 33

Slide 34 Additional SELECT Query Keywords

Logical operators work well in the query environment SQL provides useful functions that: Count Find minimum and maximum values Calculate averages SQL allows user to limit queries to entries: Having no duplicates Whose duplicates can be grouped 34

Slide 35 Ordering a Listing ORDER BY clause is useful when listing order is important Syntax - SELECT columnlist FROM tablelist [WHERE conditionlist] [ORDER BY columnlist [ASC | DESC]]; Cascading order sequence: Multilevel ordered sequence Created by listing several attributes after the ORDER BY clause 35

Slide 36 Listing Unique Values DISTINCT clause: Produces list of values that are unique Syntax - SELECT DISTINCT columnlist FROM tablelist; Placement of nulls does not affect list contents In Oracle can place nulls at top of list 36

Slide 37 Table 7.8 - Some Basic SQL Aggregate Functions 37

Slide 38 Grouping Data Frequency distributions created by GROUP BY clause within SELECT statement Syntax - SELECT columnlist FROM tablelist [WHERE conditionlist] [GROUP BY columnlist] [HAVING conditionlist] [ORDER BY columnlist [ASC | DESC]]; 38

Slide 39 HAVING Clause Extension of GROUP BY feature Applied to output of GROUP BY operation Used in conjunction with GROUP BY clause in second SQL command set Similar to WHERE clause in SELECT statement 39

Slide 40 Joining Database Tables Performed when data are retrieved from more than one table at a time Equality comparison between foreign key and primary key of related tables Tables are joined by listing tables in FROM

clause of SELECT statement DBMS creates Cartesian product of every table in the FROM clause 40

Slide 41 Joining Tables With an Alias and Recursive Joins Alias identifies source table from which data are taken Any legal table name can be used as alias Add alias after table name in FROM clause Recursive query: Table is joined to itself using alias Use aliases to differentiate the table from itself 41

1305627482_520063.pptx Slide 1 Chapter 8 Advanced SQL

Slide 2 Learning Objectives In this chapter, you will learn: How to use the advanced SQL JOIN operator syntax About the different types of subqueries and correlated queries How to use SQL functions to manipulate dates, strings, and other data About the relational set operators UNION, UNION ALL, INTERSECT, and MINUS 2

Slide 3 Learning Objectives In this chapter, you will learn: How to create and use views and updatable views How to create and use triggers and stored procedures How to create embedded SQL 3

Slide 4 SQL Join Operators Relational join operation merges rows from two tables and returns rows with one of the following: Natural join - common values in common columns Equality or inequality - meet a given join condition Outer join – common values in common columns or no matching values Inner join: Rows that meet a given criterion are selected Equality condition (natural join or equijoin) or inequality condition (theta join) Outer join: Returns matching rows and rows with unmatched attribute values for one or both joined tables 4

Slide 5 Table 8.1 - SQL Join Expression Styles 5

Slide 6 Subqueries and Correlated Queries Subquery is a query inside another query Subquery can return: One single value - One column and one row A list of values - One column and multiple rows A virtual table - Multicolumn, multirow set of values No value - Output of the outer query might result in an error or a null empty set 6

Slide 7 WHERE Subqueries Uses inner SELECT subquery on the right side of a WHERE comparison expression Value generated by the subquery must be of a comparable data type If the query returns more than a single value, the DBMS will generate an error Can be used in combination with joins 7

Slide 8 IN and HAVING Subqueries IN subqueries Used to compare a single attribute to a list of values HAVING subqueries HAVING clause restricts the output of a GROUP BY query by applying conditional criteria to the grouped rows 8

Slide 9 Multirow Subquery Operators: ANY and ALL ALL operator Allows comparison of a single value with a list of values returned by the first subquery Uses a comparison operator other than equals ANY operator Allows comparison of a single value to a list of values and selects only the rows for which the value is greater than or less than any value in the list 9

Slide 10 FROM and Attribute List Subqueries FROM clause: Specifies the tables from which the data will be drawn Can use SELECT subquery SELECT statement uses attribute list to indicate what columns to project in the resulting set Inline subquery Subquery expression included in the attribute list that must return one value Column alias cannot be used in attribute list computation if alias is defined in the same attribute list 10

Slide 11 Correlated Subqueries Executes once for each row in the outer query Inner query references a column of the outer subquery Can be used with the EXISTS special operator 11

Slide 12 SQL Functions Functions always use a numerical, date, or string value Value may be part of a command or may be an attribute located in a table Function may appear anywhere in an SQL statement where a value or an attribute can be used

12

Slide 13 SQL Functions 13

Slide 14 Relational Set Operators SQL data manipulation commands are set-oriented Set-oriented: Operate over entire sets of rows and columns at once UNION, INTERSECT, and Except (MINUS) work properly when relations are union-compatible Union-compatible: Number of attributes are the same and their corresponding data types are alike UNION Combines rows from two or more queries without including duplicate rows 14

Slide 15 Relational Set Operators Syntax - query UNION query UNION ALL Produces a relation that retains duplicate rows Can be used to unite more than two queries INTERSECT Combines rows from two queries, returning only the rows that appear in both sets Syntax - query INTERSECT query 15

Slide 16 Relational Set Operators EXCEPT (MINUS) Combines rows from two queries and returns only the rows that appear in the first set Syntax query EXCEPT query query MINUS query Syntax alternatives IN and NOT IN subqueries can be used in place of INTERSECT 16

Slide 17 Virtual Tables: Creating a View View: Virtual table based on a SELECT query Base tables: Tables on which the view is based CREATE VIEW statement: Data definition command that stores the subquery specification in the data dictionary CREATE VIEW command CREATE VIEW viewname AS SELECT query 17

Slide 18 Updatable Views Used to update attributes in any base tables used in the view Batch update routine: Pools multiple transactions into a single batch to update a master table field in a single operation Updatable view restrictions GROUP BY expressions or aggregate functions cannot be used Set operators cannot be used JOINs or group operators cannot be used

18

Slide 19 Oracle Sequences Independent object in the database Have a name and can be used anywhere a value expected Not tied to a table or column Generate a numeric value that can be assigned to any column in any table Table attribute with an assigned value can be edited and modified 19

Slide 20 Figure 8.27 - Oracle Sequence 20

Slide 21 Procedural SQL Performs a conditional or looping operation by isolating critical code and making all application programs call the shared code Yields better maintenance and logic control Persistent stored module (PSM): Block of code containing: Standard SQL statements Procedural extensions that is stored and executed at the DBMS server 21

Slide 22 Procedural SQL Procedural Language SQL (PL/SQL) Use and storage of procedural code and SQL statements within the database Merging of SQL and traditional programming constructs Procedural code is executed as a unit by DBMS when invoked by end user End users can use PL/SQL to create: Anonymous PL/SQL blocks and triggers Stored procedures and PL/SQL functions 22

Slide 23 Table 8.9 - PL/SQL Basic Data Types 23

Slide 24 Triggers Procedural SQL code automatically invoked by RDBMS when given data manipulation event occurs Parts of a trigger definition Triggering timing - Indicates when trigger’s PL/SQL code executes Triggering event - Statement that causes the trigger to execute Triggering level - Statement- and row-level Triggering action - PL/SQL code enclosed between the BEGIN and END keywords 24

Slide 25 Triggers DROP TRIGGER trigger_name command Deletes a trigger without deleting the table Trigger action based on DML predicates Actions depend on the type of DML statement that fires the trigger 25

Slide 26 Stored Procedures Named collection of procedural and SQL statements Advantages Reduce network traffic and increase performance Reduce code duplication by means of code isolation and code sharing 26

Slide 27 PL/SQL Processing with Cursors Cursor: Special construct used to hold data rows returned by a SQL query Implicit cursor: Automatically created when SQL statement returns only one value Explicit cursor: Holds the output of a SQL statement that may return two or more rows Cursor-style processing involves retrieving data from the cursor one row at a time Current row is copied to PL/SQL variables 27

Slide 28 Table 8.10 - Cursor Processing Commands 28

Slide 29 Table 8.11 - Cursor Attributes 29

Slide 30 PL/SQL Stored Functions Stored function: Named group of procedural and SQL statements that returns a value As indicated by a RETURN statement in its program code Can be invoked only from within stored procedures or triggers 30

Slide 31 Embedded SQL SQL statements contained within an application programming language Host language: Any language that contains embedded SQL statements Differences between SQL and procedural languages Run-time mismatch SQL is executed one instruction at a time Host language runs at

client side in its own memory space 31

Slide 32 Embedded SQL Processing mismatch Conventional programming languages process one data element at a time Newer programming environments manipulate data sets in a cohesive manner Data type mismatch Data types provided by SQL might not match data types used in different host languages 32

Slide 33 Embedded SQL Embedded SQL framework defines: Standard syntax to identify embedded SQL code within the host language Standard syntax to identify host variables Communication area used to exchange status and error information between SQL and host language 33

Slide 34 Table 8.12 - SQL Status and Error Reporting Variables 34

Slide 35 Embedded SQL Static SQL: Programmer uses predefined SQL statements and parameters SQL statements will not change while application is running Dynamic SQL: SQL statement is generated at run time Attribute list and condition are not known until end user specifies them Slower than static SQL Requires more computer resources 35

1305627482_520064.pptx Slide 1 Chapter 9 Database Design

Slide 2 Learning Objectives In this chapter, you will learn: That successful database design must reflect the information system of which the database is a part That successful information systems are developed within a framework known as the Systems Development Life Cycle (SDLC) 2

Slide 3 Learning Objectives In this chapter, you will learn: That within the information system, the most successful databases are subject to frequent evaluation and revision within a framework known as the Database Life Cycle (DBLC) How to conduct evaluation and revision within the SDLC and DBLC frameworks About database design strategies: top-down vs. bottom-up design and centralized vs. decentralized design 3

Slide 4 The Information System Provides for data collection, storage, and retrieval Composed of: People, hardware, software Database(s), application programs, procedures Systems analysis: Process that establishes need for and extent of information system Systems development: Process of creating information system 4

Slide 5 Performance Factors of an Information System Database design and implementation Application design and implementation Administrative procedures Database development: Process of database design and its implementation 5

Slide 6 Systems Development Life Cycle (SDLC) Traces history of an information system Provides a picture within which database design and application development are mapped out and evaluated Iterative rather than sequential process 6

Slide 7 Figure 9.2 - The Systems Development Life Cycle (SDLC) 7

Slide 8 Computer-Aided Systems Engineering (CASE) Includes System Architect and Visio Professional Helps produce better systems in a reasonable amounts of time and reasonable cost Applications are more structured, better documented and standardized Prolongs operational life of systems Easier and cheaper to update and maintain 8

Slide 9 Figure 9.3 - The Database Life Cycle (DBLC) 9

Slide 10 Purpose of Database Initial Study 10

Slide 11 Figure 9.4 - A Summary of Activities in the Database Initial Study 11

Slide 12 Database Design Supports company’s operations and objectives Most critical phase Ensures final product meets user and system requirements Points for examining completion procedures Data component is an element of whole system System analysts/programmers design procedures to convert data into information Database design is an iterative process 12

Slide 13 Figure 9.5 - Two Views of Data: Business Manager and Database Designer 13

Slide 14 Implementation and Loading Install the DBMS Virtualization: Creates logical representations of computing resources independent of underlying physical computing resources Create the databases Requires the

creation of special storage-related constructs to house the end-user tables Load or convert the data Requires aggregating data from multiple sources 14

Slide 15 Figure 9.6 - Database Design Process 15

Slide 16 Testing and Evaluation Physical security Password security Access rights Audit trails Data encryption Diskless workstations Optimization 16

Slide 17 Levels of Database Backups Full backup/dump: All database objects are backed up in their entirety Differential backup: Only modified/updated objects since last full backup are backed up Transaction log backup: Only the transaction log operations that are not reflected in a previous backup are backed up Backups are provided with high security 17

Slide 18 Table 9.1 – Common Sources of Database Failure 18

Slide 19 Maintenance and Evolution Preventive maintenance (backup) Corrective maintenance (recovery) Adaptive maintenance Assignment of access permissions and their maintenance for new and old users Generation of database access statistics Periodic security audits Periodic system-usage summaries 19

Slide 20 Figure 9.8 - Parallel Activities in the DBLC and the SDLC 20

Slide 21 Conceptual Design

Designs a database independent of database software and physical details Conceptual data model - Describes main data entities, attributes, relationships, and constrains Designed as software and hardware independent Minimum data rule: All that is needed is there, and all that is there is needed 21

Slide 22 Table 9.2 - Conceptual Design Steps 22

Slide 23 Data Analysis and Requirements Designers efforts are focused on Information needs, users, sources and constitution Answers obtained from a variety of sources Developing and gathering end-user data views Directly observing current system: existing and desired output Interfacing with the systems design group 23

Slide 24 Description of Operations 24

Slide 25 Table 9.3 - Developing the Conceptual Model Using ER Diagrams 25

Slide 26 Figure 9.10 - ER Modeling is an Iterative Process Based on Many Activities 26

Slide 27 Figure 9.11 - Conceptual Design Tools and Information Sources 27

Slide 28 Data Model Verification Verified against proposed system processes Module: Information system component that handles specific business function Better if modules’ ER fragments are merged into a single enterprise ER model which triggers Careful reevaluation of entities Detailed examination of attributes describing entities Resulting model verified against each of the module’s processes 28

Slide 29 Table 9.5 - The ER Model Verification Process 29

Slide 30 Figure 9.12 - Iterative ER Model Verification Process 30

Slide 31 Cohesivity and Module Coupling Cohesivity: Strength of the relationships among the module’s entities Module coupling: Extent to which modules are independent to one another Low coupling decreases unnecessary intermodule dependencies 31

Slide 32 Distributed Database Design Portions of database may reside in different physical locations Database fragment: Subset of a database stored at a given location Ensures database integrity, security, and performance 32

Slide 33 DBMS Software Selection Cost DBMS features and tools Underlying model Portability DBMS hardware requirements 33

Slide 34 Logical and Physical Design Logical design: Designs an enterprise-wide database that is based on a specific data model but independent of physical-level details Validates logical model: Using normalization Integrity constraints Against user requirements Physical design: Process of data storage organization and data access characteristics of the database 34

Slide 35 Table 9.6 - Logical Design Steps 35

Slide 36

Table 9.7 - Mapping the Conceptual Model to the Relational Model 36

Slide 37 Table 9.8 - Physical Design Steps 37

Slide 38 Clustered Tables and Database Role 38

Slide 39 Figure 9.14 - Top-down vs. Bottom-up Design Sequencing 39

Slide 40 Figure 9.15 - Centralized Design 40

Slide 41 Figure 9.16 - Decentralized Design 41

Slide 42 Figure 9.17 - Summary of Aggregation Problems 42

1305627482_520065.pptx Slide 1 Chapter 10 Transaction Management and Concurrency Control

Slide 2 Learning Objectives In this chapter, you will learn: About database transactions and their properties What concurrency control is and what role it plays in maintaining the database’s integrity What locking methods are and how they work How stamping methods are used for concurrency control How optimistic methods are used for concurrency control How database recovery management is used to maintain database integrity 2

Slide 3 Learning Objectives In this chapter, you will learn: How stamping methods are used for concurrency control How optimistic methods are used for concurrency control How database recovery management is used to maintain database integrity 3

Slide 4 What is a Transaction?

Logical unit of work that must be entirely completed or aborted Consists of: SELECT statement Series of related UPDATE statements Series of INSERT statements Combination of SELECT, UPDATE, and INSERT statements 4

Slide 5 What is a Transaction?

Consistent database state: All data integrity constraints are satisfied Must begin with the database in a known consistent state to ensure consistency Formed by two or more database requests Database requests: Equivalent of a single SQL statement in an application program or transaction 5

Slide 6 Evaluating Transaction Results Not all transactions update database SQL code represents a transaction because it accesses a database Improper or incomplete transactions can have devastating effect on database integrity Users can define enforceable constraints based on business rules Other integrity rules are

automatically enforced by the DBMS 6

Slide 7 Transaction Properties 7

Slide 8 Transaction Management with SQL SQL statements that provide transaction support COMMIT ROLLBACK Transaction sequence must continue until: COMMIT statement is reached ROLLBACK statement is reached End of program is reached Program is abnormally terminated 8

Slide 9 The Transaction Log Keeps track of all transactions that update the database DBMS uses the information stored in a log for: Recovery requirement triggered by a ROLLBACK statement A program’s abnormal termination A system failure 9

Slide 10 Table 10.1 – A Transaction Log 10

Slide 11 Concurrency Control Coordination of the simultaneous transactions execution in a multiuser database system Objective Ensures serializability of transactions in a multiuser database environment 11

Slide 12 Problems in Concurrency Control 12

Slide 13 The Scheduler Establishes the order in which the operations are executed within concurrent transactions Interleaves the execution of database operations to ensure serializability and isolation of

transactions Based on concurrent control algorithms to determine the appropriate order Creates serialization schedule Serializable schedule: Interleaved execution of transactions yields the same results as the serial execution of the transactions 13

Slide 14 Concurrency Control with Locking Methods Locking methods - Facilitate isolation of data items used in concurrently executing transactions Lock: Guarantees exclusive use of a data item to a current transaction Pessimistic locking: Use of locks based on the assumption that conflict between transactions is likely Lock manager: Responsible for assigning and policing the locks used by the transactions 14

Slide 15 Lock Granularity Indicates the level of lock use Levels of locking Database-level lock Table-level lock Page-level lock Page or diskpage: Directly addressable section of a disk Row-level lock Field-level lock 15

Slide 16 Figure 10.3 - Database-Level Locking Sequence 16

Slide 17 Figure 10.4 - An Example of a Table-Level Lock 17

Slide 18 Figure 10.5 - An Example of a Page-Level Lock 18

Slide 19 Figure 10.6 - An Example of a Row-Level Lock 19

Slide 20 Lock Types 20

Slide 21 Problems in Using Locks Resulting transaction schedule might not be serializable Schedule might create deadlocks 21

Slide 22 Two-Phase Locking (2PL) Defines how transactions acquire and relinquish locks Guarantees serializability but does not prevent deadlocks Phases Growing phase - Transaction acquires all required locks without unlocking any data Shrinking phase - Transaction releases all locks and cannot obtain any new lock 22

Slide 23 Two-Phase Locking (2PL) Governing rules Two transactions cannot have conflicting locks No unlock operation can precede a lock operation in the same transaction No data are affected until all locks are obtained 23

Slide 24 Figure 10.7 - Two-Phase Locking Protocol 24

Slide 25 Deadlocks Occurs when two transactions wait indefinitely for each other to unlock data Known as deadly embrace Control techniques Deadlock prevention Deadlock detection Deadlock avoidance Choice of deadlock control method depends on database environment 25

Slide 26 Table 10.13 - How a Deadlock Condition is Created 26

Slide 27 Time Stamping Assigns global, unique time stamp to each transaction Produces explicit order in which transactions are submitted to DBMS Properties Uniqueness: Ensures no equal time stamp values exist Monotonicity: Ensures time stamp values always increases 27

Slide 28 Time Stamping Disadvantages Each value stored in the database requires two additional stamp fields Increases memory needs Increases the database’s processing overhead Demands a lot of system resources 28

Slide 29 Table 10.14 - Wait/Die and Wound/Wait Concurrency Control Schemes 29

Slide 30 Concurrency Control with Optimistic Methods Optimistic approach: Based on the assumption that the majority of database operations do not conflict Does not require locking or time stamping techniques Transaction is executed without restrictions until it is committed 30

Slide 31 Phases of Optimistic Approach 31

Slide 32 Table 10.15 - Transaction Isolation Levels 32

Slide 33 Database Recovery Management Database recovery: Restores database from a given state to a previously consistent state Recovery transactions are based on the atomic transaction property Atomic transaction property: All portions of a transaction must be treated as a single logical unit of work If transaction operation cannot be completed: Transaction must be aborted Changes to database must be rolled back 33

Slide 34 Concepts that Affect Transaction Recovery 34

Slide 35 Techniques Used in Transaction Recovery Procedures

35

Slide 36 Recovery Process in Deferred-Write Technique Identify the last check point in the transaction log If transaction was committed before the last check point Nothing needs to be done If transaction was committed after the last check point Transaction log is used to redo the transaction If transaction had a ROLLBACK operation after the last check point Nothing needs to be done 36

Slide 37 Recovery Process in Write-Through Technique Identify the last checkpoint in the transaction log If transaction was committed before the last check point Nothing needs to be done If transaction was committed after the last checkpoint Transaction must be redone If transaction had a ROLLBACK operation after the last check point Transaction log is used to ROLLBACK the operations 37

1305627482_520066.pptx Slide 1 Chapter 11 Database Performance Tuning and Query Optimization

Slide 2 Learning Objectives In this chapter, you will learn: Basic database performance-tuning concepts How a DBMS processes SQL queries About the importance of indexes in query processing About the types of decisions the query optimizer has to make Some common practices used to write efficient SQL code How to formulate queries and tune the DBMS for optimal performance 2

Slide 3 Database Performance-Tuning Concepts Goal of database performance is to execute queries as fast as possible Database performance tuning: Set of activities and procedures that reduce response time of database system Fine-tuning the performance of a system requires that all factors must operate at optimum level with minimal bottlenecks 3

Slide 4 Table 11.1 - General Guidelines for Better System Performance 4

Slide 5 Performance Tuning: Client and Server Client side SQL performance tuning: Generates SQL query that returns correct answer in least amount of time Using minimum amount of resources at server Server side DBMS performance tuning: DBMS environment configured to respond to clients’ requests as fast as possible Optimum use of existing resources 5

Slide 6 DBMS Architecture All data in a database are stored in data files Data files automatically expand in predefined increments known as extends Data files are grouped in file groups or table spaces Table space or file group: Logical grouping of several data files that store data with similar characteristics Data cache or buffer cache: Shared, reserved memory area Stores most recently accessed data blocks in RAM

6

Slide 7 DBMS Architecture SQL cache or procedure cache: Stores most recently executed SQL statements or PL/SQL procedures DBMS retrieves data from permanent storage and places them in RAM Input/output request: Low-level data access operation that reads or writes data to and from computer devices Data cache is faster than working with data files Majority of performance-tuning activities focus on minimizing I/O operations 7

Slide 8 Figure 11.1 - Basic DBMS Architecture 8

Slide 9 Database Query Optimization Modes Algorithms proposed for query optimization are based on: Selection of the optimum order to achieve the fastest execution time Selection of sites to be accessed to minimize communication costs Evaluated on the basis of: Operation mode Timing of its optimization 9

Slide 10 Classification of Operation Modes Automatic query optimization: DBMS finds the most cost-effective access path without user intervention Manual query optimization: Requires that the optimization be selected and scheduled by the end user or programmer 10

Slide 11 Classification Based on Timing of Optimization Static query optimization: best optimization strategy is selected when the query is compiled by the DBMS Takes place at compilation time Dynamic query optimization: Access strategy is dynamically determined by the DBMS at run time, using the most up-to-date information about the database Takes place at execution time 11

Slide 12 Classification Based on Type of Information Used to Optimize the Query Statistically based query optimization algorithm: Statistics are used by the DBMS to determine the best access strategy Statistical information is generated by DBMS through: Dynamic statistical generation mode Manual statistical generation mode Rule-based query optimization algorithm:

based on a set of user-defined rules to determine the best query access strategy 12

Slide 13 Table 11.2 - Sample Database Statistics Measurements 13

Slide 14 Query Processing 14

Slide 15 Figure 11.2 – Query Processing 15

Slide 16 SQL Parsing Phase Query is broken down into smaller units Original SQL query transformed into slightly different version of original SQL code which is fully equivalent and more efficient Query optimizer: Analyzes SQL query and finds most efficient way to access data Access plans: DBMS-specific and translate client’s SQL query into a series of complex I/O operations If access plan already exists for query in SQL cache, DBMS reuses it If not, optimizer evaluates various plans and chooses one to be placed in SQL cache for use 16

Slide 17 SQL Execution Phase All I/O operations indicated in the access plan are executed Locks are acquired Data are retrieved and placed in data cache Transaction management commands are processed 17

Slide 18 SQL Fetching Phase Rows of resulting query result set are returned to client DBMS may use temporary table space to store temporary data Database server coordinates the movement of the result set rows from the server cache to the client cache 18

Slide 19 Query Processing Bottlenecks

Delay introduced in the processing of an I/O operation that slows the system Caused by the: CPU RAM Hard disk Network Application code 19

Slide 20 Indexes and Query Optimization Indexes Help speed up data access Facilitate searching, sorting, using aggregate functions, and join operations Ordered set of values that contain the index key and pointers More efficient than a full table scan 20

Slide 21 Indexes and Query Optimization Data sparsity: Number of different values a column could have Data structures used to implement indexes: Hash indexes B-tree indexes Bitmap indexes DBMSs determine best type of index to use 21

Slide 22 Optimizer Choices Rule-based optimizer: Uses preset rules and points to determine the best approach to execute a query Cost-based optimizer: Uses algorithms based on statistics about objects being accessed to determine the best approach to execute a query 22

Slide 23 Using Hints to Affect Optimizer Choices Optimizer might not choose the best execution plan Makes decisions based on existing statistics, which might be old Might choose less-efficient decisions Optimizer hints: Special instructions for the optimizer, embedded in the SQL command text 23

Slide 24 Table 11.5 - Optimizer Hints 24

Slide 25 SQL Performance Tuning Evaluated from client perspective Most current relational DBMSs perform automatic query optimization at the server end Most SQL performance optimization techniques are DBMS-specific and thus rarely portable Majority of performance problems are related to poorly written SQL code 25

Slide 26 Index Selectivity Measure of the likelihood that an index will be used in query processing Indexes are used when a subset of rows from a large table is to be selected based on a given condition Index cannot always be used to improve performance Function-based index: Based on a specific SQL function or expression 26

Slide 27 Conditional Expressions Expressed within WHERE or HAVING clauses of a SQL statement Restricts the output of a query to only rows matching conditional criteria Guidelines to write efficient conditional expressions in SQL code Use simple columns or literals as operands Numeric field comparisons are faster than character, date, and NULL comparisons 27

Slide 28 Conditional Expressions Equality comparisons are faster than inequality comparisons Transform conditional expressions to use literals Write equality conditions first when using multiple conditional expressions When using multiple AND conditions, write the condition most likely to be false first When using multiple OR conditions, put the condition most likely to be true first Avoid the use of NOT logical operator 28

Slide 29 Table 11.6 – Conditional Criteria 29

Slide 30 Query Formulation Identify what columns and computations are required Identify source tables Determine how to join tables Determine what selection criteria are needed Determine the order in which to display the output 30

Slide 31 DBMS Performance Tuning Managing DBMS processes in primary memory and the structures in physical storage DBMS performance tuning at server end focuses on setting parameters used for: Data cache SQL cache Sort cache Optimizer mode In-memory database: Store large portions of the database in primary storage

31

Slide 32 DBMS Performance Tuning Recommendations for physical storage of databases: Use RAID (Redundant Array of Independent Disks) to provide a balance between performance improvement and fault tolerance Minimize disk contention Put high-usage tables in their own table spaces Assign separate data files in separate storage volumes for indexes, system, and high-usage tables 32

Slide 33 DBMS Performance Tuning Take advantage of the various table storage organizations in the database Index-organized table or clustered index table: Stores the end-user data and the index data in consecutive locations in permanent storage Partition tables based on usage Use denormalized tables where appropriate Store computed and aggregate attributes in tables 33

1305627482_520067.pptx Slide 1 Chapter 12 Distributed Database Management Systems

Slide 2 Learning Objectives In this chapter, you will learn: About distributed database management systems (DDBMSs) and their components How database implementation is affected by different levels of data and process distribution How transactions are managed in a distributed database environment How distributed database design draws on data partitioning and replication to balance performance, scalability, and availability About the trade-offs of implementing a distributed data system 2

Slide 3 Evolution of Database Management Systems Distributed database management system (DDBMS) governs storage and processing of logically related data over interconnected computer systems Data and processing functions are distributed among several sites Centralized database management system Required that corporate data be stored in a single central site Data access provided through dumb terminals 3

Slide 4 Figure 12.1 - Centralized Database Management System 4

Slide 5 Changes that Affected the Nature of Systems Globalization of business operation Market needs for an on-demand transaction style, based on web-based services Rapid social and technological changes fueled by low-cost smart mobile devices Data realms converging in the digital world Advent of social media to reach new customers and markets 5

Slide 6 Database Requirements in a Dynamic Business Environment Rapid ad hoc data access Crucial in the quick-response decision-making environment Distributed data access Needed to support geographically dispersed business units 6

Slide 7 Factors that Influenced DDBMS 7

Slide 8 Potential Centralized DBMS Problems 8

Slide 9 Table 12.1 – Distributed DDBM Advantages and Disadvantages 9

Slide 10 Distributed Processing and Distributed Databases Distributed processing: Database’s logical processing is shared among two or more physically independent sites via network Distributed database: Stores logically related database over two or more physically independent sites via computer network Database fragments: Database composed of many parts in distributed database system 10

Slide 11 Figure 12.2 – Distributed Processing Environment 11

Slide 12 Figure 12.3 – Distributed Database Environment 12

Slide 13 Characteristics of Distributed Management Systems 13

Slide 14 Functions of Fully Distributed DBMS Receive the request of an application or end user Validate, analyze, and decompose the request Map request’s logical-to-physical data components Decompose request into several I/O operations Search, locate, read and validate data Ensure database consistency, security, and integrity Validate data for conditions specified by request Present data in required format Handle all necessary functions transparently to user

14

Slide 15 Figure 12.4 - A Fully Distributed Database Management System 15

Slide 16 DDBMS Components Computer workstations or remote devices Network hardware and software components Communications media Transaction processor (TP): Software component in each computer or device that requests data Known as transaction manager (TM) or application processor (AP) Data processor (DP) or data manager (DM) Software component on each computer or device that stores and retrieves data from located at the site 16

Slide 17 Single-Site Processing, Single-Site Data (SPSD) Processing is done on a single host computer Data stored on host computer’s local disk Processing restricted on end user’s side DBMS is accessed by dumb terminals 17

Slide 18 Figure 12.6 - Single-Site Processing, Single-Site Data (Centralized) 18

Slide 19 Multiple-Site Processing, Single-Site Data (MPSD) Multiple processes run on different computers sharing a single data repository Require network file server running conventional applications Accessed through LAN Client/server architecture Reduces network traffic Processing is distributed Supports data at multiple sites 19

Slide 20 Figure 12.7 - Multiple-Site Processing, Single-Site Data 20

Slide 21 Multiple-Site Processing, Multiple-Site Data (MPMD) Fully distributed database management system Support multiple data processors and transaction processors at multiple sites Classifications: Homogeneous: Integrate multiple instances of same

DBMS over a network Heterogeneous: Integrate different types of DBMSs over a network Fully heterogeneous: Support different DBMSs, each supporting different data model running under different computer systems 21

Slide 22 Restrictions of DDBMS Remote access is provided on a read-only basis Restrictions on the number of remote tables that may be accessed in a single transaction Restrictions on the number of distinct databases that may be accessed Restrictions on the database model that may be accessed 22

Slide 23 Distributed Database Transparency Features 23

Slide 24 Distribution Transparency Allows management of physically dispersed database as if centralized Levels: Fragmentation, location and local mapping Unique fragment: Each row is unique, regardless of the fragment in which it is located Supported by distributed data dictionary (DDD) or distributed data catalog (DDC) DDC contains the description of the entire database as seen by the database administrator Distributed global schema: Common database schema to translate user requests into subqueries 24

Slide 25 Transaction Transparency Ensures database transactions will maintain distributed database’s integrity and consistency Ensures transaction completed only when all database sites involved complete their part Distributed database systems require complex mechanisms to manage transactions 25

Slide 26 Distributed Requests and Distributed Transactions 26

Slide 27 Distributed Concurrency Control Concurrency control is especially important in distributed databases environment Multi-site, multiple-process operations are more likely to create inconsistencies and deadlocked transactions Solution to inconsistent database is a two-phase commit protocol

27

Slide 28 Figure 12.14 - The Effect of Premature COMMIT 28

Slide 29 Two-Phase Commit Protocol (2PC) Guarantees if a portion of a transaction operation cannot be committed, all changes made at the other sites will be undone To maintain a consistent database state Requires that each DP’s transaction log entry be written before database fragment is updated DO-UNDO-REDO protocol: Roll transactions back and forward with the help of the system’s transaction log entries 29

Slide 30 Two-Phase Commit Protocol (2PC) Write-ahead protocol: Forces the log entry to be written to permanent storage before actual operation takes place Defines operations between coordinator and subordinates Phases of implementation Preparation The final COMMIT 30

Slide 31 Performance and Failure Transparency Performance transparency allows a DDBMS to perform as if it were a centralized database Failure transparency ensures the system will operate in case of network failure Objective of query optimization is to minimize total costs which are a function of: Access time (I/O) cost involved in accessing data from multiple remote sites Communication costs associated with data transmission CPU time cost associated with the processing overhead 31

Slide 32 Performance and Failure Transparency Considerations for resolving data requests in a distributed data environment: Data distribution and data replication Replica transparency: DDBMS’s ability to hide multiple copies of data from the user Network and node availability Network latency: Delay imposed by the amount of time required for a data packet to make a round trip Network partitioning: Delay imposed when nodes become suddenly unavailable due to a network failure 32

Slide 33 Distributed Database Design 33

Slide 34 Data Fragmentation Breaks a single object into two or more segments Information is stored in distributed data catalog (DDC) Strategies Horizontal fragmentation: Division of a relation into subsets (fragments) of tuples (rows) Vertical fragmentation: Division of a relation into attribute (column) subsets Mixed fragmentation: Combination of horizontal and vertical strategies 34

Slide 35 Data Replication Storage of data copies at multiple sites served by a computer network Mutual consistency rule requires all copies of data fragments be identical Styles of replication Push replication focuses on maintaining data consistency Pull replication focuses on maintaining data availability and allows for temporary data inconistencies 35

Slide 36 Data Replication Scenarios 36

Slide 37 Data Allocation Strategies 37

Slide 38 The CAP Theorem CAP stands for: Consistency Availability Partition tolerance Trade-off between consistency and availability generated in a new system in which data is basically available, soft state, eventually consistent (BASE) Data changes are not immediate but propagate slowly through the system until all replicas are consistent 38

Slide 39 Table 12.8 - Distributed Database Spectrum 39

Slide 40 Table 12.9 - C. J. Date’s 12 Commandments for Distributed Databases 40

Slide 41 Table 12.9 - C. J. Date’s 12 Commandments for Distributed Databases (cont’d.)

41

1305627482_520068.pptx Slide 1 Chapter 13 Business Intelligence and Data Warehouses

Slide 2 Learning Objectives In this chapter, you will learn: How business intelligence provides a comprehensive business decision support framework About business intelligence architecture, its evolution, and reporting styles About the relationship and differences between operational data and decision support data What a data warehouse is and how to prepare data for one 2

Slide 3 Learning Objectives In this chapter, you will learn: What star schemas are and how they are constructed About data analytics About online analytical processing (OLAP) How SQL extensions are used to support OLAP-type data manipulations 3

Slide 4 Business Intelligence (BI) Comprehensive, cohesive, integrated set of tools and processes Captures, collects, integrates, stores, and analyzes data Generates and presents information to support business decision making Allows a business to transform: Data into information Information into knowledge Knowledge into wisdom 4

Slide 5 Business Intelligence (BI) Concepts, practices, tools and techniques to help business Understand its core capabilities Provide snapshots of the company situation Identify key opportunities to create a competitive advantage Provides a framework for Collecting and storing operational data and aggregating it into decision support data Analyzing decision support data and presenting generated information to end users to support business decisions Making business decision which generates more data Monitoring results to evaluate outcomes and predicting future outcomes with a high degree of accuracy 5

Slide 6 Figure 13.1 - Business Intelligence Framework

6

Slide 7 Table 13.3 – Sample of Business Intelligence Tools 7

Slide 8 Table 13.3 – Sample of Business Intelligence Tools (cont’d) 8

Slide 9 Practices to Manage Data Master data management (MDM): Collection of concepts, techniques, and processes for identification, definition, and management of data elements Governance: Method of government for controlling business health and for consistent decision making Key performance indicators (KPI): Numeric or scale-based measurements that assess company’s effectiveness in reaching its goals 9

Slide 10 Practices to Manage Data Data visualization: Abstracting data to provide information in a visual format Enhances the user’s ability to efficiently comprehend the meaning of the data Techniques: Pie charts and bar charts Line graphs Scatter plots Gantt charts Heat maps 10

Slide 11 Reporting Styles of a Modern BI System 11

Slide 12 Business Intelligence Benefits 12

Slide 13 Table 13.4 - Business Intelligence Evolution 13

Slide 14 Table 13.4 - Business Intelligence Evolution (cont’d)

14

Slide 15 Figure 13.3 - Evolution of BI Information Dissemination Formats 15

Slide 16 Business Intelligence Technology Trends 16

Slide 17 Decision Support Data Effectiveness of BI depends on quality of data gathered at operational level Operational data Seldom well-suited for decision support tasks Stored in relational database with highly normalized structures Optimized to support transactions representing daily operations 17

Slide 18 Decision Support Data Differ from operational data in: Time span Granularity Drill down: Decomposing a data to a lower level Roll up: Aggregating a data into a higher level Dimensionality 18

Slide 19 Table 13.5 - Contrasting Operational and Decision Support Data Characteristics 19

Slide 20 Decision Support Database Requirements Database schema Must support complex, non-normalized data representations Data must be aggregated and summarized Queries must be able to extract multidimensional time slices 20

Slide 21 Decision Support Database Requirements Data extraction and loading Allow batch and scheduled data extraction Support different data sources and check for inconsistent data or data validation rules Support advanced integration, aggregation, and classification Database size should support Very large databases (VLDBs) Advanced storage technologies Multiple-processor technologies

21

Slide 22 Table 13.8 - Characteristics of Data Warehouse Data and Operational Database Data 22

Slide 23 Figure 13.5 - The ETL Process Cengage Learning © 2015 23

Slide 24 Data Marts Small, single-subject data warehouse subset Provide decision support to a small group of people Benefits over data warehouses Lower cost and shorter implementation time Technologically advanced Inevitable people issues 24

Slide 25 Table 13.9 - Twelve Rules for a Data Warehouse 25

Slide 26 Star Schema Data-modeling technique Maps multidimensional decision support data into a relational database Creates the near equivalent of multidimensional database schema from existing relational database Yields an easily implemented model for multidimensional data analysis 26

Slide 27 Components of Star Schemas 27

Slide 28 Star Schema Representation Facts and dimensions represented by physical tables in data warehouse database Many-to-one (M:1) relationship between fact table and each dimension table Fact and dimension tables Related by foreign keys Subject to primary and foreign key constraints Primary key of a fact table Is a composite primary key because the fact table is related to many dimension tables Always formed by combining the foreign keys pointing to the related dimension tables

28

Slide 29 Performance-Improving Techniques for the Star Schema Normalizing dimensional tables Snowflake schema: Dimension tables can have their own dimension tables Maintaining multiple fact tables to represent different aggregation levels Denormalizing fact tables 29

Slide 30 Performance-Improving Techniques for the Star Schema Partitioning and replicating tables Partitioning: Splits tables into subsets of rows or columns and places them close to customer location Replication: Makes copy of table and places it in a different location Periodicity: Provides information about the time span of the data stored in the table 30

Slide 31 Online Analytical Processing (OLAP) Advanced data analysis environment that supports decision making, business modeling, and operations research Characteristics: Multidimensional data analysis techniques Advanced database support Easy-to-use end-user interfaces 31

Slide 32 Multidimensional Data Analysis Techniques Data are processed and viewed as part of a multidimensional structure Augmented by the following functions: Advanced data presentation functions Advanced data aggregation, consolidation, and classification functions Advanced computational functions Advanced data-modeling functions 32

Slide 33 Advanced Database Support OLAP tools must have the following features to deliver efficient decision support: Access to many different kinds of DBMSs, flat files, and internal and external data sources Access to aggregated data warehouse data and operational database detail data Advanced data navigation features Rapid and consistent query response times Ability to map end-user requests Support for very large databases 33

Slide 34 Easy-to-Use End-User Interface

Proper implementation leads to simple navigation and accelerated decision making or data analysis Advanced OLAP features are more useful when access is kept simple Many interface features are borrowed from previous generations of data analysis tools 34

Slide 35 Figure 13.17 - OLAP Architecture 35

Slide 36 Figure 13.18 - OLAP Server with Local Mini Data Marts 36

Slide 37 Relational Online Analytical Processing (ROLAP) Provides OLAP functionality using relational databases and familiar relational tools to store and analyze multidimensional data Extensions added to traditional RDBMS technology Multidimensional data schema support within the RDBMS Data access language and query performance optimized for multidimensional data Support for very large databases (VLDBs) 37

Slide 38 Multidimensional Online Analytical Processing (MOLAP) Extends OLAP functionality to multidimensional database management systems (MDBMSs) MDBMS: Uses proprietary techniques store data in matrix-like n-dimensional arrays End users visualize stored data as a 3D data cube Grow to n dimensions, becoming hypercubes Held in memory in a cube cache to speed access Sparsity: Measures the density of the data held in the data cube 38

Slide 39 Table 13.12 - Relational vs. Multidimensional OLAP 39

Slide 40 SQL Extensions for OLAP 40

Slide 41 Materialized View

Dynamic table that contains SQL query command to generate rows and stores the actual rows Created the first time query is run Summary rows are stored in the table Automatically updated when base tables are updated Requires specified privileges 41

1305627482_520069.pptx Slide 1 Chapter 15 Database Connectivity and Web Technologies

Slide 2 Learning Objectives In this chapter, you will learn: About database connectivity fundamentals About various database connectivity technologies How web-to-database middleware is used to integrate databases with the Internet What services are provided by web application servers What Extensible Markup Language (XML) is and why it is important for Web database development About cloud computing and how it enables the database-as-a-service model 2

Slide 3 Database Connectivity Database middleware: Provides an interface between the application program and the database Data repository - Data management application used to store data generated by an application program Universal Data Access (UDA): Collection of technologies used to access any type of data source and manage the data through a common interface ODBC, OLE-DB, ADO.NET form the backbone of MS UDA architecture 3

Slide 4 Native SQL Connectivity Connection interface provided by database vendors, which is unique to each vendor Interfaces are optimized for particular vendor’s DBMS Maintenance is a burden for the programmer 4

Slide 5 ODBC, DAO, and RDO Open Database Connectivity (ODBC):Microsoft’s implementation of a superset of SQL Access Group Call Level Interface (CLI) standard for database access Widely supported database connectivity interface Allows Windows application to access relational data sources by using SQL via standard application programming interface (API) 5

Slide 6 ODBC, DAO, and RDO

Data Access Objects (DAO): Object-oriented API used to access desktop databases such as MS Access and FileMaker Pro Provides an optimized interface that expose functionality of Jet data engine to programmers DAO interface can be used to access other relational style data sources 6

Slide 7 ODBC, DAO, and RDO Remote Data Objects (RDO) Higher-level object-oriented application interface used to access remote database servers Dynamic-link libraries (DLLs) Implements ODBC, DAO, and RDO as shared code that is dynamically linked to the Windows operating environment 7

Slide 8 Components of ODBC Architecture High-level ODBC API through which application programs access ODBC functionality Driver manager that is in charge of managing all database connections ODBC driver that communicates directly to DBMS 8

Slide 9 Figure 15.2 - Using ODBC, DAO, and RDO to Access Databases 9

Slide 10 Figure 15.3 - Configuring an Oracle Data Source 10

Slide 11 Object Linking and Embedding for Database (OLE-DB) Database middleware that adds object-oriented functionality for access to data Series of COM objects provides low-level database connectivity for applications Types of objects based on functionality Consumers (applications or processes) Providers (data or service) 11

Slide 12 Object Linking and Embedding for Database (OLE-DB) Does not provide support for scripting languages ActiveX Data Objects (ADO) provides: High-level application-oriented interface to interact with OLE-DB, DAO, and RDO Unified interface to access data from any programming language that uses the underlying OLE-DB objects 12

Slide 13 Figure 15.5 - OLE-DB Architecture 13

Slide 14 ADO.NET Data access component of Microsoft’s .NET application development framework Microsoft’s .NET framework Component-based platform for developing distributed, heterogeneous, interoperable applications Manipulates any type of data using any combination of network, operating system, and programming language Extends and enhances functionality critical for the development of distributed applications 14

Slide 15 ADO.NET DataSet: Disconnected memory-resident representation of the database Contains tables, columns, rows, relationships and constraints Internally stored in XML format Data in DataSet is made persistent as XML documents 15

Slide 16 Figure 15.6 - ADO.NET Framework 16

Slide 17 Java Database Connectivity (JDBC) Application programming interface that allows a Java program to interact with a wide range of data sources Advantages of JDBC: Company can leverage existing technology and personnel training Allows direct access to database server or access via database middleware Allows programmers to use their SQL skills to manipulate the data in the company's databases Provides a way to connect to databases through an ODBC driver 17

Slide 18 Figure 15.7 - JDBC Architecture 18

Slide 19 Database Internet Connectivity Allows new innovative services that: Permit rapid response by bringing new services and products to market quickly Increase customer satisfaction through creation of web-based support services

Allow anywhere, anytime data access using mobile smart devices via the Internet Yield fast and effective information dissemination through universal access 19

Slide 20 Table 15.3 - Characteristics and Benefits of Internet Technologies 20

Slide 21 Web-to-Database Middleware Web server is the main hub through which Internet services are accessed Server-side extension: Program that interacts directly with the web server Provides its services to the web server in a way that is totally transparent to the client browser Known as web-to-database middleware 21

Slide 22 Figure 15.8 - Web-to-Database Middleware 22

Slide 23 Figure 15.9 - Web Server CGI and API Interfaces 23

Slide 24 The Web Browser Software that lets users navigate the web from their client computer Interprets HTML code received from web server Presents different page components in standard way Web is a stateless system Stateless system: Web server does not know the status of any clients 24

Slide 25 Client-Side Extensions Add functionality to Web browser Types Plug-in: External application automatically invoked by the browser when needed Java and JavaScript: Embedded in web page Downloaded with the Web page and activated by an event ActiveX and VBScript: Embedded in web page Downloaded with page and activated by event Oriented to Windows applications 25

Slide 26 Web Application Servers

Middleware application that expands the functionality of web servers by linking them to a wide range of services Used to: Connect to and query database from web page Present database data in a webpage using various formats Create dynamic web search pages Create webpages to insert, update and delete data Enforce referential integrity Use simple and nested queries and program logic to represent business rules 26

Slide 27 Web Application Server Features Integrated development environment Security and user authentication Computational languages Automation generation of HTML pages Performance and fault -tolerant features Database access with transaction management capabilities Access to multiple services 27

Slide 28 Web Database Development Process of interfacing databases with the web browser Code examples ColdFusion PHP 28

Slide 29 Extensible Markup Language (XML) Represents and manipulates data elements Facilitates the exchange of structured documents over the Web Characteristics: Allows definition of new tags Case sensitive Must be well-formed and properly nested Comments indicated with <- and -> XML and xml prefixes reserved for XML tags only 29

Slide 30 Document Type Definitions (DTD) File with .dtd extension that describes elements Provides composition of database’s logical model Defines the syntax rules or valid tags for each type of XML document Companies engaging in e-commerce transaction must develop and share DTDs DTD referenced from inside XML document 30

Slide 31 XML Schemas Advanced data definition language Describes the structure of XML data documents Advantage More closely maps to database terminology and features XML schema definition (XSD): File uses syntax similar to XML document 31

Slide 32

XML Presentation XML separates data structure from presentation and processing Extensible Style Language (XSL) displays XML data Defines the rules by which XML data are formatted and displayed Parts: Extensible Style Language Transformations (XSLT) XSL style sheets 32

Slide 33 XML Applications B2B exchanges Legacy systems integration Web page development Database support Database meta-dictionaries XML databases XML services 33

Slide 34 Cloud Computing Services Computing model that enables access to a shared pool of configurable computer resources that can be: Rapidly provisioned Released with minimal management effort or service provider interaction Potential to become a game changer Eliminates financial and technological barriers 34

Slide 35 Figure 15.20 - Cloud Services 35

Slide 36 Cloud Implementation Types 36

Slide 37 Characteristics of Cloud Services Ubiquitous access via Internet technologies Shared infrastructure Lower costs and variable pricing Flexible and scalable services Dynamic provisioning Service orientation Managed operations 37

Slide 38 Figure 15.22 - Types of Cloud Services 38

Slide 39 Table 15.4 – Advantages and Disadvantages of Cloud Computing

39

Slide 40 SQL Data Services Cloud computing-based data management service Provides relational data management to companies Hosted data management and standard protocols Common programming interface Advantages: Reliable and scalable at a lower cost than in-house systems High level of fault tolerance Dynamic and automatic load balancing, automated data backup and disaster recovery are included Dynamic creation and allocation of processes and storage 40

1305627482_520070.pptx Slide 1 Chapter 16 Database Administration and Security

Slide 2 Learning Objectives In this chapter, you will learn: That data are a valuable business asset requiring careful management How a database plays a critical role in an organization That the introduction of a DBMS has important technological, managerial, and cultural consequences for an organization About the database administrator’s managerial and technical roles 2

Slide 3 Learning Objectives In this chapter, you will learn: About data security, database security, and the information security framework About several database administration tools and strategies How cloud-based data services impact the DBA’s role How various technical tasks of database administration are performed with Oracle 3

Slide 4 Figure 16.1 - The Data-Information-Decision Making Cycle 4

Slide 5 Data 5

Slide 6 Need for and Role of a Database in an Organization 6

Slide 7 Introduction of a Database: Special Considerations 7

Slide 8

Evolution of the Database Administration Function 8

Slide 9 Figure 16.2 - The IS Department’s Internal Organization 9

Slide 10 Figure 16.3 - The Placement of the DBA Function 10

Slide 11 Figure 16.4 - A DBA Functional Organization 11

Slide 12 Table 16.1 - Contrasting DA and DBA Activities and Characteristics 12

Slide 13 Table 16.2 - Desired DBA Skills 13

Slide 14 DBA’s Managerial Role Provide end-user support Enforce policies, procedures, and standards for correct data creation, usage, and distribution within the database Manage data security, privacy, and integrity Manage data backup and recovery Fully recover data in case of data loss Database security officer (DSO): Ensures database security and integrity 14

Slide 15 DBA’s Managerial Role Disaster management: Planning, organizing, and testing of database contingency plans and recovery procedures Backup and recovery measures must include at least periodic data and application backups: Full backup or database dump: Produces a complete copy of the entire database Incremental backup: Produces a backup of all data since the last backup date Concurrent backup: Takes place while the user is working on the database 15

Slide 16 DBA’s Managerial Role Backup and recovery measures must include at least: Proper backup identification Convenient and safe backup storage Physical protection of both hardware and software Personal access control to the software of a database installation Insurance coverage for the data in the database 16

Slide 17 DBA’s Managerial Role Additional points: Data recovery and contingency plans must be tested, evaluated and practiced frequently Backup and recovery plan not likely to cover all information system components Ensure data is distributed to the right people at the right time and in the right format 17

Slide 18 DBA’s Technical Role Evaluate, select, and install DBMS and related utilities Design and implement databases and applications Test and evaluate databases and applications Operate the DBMS, utilities, and applications Train and support users Maintain the DBMS, utilities, and applications 18

Slide 19 Security Goals Confidentiality: Protecting data against unauthorized access Compliance: Activities that meet data privacy and security reporting guidelines Integrity: Keeping data consistent and free of errors or anomalies Availability: Accessibility of data whenever required by authorized users and for authorized purposes 19

Slide 20 Security Policies Collection of standards, policies, and procedures created to guarantee security Ensures auditing and compliance Security audit process Identifies security vulnerabilities Identifies measures to protect the system 20

Slide 21 Security Vulnerabilities Weakness in a system component that could allow unauthorized access or cause service disruptions Categories: Technical, managerial, cultural, and procedural Security threat: Imminent security violation Security breach: Occurs when a security threat is exploited and could lead to a database whose integrity is preserved or corrupted

21

Slide 22 Table 16.4 - Sample Security Vulnerabilities and Related Protective Measures 22

Slide 23 Table 16.4 - Sample Security Vulnerabilities and Related Protective Measures 23

Slide 24 Database Security DBMS features and related measures that comply with the security requirements Authorization management: Procedures to protect database security and integrity User access management View definition DBMS access control DBMS usage monitoring Audit log: Automatically records description of database operations performed by all users 24

Slide 25 Database Administration Tools Database monitoring Database load testing Database performance tuning SQL code optimization Database bottleneck identification and remediation Database modeling and design Database data extraction, transformation, and loading 25

Slide 26 Data Dictionary Two main types: Integrated - Included with the DBMS Standalone - Third-party systems Active data dictionary: Automatically updated by the DBMS with every database access Passive data dictionary: Requires running a batch process Main function - Store description of all objects that interact with the database 26

Slide 27 Data Dictionary Key element of information resource management Can be described as the information resource dictionary Metadata is the basis for monitoring database use and for assigning access rights to users DBA uses data dictionary to support data analysis and design 27

Slide 28 Computer-Aided Systems Engineering (CASE) Tools Automated framework for the Systems Development Life Cycle (SDLC) Use structured methodologies and powerful graphical interfaces Classified according to extent of support provided: Front-end CASE tools: Provide support for the planning, analysis, and design phases Back-end CASE tools: Provide support for the coding and implementation phases 28

Slide 29 Components of a CASE Tool 29

Slide 30 Developing a Data Administration Strategy Information engineering (IE): Translates strategic goals into data and applications Information systems architecture (ISA): Helps plan, develop, and control future information systems Critical success factors: Management commitment and defined standards Thorough analysis of the company situation End-user involvement Training and a small pilot project 30

Slide 31 DBA’s Role in the Cloud Significant impact on role of DBAs Tasks split between internal DBA and cloud service provider Cloud service partner company provides: DBMS installation and updates Server/network management Backup and recovery operations 31

Slide 32 Oracle Database Administration Tools Ensure the RDBMS starts automatically Create tablespaces and datafiles Tablespace: Logical storage space Datafile: Physically stores the database’s data Manage users and establish security User: Allows a given person to log on to the database Role: Authorizes a user to connect to the database and use its system resources Profile: Controls how much of the database resource a given user can access 32

Slide 33 Types of Tablespace 33

Slide 34

Customize Database Initialization Parameters Fine-tuning a database is an important task that usually requires modification of parameters Initialization parameters reserve resources used by the database at run time After modifying parameters database restart may be required 34

1305627482_531329.pptx Slide 1 Chapter 14 Big Data Analytics and NoSQL

Slide 2 Learning Objectives In this chapter, you will learn: What Big Data is and why it is important in modern business The primary characteristics of Big Data and how these go beyond the traditional “3 Vs” How the core components of the Hadoop framework, HDFs and MapReduce operate What the major components of the Hadoop ecosystems are 2

Slide 3 Learning Objectives In this chapter, you will learn: The four major approaches of the NoSQL data model and how the differ from the relational model About data analytics, including data mining and predictive analytics 3

Slide 4 Big Data Volume: Quantity of data to be stored Scaling up is keeping the same number of systems but migrating each one to a larger system Scaling out means when the workload exceeds server capacity, it is spread out across a number of servers Velocity: Speed at which data is entered into system and must be processed Stream processing focuses on input processing and requires analysis of data stream as it enters the system Feedback loop processing refers to the analysis of data to produce actionable results 4

Slide 5 Figure 14.2 – Current View of Big Data 5

Slide 6 Figure 14.3 – Feedback Loop Processing 6

Slide 7 Big Data

Variety: Variations in the structure of data to be stored Structured data fits into a predefined data model Unstructured data dies not fit into a predefined model Other characteristics: Variability: Changes in meaning of data based on context Sentimental analysis attempts to determine attitude Veracity: Trustworthiness of data Value: Degree data can be analyzed for meaningful insight Visualization: Ability to graphically resent data to make it understandable to users 7

Slide 8 Big Data Characteristics important in working with data in relational models are universal and also apply to Big Data Relational databases not necessarily best for storing and managing all organizational data Polyglot persistence: Coexistence of a variety of data storage and management technologies within an organization’s infrastructure 8

Slide 9 Hadoop De facto standard for most Big Data storage and processing Java-based framework for distributing and processing very large data sets across clusters of computers Most important components: Hadoop Distributed File System (HDFS): Low-level distributed file processing system that can be used directly for data storage MapReduce: Programming model that supports processing large data sets 9

Slide 10 Hadoop Distributed File System (HDFS) Approach based on several key assumptions: High volume - Default block sizes is 64 MB and can be configured to even larger values Write-once, read-many - Model simplifies concurrent issues and improves data throughput Streaming access - Hadoop is optimized for batch processing of entire files as a continuous stream of data Fault tolerance – HDFS is designed to replicate data across many different devices so that when one fails, data is still available from another device 10

Slide 11 Hadoop Distributed File System (HDFS) Uses several types of nodes (computers): Data node store the actual file data Name node contains file system metadata Client node makes requests to the file system as needed to support user applications Data node communicates with name node by regularly sending block reports and heartbeats 11

Slide 12 Figure 14.4 – Hadoop Distributed File System (HDFS)

12

Slide 13 MapReduce Framework used to process large data sets across clusters Breaks down complex tasks into smaller subtasks, performing the subtasks and producing a final result Map function takes a collection of data and sorts and filters it into a set of key-value pairs Mapper program performs the map function Reduce summaries results of map function to produce a single result Reducer program performs the reduce function 13

Slide 14 MapReduce Implementation complements HDFS structure Uses a job tracker or central control program to accept, distribute, monitor and report on jobs in a Hadoop environment Task tracker is a program in MapReduce responsible for reducing tasks on a node System uses batch processing which runs tasks from beginning to end with no user interaction 14

Slide 15 Figure 14.6 – A Sample of the Hadoop Ecosystem 15

Slide 16 Hadoop Ecosystem Map Reduce Simplification Applications: Hive is a data warehousing system that sites on top of HDFS and supports its own SQL-like language Pig compiles a high-level scripting language (Pig Latin) into MapReduce jobs for executing in Hadoop Data Ingestion Applications: Flume is a component for ingesting data in Hadoop Sqoop is a tool for converting data back and forth between a relational database and the HDFS 16

Slide 17 Hadoop Ecosystem Direct Query Applications: HBase is a column-oriented NoSQL database designed to sit on top of the HDFS that quickly processes sparse datasets Impala was the first SQL-on-Hadoop application 17

Slide 18 NoSQL Name given to non-relational database technologies developed to address Big Data challenges Key-value (KV) databases store data as a collection of key-value pairs organized as buckets which

are the equivalent of tables Document databases store data in key-value pairs in which the value components are tag-encoded documents grouped into logical groups called collections 18

Slide 19 Figure 14.7- Key-Value Database Storage 19

Slide 20 Figure 14.8- Document Database Tagged Format 20

Slide 21 NoSQL Column-oriented databases refers to two technologies: Column-centric storage: Data stored in blocks which hold data from a single column across many rows Row-centric storage: Data stored in block which hold data from all columns of a given set of rows Graph databases store data on relationship-rich data as a collection of nodes and edges Properties are the attributes of a node or edge of interest to a user Traversal is a query in a graph database 21

Slide 22 Figure 14.9- Comparison of Row-Centric and Column-Centric Storage 22

Slide 23 Figure 14.10- Graph Database Representation 23

Slide 24 NewSQL Databases Database model that attempts to provide ACID-compliant transactions across a highly distributed infrastructure Latest technologies to appear in the data management area to address Big Data problems No proven track record Have been adopted by relatively few organizations 24

Slide 25 Data Analytics Subset of business intelligence (BI) functionality that encompasses mathematical, statistical, and modeling techniques used to extract knowledge from data Continuous spectrum of knowledge

acquisition that goes from discovery to explanation to prediction Explanatory analytics focuses on discovering and explaining data characteristics based on existing data Predictive analytics focuses on predicting future data outcomes with a high degree of accuracy 25

Slide 26 Data Mining Focuses on the discovery and explanation stages of knowledge acquisition by: Analyzing massive amounts of data to uncover hidden trends, patterns, and relationships Forming computer models to simulate and explain findings and using them to support decision making Can be run in two modes: Guided – End-user decides techniques to apply to data Automated – End-user sets up the tool to run automatically and the data-mining tool applies multiple techniques to find significant relationships 26

Slide 27 Figure 14.12- Extracting Knowledge From Data 27

Slide 28 Figure 14.13- Data-Mining Phases 28

Slide 29 Predictive Analytics Refers to the use of advanced mathematical, statistical, and modeling tools to predict future business outcomes with a high degree of accuracy Focuses on creating actionable models to predict future behaviors and events Most BI vendors are dropping the term data mining and replacing it with predictive analytics Models used in customer service, fraud detection, targeted marketing and optimized pricing Can add value in many different ways but needs to be monitored and evaluated to determine return on investment 29