Business Intelligence and Analytics Systems for Decision Support, 10E Ramesh Sharda Solution Manuals

Business Intelligence and Analytics Systems for Decision Support, 10E By Ramesh

Email: Richard@qwconsultancy.com

Chapter 1: An Overview of Business Intelligence, Analytics, and Decision Support

Learning Objectives for Chapter 1 1. Understand today’s turbulent business environment and describe how organizations survive and even excel in such an environment (solving problems and exploiting opportunities) 2. Understand the need for computerized support of managerial decision making 3. Understand an early framework for managerial decision making 4. Learn the conceptual foundations of the decision support systems (DSS1) methodology 5. Describe the business intelligence (BI) methodology and concepts and relate them to DSS 6. Understand the various types of analytics 7. List the major tools of computerized decision support

CHAPTER OVERVIEW The business environment (climate) is constantly changing, and it is becoming more and more complex. Organizations, private and public, are under pressures that force them to respond quickly to changing conditions and to be innovative in the way they operate.

Such activities require organizations to be agile and to make frequent and quick strategic, tactical, and operational decisions, some of which are very complex. Making such decisions may require considerable amounts of relevant data, information, and knowledge. Processing these, in the framework of the needed decisions, must be done quickly, frequently in real time, and usually requires some computerized support. This book is about using business analytics as computerized support for managerial decision making. It concentrates both on the theoretical and conceptual foundations of decision support, as well as on the commercial tools and techniques that are available. This

1

The acronym DSS is treated as both singular and plural throughout this book. Similarly, other acronyms, such as MIS and GSS, designate both plural and singular forms.

1 .

introductory chapter provides more details of these topics as well as an overview of the book. This chapter has the following sections:

CHAPTER OUTLINE 1.1

OPENING VIGNETTE: MAGPIE SENSING EMPLOYS ANALYTICS TO MANAGE A VACCINE SUPPLY CHAIN EFFECTIVELY AND SAFELY  Questions for the Opening Vignette A. WHAT WE CAN LEARN FROM THIS VIGNETTE

1.2

CHANGING BUSINESS ENVIRONMENTS AND COMPUTERIZED DECISION SUPPORT A. THE BUSINESS PRESSURES–RESPONSES–SUPPORT MODEL 1.

The Business Environment 2.

Organizational Responses: Be Reactive, Anticipative, Adaptive, and Proactive 3.

Closing the Strategy Gap  Section 1.2 Review Questions

1.3

MANAGERIAL DECISION MAKING A. THE NATURE OF MANAGERS’ WORK B. THE DECISION MAKING PROCESS  Section 1.3 Review Questions

1.4

INFORMATION SYSTEMS SUPPORT FOR DECISION MAKING  Section 1.4 Review Questions

1.5 AN EARLY FRAMEWORK FOR COMPUTERIZED DECISION SUPPORT A. THE GORRY AND SCOTT-MORTON CLASSICAL FRAMEWORK 1.

Degree of Structuredness 2.

Types of Control 3.

The Decision Support Matrix B. COMPUTER SUPPORT FOR STRUCTURED DECISIONS C. COMPUTER SUPPORT FOR UNSTRUCTURED DECISIONS D. COMPUTER SUPPORT FOR SEMISTRUCTURED PROBLEMS  Section 1.5 Review Questions 1.6

THE CONCEPT OF DECISION SUPPORT SYSTEMS (DSS) A. DSS AS AN UMBRELLA TERM B. EVOLUTION OF DSS INTO BUSINESS INTELLIGENCE  Section 1.6 Review Questions

1.7

A FRAMEWORK FOR BUSINESS INTELLIGENCE (BI) A. DEFINITIONS OF BI 2 .

B. C. D. E.

F. G.

A BRIEF HISTORY OF BI THE ARCHITECTURE OF BI STYLES OF BI THE ORIGINS AND DRIVERS OF BI ⧫ Application Case 1.1: Sabre Helps Its Clients Through Dashboards and Analytics A MULTIMEDIA EXERCISE IN BUSINESS INTELLIGENCE THE DSS–BI CONNECTION  Section 1.7 Review Questions

1.8

BUSINESS ANALYTICS OVERVIEW A. DESCRIPTIVE ANALYTICS ⧫ Application Case 1.2: Eliminating Inefficiencies at Seattle Children’s Hospital ⧫ Application Case 1.3: Analysis at the Speed of Thought B. PREDICTIVE ANALYTICS ⧫ Application Case 1.4: Moneyball: Analytics in Sports and Movies ⧫ Application Case 1.5: Analyzing Athletic Injuries C. PRESCRIPTIVE ANALYTICS ⧫ Application Case 1.6: Industrial and Commercial Bank of China (ICBC) Employs Models to Reconfigure Its Branch Network D. ANALYTICS APPLIED TO DIFFERENT DOMAINS E. ANALYTICS OR DATA SCIENCE?  Section 1.8 Review Questions

1.9

BRIEF INTRODUCTION TO BIG DATA ANALYTICS A. WHAT IS BIG DATA? ⧫ Application Case 1.7: Gilt Groupe’s Flash Sales Streamlined by Big Data Analytics  Section 1.9 Review Questions

1.10

PLAN OF THE BOOK A. PART I: BUSINESS ANALYTICS: AN OVERVIEW B. PART II: DESCRIPTIVE ANALYTICS C. PART III: PREDICTIVE ANALYTICS D. PART IV: PRESCRIPTIVE ANALYTICS E. PART V: BIG DATA AND FUTURE DIRECTIONS FOR BUSINESS ANALYTICS

1.11

RESOURCES, LINKS, AND THE TERADATA UNIVERSITY NETWORK CONNECTION A. RESOURCES AND LINKS B. VENDORS, PRODUCTS, AND DEMOS C. PERIODICALS 3 .

D. E.

THE TERADATA UNIVERSITY NETWORK CONNECTION THE BOOK’S WEB SITE

Chapter Highlights Key Terms Questions for Discussion Exercises Teradata University Network (TUN) and Other Hands-On Exercises End of Chapter Application Case: Nationwide Insurance Used BI to Enhance Customer Service  Questions for the Case References TEACHING TIPS/ADDITIONAL INFORMATION

       

It is important to stress the frameworks when covering this chapter. The specifics of each subject are less important at this stage, as they will be revisited in more detail in the chapters to come. Going into detail about, say, BI or integration at this stage will mean taking longer on this chapter than the course has time for or skimping on the perspective that students should get from the start.

It can also be useful at this stage to position DSS in the overall context of organizational information systems. They stand above transaction processing systems in the dual sense of typically being used higher up in the organization and having greater impact on it each time they are used. However, they are not independent of other systems: Transaction processing is a major source of data for decision support, in general, and for data warehouses, in particular. External information sources are vital to any DSS. This point is brought out repeatedly throughout the chapter, but it doesn’t always jump out at the typical student reader so you may have to draw attention to it.

Some of the frameworks in the chapter relate more to decision making, while others are more on the information systems side. Specifically, Sections 1.2, 1.3, 1.6, and 1.8 are more about decision making as a managerial activity independent of how it is supported through technology. Sections 1.4 and 1.9 are more on the technology side.

Sections 1.5 and 1.7 have elements of both in different subsections. (Sections 1.1 and 1.10 cannot be positioned along this dimension.) It will help students put the material in perspective if you position the topics for them: “This is process, this is technology, and now we’re back to process.”

You may find some transitions from one section to the next abrupt. This is efficient in terms of content density but may leave students wondering how things are related. For example, Section 1.3 (Managerial Decision Making) jumps right into the importance of decision making as a managerial activity without explaining its importance to the study of DSS. This is obvious, of course, to any instructor who thinks about it: How can one develop systems to support decision making, if one doesn’t understand how decisions are made? It may not be equally obvious to younger students. You will have to make some of the transitions among sections explicit where they are implicit in the book.

You may also be used to a more rigorous distinction between decision making and problem solving than is used here. This book uses the two terms interchangeably. You 4 .

should be aware of their usage from the start to inform how you discuss what the book says.

A secondary purpose of any introductory chapter is to motivate students to be interested in the remainder of the course (and book). The real-life cases, beginning with Magpie Sensing and continuing with the others, will show students that decision support and analytics are not just academic subjects; they are things real companies use that make a noticeable difference to their bottom line. So, try to relate the subject matter to these cases. For example, consider the types of actions managers take to counter pressures (Section 1.2), especially the list of organizational responses. The opening case about Magpie illustrates several of the options available to health care companies, such as innovation, partnerships with others in the cold chain, and the use of IT to improve data access. The other cases in the chapter offer other examples of managerial actions taken in response to pressure. By referring back to this list when discussing other cases, you demonstrate the unity of the analytics field.

All this should show students that a new professional who understands how information systems can support decision making, and can help his or her employer obtain those benefits, has a bright career path. Since students in this course are typically within a year of graduation, that will get their attention!

ANSWERS TO END OF SECTION REVIEW QUESTIONS

     

Section 1.1 Review Questions 1. What information is provided by the descriptive analytics employed at Magpie Sensing?

The descriptive analytics monitor and report the properties of the cold storage system, including the set point of each thermostat, the typical range of temperatures in the system, and the duty cycle of each compressor. These values tell trained personnel whether each storage unit is properly configured for storing particular products.

2. What type of support is provided by the predictive analytics employed at Magpie Sensing?

The predictive analytics use the descriptive data to alert users when a unit is not configured properly for storing the products. It also sends an alert when average temperature and compressor cycle runs signal that a temperature may go out of range (for example, after a power failure) or that there may have been a human error, such as failure to shut a door. These analytics tell users about a problem that may occur, giving them time to prevent the problem.

3. How does prescriptive analytics help in business decision making?

5 .

Prescriptive analytics guides decision makers to the alternative with the greatest benefits. In Magpie’s cold chain system, prescriptive analytics uses data about storage unit performance to help buyers select the best storage units. And based on data about storage system efficiency and product sensitivity, prescriptive analytics guides decisions about where to distribute particular products in the supply chain.

4. In what ways can actionable information be reported in real time to concerned users of the system?

The system uses shippable wireless monitors. These continuously measure temperature, humidity, and location and transmit the data to the computer that analyzes the data.

5. In what other situations might real-time monitoring applications be needed?

Answers will vary, but students should consider other systems where current performance provides information about changes or decisions that would improve future performance. Examples include monitoring inventory levels at a manufacturing company to determine when to replenish stocks, monitoring sales in a store to identify when and how to adjust the mix of products, and monitoring patient status in a hospital to identify situations where different treatment is required or devices are malfunctioning. (If students have difficulty with this question, it suggests a need to devote some class time to discussing practical applications of data and analytics in business, government, and nonprofit sectors.)

Section 1.2 Review Questions 1.

List the components of and explain the Business Pressures–Responses–Support model.

The components of the pressure-response-support model are business pressures, companies’ responses to these pressures, and computerized support. The model suggests that responses are made to counter the pressures or to take advantage of opportunities, support facilitates monitoring the environment (e.g., for opportunities) and enhances the quality of the responses.

2.

What are the major factors in today’s business environment?

The major factors in today’s business environment are: • (Markets) strong competition, expansion of global markets, booming electronic markets on the Internet, innovative marketing methods, opportunities for outsourcing with IT support, and need for real time ondemand transactions

6 .

• • •

3.

(Consumer demand) wanting customization; wanting quality, diversity of products, and speed of delivery; and customers becoming powerful and less loyal (Technology) more innovations, new products and services, increasing obsolescence rate, increasing information overload, social networking, Web 2.0 and beyond (Societal) growth of government regulations and deregulation; work force becoming more diversified, older, and composed of more women; prime concerns of homeland security and terrorist attacks; necessity of compliance with the Sarbanes-Oxley Act and other reporting-related legislation; increasing social responsibility; greater emphasis on sustainability

What are some of the major response activities organizations take?

Responses taken by organizations include, but are not limited to: • Employ strategic planning • Use new and innovative business models • Restructure business processes • Participate in business alliances • Improve corporate information systems • Improve partnership relationships • Encourage innovation and creativity • Improve customer service and relationships • Move to electronic commerce (e-commerce) • Move to make-to-order production and on-demand manufacturing and services • Use new IT to improve communication, data access (discovery of information), and collaboration • Respond quickly to competitors’ actions (e.g., in pricing, promotions, new products and services) • Automate many tasks of white-collar employees • Automate certain decision processes, especially those dealing with customers • Improve decision making by employing analytics (Since the question reads “some,” a correct answer need not list all of these.)

Section 1.3 Review Questions 1.

Describe the three major managerial roles, and list some of the specific activities in each.

Managerial roles fall into the three major categories: • Interpersonal activities. Examples of interpersonal activities include being a figurehead, a leader and a liaison.

7 .

• •

2.

Informational activities. Examples of informational activities include monitoring, disseminating, and acting as a spokesperson.

Decisional activities. Examples of decisional roles include being an entrepreneur, handling disturbances, allocating resources, and negotiating.

Why have some argued that management is the same as decision making?

Some argue that decision making is consistently rated as either the most important or one of the most important components of managing. However, as also seen in this section, others have defined many managerial roles besides decision making.

3.

Describe the four steps managers take in making a decision.

Step 1. Define the problem (i.e., a decision situation that may deal with some difficulty or with an opportunity).

Step 2. Construct a model that describes the real-world problem.

Step 3. Identify possible solutions to the modeled problem and evaluate the solutions.

Step 4. Compare, choose, and recommend a potential solution to the problem.

Section 1.4 Review Questions 1.

What are some of the key system-oriented trends that have fostered IS-supported decision making to a new level?

Important trends include the development of software for problem analysis and solution applications, the availability of networking, especially with Web-based technologies and mobile devices, and the creation of tools for analytics and business intelligence. Social media platforms also have contributed to taking ISsupported decision making to a new level. (Since the question reads “some,” a correct answer should include at least three of these.)

2.

List some capabilities of computing that can facilitate managerial decision making.

These include group communication and collaboration, improved data management, managing giant data warehouses and Big Data, analytical support, overcoming cognitive limits in information processing and storage, knowledge management, and the ability to access information anywhere, anytime. (Since the question reads “some,” a correct answer need not list all of these.)

3.

How can a computer help overcome the cognitive limits of humans?

Computers can store, access, and process vast amounts of information, beyond the limited ability of the human mind to do these.

8 .

Section 1.5 Review Questions 1. What are structured, unstructured, and semistructured decisions? Provide two examples of each.

An unstructured decision is one in which none of the three decision phases (intelligence, design, choice) is structured. Examples: writing a corporate mission statement, selecting a location for a company picnic.

A structured decision is one in which all phases are structured. Examples: finding an appropriate inventory level, choosing an optimal investment strategy.

Semistructured decisions fall between structured and unstructured problems, having some structured elements and some unstructured elements. Examples: trading bonds, setting marketing budgets for consumer products, performing capital acquisition analysis.

(Other correct examples are possible.)

2. Define operational control, managerial control, and strategic planning. Provide two examples of each.

Operational control is the efficient and effective execution of specific tasks.

Examples: scheduling computer storage backups, planning next week’s company cafeteria menu.

Management control is the acquisition and efficient use of resources to accomplish organizational goals. Examples: hiring a production coordinator, planning an advertising program.

Strategic planning is defining long-range goals and policies for resource allocation.

Examples: choosing which of three new products to develop, deciding whether or not to outsource customer telephone support to a region with lower labor costs than where it is now based.

(Other correct examples are possible.)

3.

What are the nine cells of the decision framework? Explain what each is for.

Each of the nine cells represents a combination of one degree of decision structure with one type of management control. Cell 1 represents a structured decision that falls within the span of operational control, and so on.

4.

How can computers provide support for making structured decisions?

The high level of structure that structured problems have makes it possible to abstract them, analyze them, and classify them into specific categories. For each 9 .

category, it is possible to develop a standard solution approach, usually with a standard quantitative model. This model can then be implemented in easy-toapply software.

5.

How can computers provide support to semistructured and unstructured decisions?

Unstructured decisions can be only partially supported by standard computerized quantitative methods. Usually it is necessary to develop a customized solution.

Intuition and judgment play a large role in this type of decision. They may benefit from computerized communication and collaboration technologies and from knowledge management.

Making semistructured decisions may involve a combination of both standard solution procedures and human judgment. Management science can provide models for the portion of the decision-making problem that is structured. For the unstructured portion, a DSS can improve the quality of the information on which the decision is based by providing, for example, not only a single solution but also a range of alternative solutions along with their potential impacts. These capabilities help managers to better understand the nature of problems and thus to make better decisions.

Section 1.6 Review Questions 1.

Provide two definitions of DSS. The two definitions of DSS given in the text are: • Interactive computer-based systems, which help decision makers utilize data and models to solve unstructured problems.

• [They] couple the intellectual resources of individuals with the capabilities of the computer to improve the quality of decisions. It is a computer-based support system for management decision makers who deal with semistructured problems.

(Other correct definitions are possible. Students who provide different definitions, unless they wrote those definitions themselves, should be expected to cite their sources.)

2.

Describe DSS as an umbrella term.

Used in this way, DSS describes any computerized system that supports decision making in an organization. An organization may have (for example) a knowledge management system to guide all its personnel in their problem solving, it may have separate support systems for marketing, finance, and accounting, a supply chain management (SCM) system for production, and several expert systems for product repair diagnostics and help desks. The term DSS encompasses them all.

10 .

Section 1.7 Review Questions 1.

Define BI. Business Intelligence (BI) is an umbrella term that combines architectures, tools, databases, analytical tools, applications, and methodologies. Its major objective is to enable interactive access (sometimes in real time) to data, enable manipulation of these data, and to provide business managers and analysts the ability to conduct appropriate analysis.

2.

List and describe the major components of BI. BI systems have four major components: the data warehouse (with its source data), business analytics (a collection of tools for manipulating, mining, and analyzing the data in the data warehouse), business performance management (for monitoring and analyzing performance), and the user interface (e.g., a dashboard).

3.

What are the major similarities and differences of DSS and BI? • • • • •

BI uses a data warehouse, whereas DSS can use any data source (including a data warehouse).

Most DSS are built to support decision making directly, whereas most BI systems are built to provide information that it is believed will lead to improved decision making.

BI has a strategy/executive orientation whereas DSS are usually oriented toward analysts.

BI systems tend to be developed with commercially available tools, whereas DSS tend to use more custom programming to deal with problems that may be unstructured.

DSS methodologies and tools originated largely in academia, whereas BI arose largely from the software industry. Many BI tools, such as data mining and predictive analysis, have come to be considered DSS tools as well.

Section 1.8 Review Questions 1.

Define analytics.

The term replaces terminology referring to individual components of a decision support system with one broad word referring to business intelligence. More precisely, analytics is the process of developing actionable decisions or recommendations for actions based upon insights generated from historical data.

Students may also refer to the eight levels of analytics and this simpler descriptive language: “looking at all the data to understand what is happening, what will happen, and how to make the best of it.”

11 .

2.

What is descriptive analytics? What various tools are employed in descriptive analytics?

Descriptive analytics refers to knowing what is happening in the organization and understanding some underlying trends and causes of such occurrences. Tools used in descriptive analytics include data warehouses and visualization applications.

3.

How is descriptive analytics different from traditional reporting?

Descriptive analytics gathers more data, often automatically. It makes results available in real time and allows reports to be customized.

4.

What is a data warehouse? How can data warehousing technology help in enabling analytics?

A data warehouse, introduced in Section 1.7, is the component of a BI system that contains the source data. As described in this section, developing a data warehouse usually includes development of the data infrastructure for descriptive analytics—that is, consolidation of data sources and making relevant data available in a form that enables appropriate reporting and analysis. A data warehouse serves as the basis for developing appropriate reports, queries, alerts, and trends.

5.

What is predictive analytics? How can organizations employ predictive analytics?

Predictive analytics is the use of statistical techniques and data mining to determine what is likely to happen in the future. Businesses use predictive analytics to forecast whether customers are likely to switch to a competitor, what customers are likely to buy, how likely customers are to respond to a promotion, and whether a customer is creditworthy. Sports teams have used predictive analytics to identify the players most likely to contribute to a team’s success.

6.

What is prescriptive analytics? What kind of problems can be solved by prescriptive analytics?

Prescriptive analytics is a set of techniques that use descriptive data and forecasts to identify the decisions most likely to result in the best performance. Usually, an organization uses prescriptive analytics to identify the decisions or actions that will optimize the performance of a system. Organizations have used prescriptive analytics to set prices, create production plans, and identify the best locations for facilities such as bank branches.

7.

Define modeling from the analytics perspective.

As Application Case 1.6 illustrates, analytics uses descriptive data to create models of how people, equipment, or other variables operate in the real world.

12 .

These models can be used in predictive and prescriptive analytics to develop forecasts, recommendations, and decisions.

8.

Is it a good idea to follow a hierarchy of descriptive and predictive analytics before applying prescriptive analytics?

As noted in the analysis of Application Case 1.5, it is important in any analytics project to understand the business domain and current state of the business problem. This requires analysis of historical data, or descriptive analytics.

Although the chapter does not discuss a hierarchy of analytics, students may observe that testing a model with predictive analytics could logically improve prescriptive use of the model.

9.

How can analytics aid in objective decision making?

As noted in the analysis of Application Case 1.4, problem solving in organizations has tended to be subjective, and decision makers tend to rely on familiar processes. The result is that future decisions are no better than past decisions.

Analytics builds on historical data and takes into account changing conditions to arrive at fact-based solutions that decision makers might not have considered.

Section 1.9 Review Questions 1.

What is Big Data analytics?

The term Big Data refers to data that cannot be stored in a single storage unit.

Typically, the data is arriving in many different forms, be they structured, unstructured, or in a stream. Big Data analytics is analytics on a large enough scale, with fast enough processing, to handle this kind of data.

2.

What are the sources of Big Data?

Major sources include clickstreams from Web sites, postings on social media, and data from traffic, sensors, and the weather.

3.

What are the characteristics of Big Data?

Today Big Data refers to almost any kind of large data that has the characteristics of volume, velocity, and variety. Examples include data about Web searches, such as the billions of Web pages searched by Google; data about financial trading, which operates in the order of microseconds; and data about consumer opinions measured from postings in social media.

4.

What processing technique is applied to process Big Data?

One computer, even a powerful one, could not handle the scale of Big Data. The solution is to push computation to the data, using the MapReduce programming paradigm.

13 .

Section 1.10 Review Questions (This section has no review questions.)

Section 1.11 Review Questions (This section has no review questions.)

ANSWERS TO APPLICATION CASE QUESTIONS FOR DISCUSSION

 

Application Case 1.1: Sabre Helps Its Clients Through Dashboards and Analytics 1.

What is traditional reporting? How is it used in organizations?

The traditional reporting process is a manual process of collecting and aggregating financial and other information. Organizations have used this timeconsuming process as a way to obtain information for making decisions.

However, the resulting presentations may be flat, slow to develop, and difficult to apply to specific situations.

2.

How can analytics be used to transform traditional reporting?

Analytics can enable real-time decision support and deliver information to a userfriendly dashboard. Users of a dashboard such as the one provided by Sabre’s Enterprise Travel Data Warehouse can see at a glance a 360-degree view of the company’s overall health generated from various data sources. Many stakeholders in the organization can request data needed for particular types of decisions, and the graphical user interface makes the information easily understandable.

3.

How can interactive reporting assist organizations in decision making?

When a system incorporates interactive drill-down capabilities, users can select the data they need for evaluating a specific kind of performance and making decisions in a particular function or situation. For example, the airlines using the Sabre system can focus on data about sales performance (ticketing, seats sold, etc.) and operational performance (flight movement, inventory, etc.). This flexibility encourages decision makers to use data in support of their decisions.

Application Case 1.2: Eliminating Inefficiencies at Seattle Children’s Hospital

14 .

1.

Who are the users of the tool?

The business intelligence application at Seattle Children’s Hospital is used by data analysts, business managers, financial analysts, clinicians, doctors, and researchers.

2.

What is a dashboard?

A dashboard is a user interface that displays information in some type of visual system. For example, it shows current performance measured against standards.

3.

How does visualization help in decision making?

Visualizations help users gain insights into existing data. Exploring data visually can help users identify the root causes of problems and provide a basis for working toward solutions.

4.

What are the significant results achieved by the use of Tableau?

Tableau’s business intelligence application lets users create monthly and daily dashboards that have helped users significantly improve day-to-day decision making. Visualizing data about patient wait-times helped the hospital improve overall waiting time and improve the availability of beds. The hospital also saved about $3 million from the supply chain and has made processes more efficient.

Application Case 1.3: Analysis at the Speed of Thought 1.

What are the desired functionalities of a reporting tool?

A reporting tool should generate reports that make visualizations easily discernible. It should help users make sense out of the reports, identify problem areas, and contribute to improvements. It also should be able to handle the volume of data required and prepare reports within the necessary time frame.

2.

What advantages were derived by using a reporting tool in this case?

At Kaleida Health, Tableau’s reporting tool handles more data faster than its traditional reporting tools did. Also, the analytic capability is deeper. The reports provide detailed data for analyzing efficiency and comparing the performance of Kaleida’s hospitals with other hospitals nationwide.

Application Case 1.4: Moneyball: Analytics in Sports and Movies

15 .

1.

How is predictive analytics applied in Moneyball?

The Oakland Athletics hired an assistant general manager with a background in economics and computer expertise. He followed a Sabermetric approach, which uses baseball statistics to analyze the game. He built a prediction model that selected players based on their on-base percentage, which measures how often a batter reaches base for any reason other than fielding error, fielder’s choice, dropped or uncaught third strike, fielder’s obstruction, or catcher’s interference.

This strategy built a team that won 20 consecutive games and set a league record.

2.

What is the difference between objective and subjective approaches in decision making?

Subjective decision making relies on experience and intuition; in this case, the team’s scouts applied their experience to select players. An objective approach identifies measures associated with success to create a predictive model for decision making based on data.

Application Case 1.5: Analyzing Athletic Injuries 1.

What types of analytics are applied in the injury analysis?

The analytics used data about the type of injury, action taken, healing start and end dates, players’ position, activity, onset, and game location. The data were used to classify healing time into five periods and to associate healing time with players’ positions, severity of injury, and treatment offered. That provided information for creating neural network models using player and injury data to predict healing time in terms of the five categories.

2.

How do visualizations aid in understanding the data and delivering insights into the data?

Visualizations provide a great tool for gaining the initial insights into data, which can be further refined based on expert opinions. Visualizations also aid in generating ideas for obscured business problems, which can be pursued in building predictive models.

3.

What is a classification problem?

Classification is a technique used in developing predictive analytical applications.

In this example, various kinds of data, such as severity of injury and healing time, were classified for purposes of making decisions about how to handle injuries and which players might be available to play in the future.

16 .

4.

What can be derived by performing sequence analysis?

Drawing sequence rules can predict the relationship among types of data—in this case, the relationship among the injuries and the various body parts afflicted with injuries.

Application Case 1.6: Industrial and Commercial Bank of China (ICBC) Employs Models to Reconfigure Its Branch Network 1.

How can analytical techniques help organizations to retain competitive advantage?

In this case, analytical techniques use geographic and customer behavioral segmentation to isolate the profitability of customers and employ optimization techniques to locate the branches that deliver high profitability in each geographic segment. Encourage students to apply this example to the basics of competition in business: Organizations that know their customers better can better target their goods and services. They can operate more efficiently. They can sell the most profitable products to the most profitable customers. In these ways, they can outperform the competition.

2.

How can descriptive and predictive analytics help in pursuing prescriptive analytics?

Descriptive analytics tell the organization what is happening, and predictive analytics tell what is likely to happen. When an organization has a good view of what is happening and what is likely to happen, it can also employ other techniques—prescriptive analytics—to make the best decisions under the circumstances. According to one view, these types of analytics are somewhat independent steps, with one type of analytics leading to another. In another view, the types of analytics overlap somewhat.

3.

What kinds of prescriptive analytic techniques are employed in the case study?

At ICBC, a branch reconfiguration system uses a market potential calculation model, a branch network optimization model, and a branch site evaluation model.

The branch network optimization model uses mixed integer programming to locate branches where they cover the areas with the largest market potential.

Using the model’s recommendations has increased deposits in the branches by $21.2 billion.

4.

Are the prescriptive models once built good forever?

No, because conditions change, and new knowledge can improve a model. In the ICBC example, the branch reconfiguration tool has been improved through an iterative process.

17 .

Application Case 1.7: Gilt Groupe’s Flash Sales Streamlined by Big Data Analytics 1.

What makes this case study an example of Big Data analytics?

Big Data is data that cannot be stored in a single storage unit and typically is arriving in many different forms or in a stream. Although the case does not describe the volume of data that Gilt Groupe is analyzing, it does say the company is receiving a stream of data because it is measuring every click on the company’s website, as well as references to Gilt on Twitter. Also, the software the company uses is based on Teradata Aster’s technology solution that includes Big Data analytics technologies.

2.

What types of decisions does Gilt Groupe have to make?

Gilt Groupe must decide what to order—both the selection of brands and styles and the quantity of each item—to minimize inventory costs. It makes personalized recommendations, so it needs to decide what items to recommend to each customer. It needs to decide on the timing of e-mails that communicate the offers.

ANSWERS TO END OF CHAPTER QUESTIONS FOR DISCUSSION

  

1. Give examples for the content of each cell in Figure 1.2.

• • • • • • • • •

Structured/Operational: Ordering materials to maintain an appropriate inventory level Semistructured/Operational: Bond trading Unstructured/Operational: Selecting a location for the company picnic Structured/Managerial: Selecting the optimum mix of financial investments Semistructured/Managerial: Planning a consumer marketing campaign Unstructured/Managerial: How should offices be arranged in a new building?

Structured/Strategic: Deciding how to finance a take-over offer Semistructured/Strategic: Which applicant should be chosen as the new CFO? Unstructured/Strategic: Writing a corporate mission statement (Other correct examples are possible.)

2. Survey the literature from the past six months to find one application each of DSS, BI, and analytics. Summarize the applications on one page and submit it with the exact sources.

18 .

There is no single correct answer to this question. Answers will depend on when the question is posed and on the student’s online or library search strategy. The correctness of an answer can be evaluated by examining the sources cited in it or submitted with it.

3. Observe an organization with which you are familiar. List three decisions it makes in each of the following categories: strategic planning, management control (tactical planning), and operational planning and control.

There is no single correct answer to this question. Answers will depend on the organization that the student observes. The correctness of an answer can be evaluated by comparing characteristics of the nine decisions with the criteria in Section 1.5 for the levels in which they are placed.

4. Distinguish BI from DSS. • • • • •

BI uses a data warehouse, whereas DSS can use any data source (including a data warehouse).

Most DSS are built to support decision making directly, whereas most BI systems are built to provide information which it is believed will lead to improved decision making.

BI has a strategy/executive orientation whereas DSS are usually oriented toward analysts.

BI systems tend to be developed with commercially available tools, whereas DSS tend to use more custom programming to deal with problems that may be unstructured.

DSS methodologies and tools originated largely in academia, whereas BI arose largely from the software industry. Many BI tools, such as data mining and predictive analysis, have come to be considered DSS tools as well.

5. Compare and contrast predictive analytics with prescriptive and descriptive analytics.

Use examples.

Predictive analytics is the use of statistical techniques and data mining to determine what is likely to happen in the future. For example, an airline might use predictive analytics to forecast the impact on sales and profits if it raises baggage fees by $10. It applies information from descriptive analytics, applying historical or real-time data to know what is happening in the organization and understand some underlying trends and causes of such occurrences. In the airline example, descriptive analytics would include data about ticket prices, baggage fees, ticket sales, baggage volume, and so on, applied to find relationships among these variables. Predictive analytics may be applied to prescriptive analytics, which is a set of techniques that use descriptive data and forecasts to identify the decisions most likely to result in the best performance.

For example, predictive analytics could forecast the impact on profits of different 19 .

baggage fees. It might show, for example, that raising baggage fees by $5 will lead to the greatest profits after the airline takes into account fee revenues, ticket sales, the amount of baggage carried, and the cost to transport the baggage. (Students may use different examples, as long as they illustrate the definitions.)

ANSWERS TO END OF CHAPTER APPLICATION CASE QUESTIONS 1.

 

Why did Nationwide need an enterprise-wide data warehouse?

With more than 100 business units offering a variety of products, Nationwide experienced duplication of effort in gathering data, analyzing it, and generating reports. Data-processing environments were widely dissimilar, and there was extreme data redundancy, resulting in higher expenses. Mergers and acquisitions only added to the difficulty and cost. A single, authoritative data warehouse would apply best practices and provide clean, consistent, and complete data.

2.

How did integrated data drive the business value?

Integrated data about customers improved marketing campaigns and better targeted communications, which improved customer satisfaction and retention, as well as contributing to a gain in sales. Integrated financial data made financial reporting faster and more efficient and added tools for better risk assessment and decision support. Integrated data following mergers fostered smoother integration of businesses. Integrated data for reporting gave agents easy access to reports within seconds rather than days, significantly improving productivity.

3.

What forms of analytics are employed at Nationwide?

It uses the three basic types of business analytics. The company’s data warehouse supports descriptive analytics for all functions. For example, the data describes customer behavior, financial performance, and policy information. It uses predictive analytics in the Customer Knowledge Store to identify the kinds of customer interaction that are important for customers at different points in their lives. At the level of prescriptive analytics, the Financial Performance Management system applies financial data to a decision support system. (Students may identify additional examples of these three categories.) Optionally, students may also refer to analytics applied to different domains, such as marketing analytics, financial analytics, and even insurance analytics.

4.

With integrated data available in an enterprise data warehouse, what other applications could Nationwide potentially develop?

This would be a good question for class discussion, as answers will vary, and students can apply any work experience they have, including internships, as well as the cases in this chapter, to identify areas of business where analytics adds 20 .

value. The case described customer relationships and financial reporting. Other areas of decision making for an insurance business would include the development and pricing of products, regulatory compliance, hiring, risk management, and the location of facilities. Accept any reasonable answers.

21 .

Chapter 2: Foundations and Technologies for Decision Making

Learning Objectives for Chapter 2 1. Understand the conceptual foundations of decision making 2. Understand Simon’s four phases of decision making: intelligence, design, choice, and implementation 3. Understand the essential definition of DSS 4. Understand important DSS classifications 5. Learn how DSS support for decision making can be provided in practice 6. Understand DSS components and how they integrate CHAPTER OVERVIEW Our major focus in this book is the support of decision making through computer-based information systems. The purpose of this chapter is to describe the conceptual foundations of decision making and how decision support is provided. This chapter includes the following sections: CHAPTER OUTLINE 2.1

OPENING VIGNETTE: DECISION MODELING AT HP USING SPREADSHEETS  Questions for the Opening Vignette A. WHAT WE CAN LEARN FROM THIS VIGNETTE

2.2

DECISION MAKING: INTRODUCTION AND DEFINITIONS A. CHARACTERISTICS OF DECISION MAKING B. A WORKING DEFINITION OF DECISION MAKING C. DECISION-MAKING DISCIPLINES D. DECISION STYLE AND DECISION MAKERS 1.

Decision Style 2.

Decision Makers 1 .



Section 2.2 Review Questions

2.3

PHASES OF THE DECISION-MAKING PROCESS  Section 2.3 Review Questions

2.4

DECISION MAKING: THE INTELLIGENCE PHASE A. PROBLEM (OR OPPORTUNITY) IDENTIFICATION ⧫ Application Case 2.1: Making Elevators Go Faster!

B. PROBLEM CLASSIFICATION C. PROBLEM DECOMPOSITION D. PROBLEM OWNERSHIP  Section 2.4 Review Questions

2.5

DECISION MAKING: THE DESIGN PHASE A. MODELS B. MATHEMATICAL (QUANTITATIVE) MODELS C. THE BENEFITS OF MODELS D. SELECTION OF A PRINCIPLE OF CHOICE ⧫ Technology Insights 2.1: The Difference Between a Criterion and a Constraint E. NORMATIVE MODELS F. SUBOPTIMIZATION G. DESCRIPTIVE MODELS H. GOOD ENOUGH OR SATISFICING I. DEVELOPING (GENERATING) ALTERNATIVES J. MEASURING OUTCOMES K. RISK L. SCENARIOS M. POSSIBLE SCENARIOS N. ERRORS IN DECISION MAKING  Section 2.5 Review Questions

2.6

DECISION MAKING: THE CHOICE PHASE  Section 2.6 Review Questions

2.7

DECISION MAKING: THE IMPLEMENTATION PHASE  Section 2.7 Review Questions

2.8

HOW DECISIONS ARE SUPPORTED A. SUPPORT FOR THE INTELLIGENCE PHASE B. SUPPORT FOR THE DESIGN PHASE C. SUPPORT FOR THE CHOICE PHASE D. SUPPORT FOR THE IMPLEMENTATION PHASE  Section 2.8 Review Questions

2.9

DECISION SUPPORT SYSTEMS: CAPABILITIES 2 .

A.

A DSS APPLICATION  Section 2.9 Review Questions

2.10

DSS CLASSIFICATIONS A. THE AIS SIGDSS CLASSIFICATION FOR DSS 1.

Communications-driven and Group DSS 2.

Data-driven DSS 3.

Document-driven DSS 4.

Knowledge-driven DSS, Data Mining, and Management Expert Systems Applications 5.

Model-driven DSS 6.

Compound DSS B. OTHER DSS CATEGORIES 1.

Institutional and Ad Hoc DSS C. CUSTOM-MADE SYSTEMS VERSUS READY-MADE SYSTEMS  Section 2.10 Review Questions

2.11

COMPONENTS OF DECISION SUPPORT SYSTEMS A. THE DATA MANAGEMENT SUBSYSTEM ⧫ Application Case 2.2: Station Casinos Wins by Building Customer Relationships Using Its Data B. THE MODEL MANAGEMENT SUBSYSTEM ⧫ Application Case 2.3: SNAP DSS Helps OneNet Make Telecommunications Rate Decisions C. THE USER INTERFACE SUBSYSTEM D. THE KNOWLEDGE-BASED MANAGEMENT SUBSYSTEM ⧫ Application Case 2.4: From a Game Winner to a Doctor! ⧫ Technology Insights 2.2: Next Generation of Input Devices

Chapter Highlights Key Terms Questions for Discussion Exercises Teradata University Network (TUN) and Other Hands-On Exercises End of Chapter Application Case: Logistics Optimization in a Major Shipping Company (CSAV)  Questions for the Case References

TEACHING TIPS/ADDITIONAL INFORMATION

       

This chapter has two major themes: (a) how decisions are made, and (b) how the decision-making process (and hence the people who make them) can be supported.

Decision making is the subject of Section 2.2 through 2.7. Support is covered in Sections 2.8 through 2.11. (Section 2.1, the introductory vignette, applies equally to 3 .

both.) So, the text covers decision making first, and follows it with decision support.

Some instructors may prefer to weave in topics of support with topics of decision making, especially when covering the intelligence, design, and choice phases.

It is important to stress the relevance of decision-making methods to DSS/BI in covering this chapter. The key reason is in the word “support” in the term DSS. We are discussing systems that support people who make decisions, not systems that make decisions on their own. People who make business decisions are often high enough in the organization to have choices as to how they make their decisions, so it is important to support decision-making methods and styles that they are willing to use.

One way to view this area is to consider the decision-making system as a whole as consisting of human and automated subsystems. Inputs to the overall system are external and internal (as seen by the organization) data sources and decision requirements. Output is a decision. Internally, the two subsystems communicate in a manner determined by the designers and developers of the automated subsystem. If this is not a suitable interface for the human subsystem, the overall decision-making system will not work well. DSS/BI system designers must see themselves as subsystem designers, where they have limited control over the other major subsystem in the system they are working on, and where they must not suboptimize the automated subsystem at the expense of the overall system.

(Suboptimization is covered in this chapter too, in Section 2.5.)

Understanding the phases of decision making is important in developing automated support, as the kind of support needed depends on the decision phase.

Teachers should recognize that this subject is taught in several places in the typical business school curriculum, not always from the same point of view. Some instructors draw a strong distinction between decision making and problem solving, whereas this book considers them nearly equivalent. Some instructors consider monitoring to be a fifth phase of the process, whereas this book considers it as the intelligence phase applied to the implementation phase. It is not necessary to be dogmatic about one version of the phases versus another. It can help, however, to determine where else these concepts are taught at your institution and how they are approached there. You may also want to relate Simon’s phases with the typical steps of the SDLC process of systems development.

In Section 2.10, this book groups simulation models and other types of mathematical models, such as linear programming, as “mathematical (quantitative) models.” Students may have been taught a distinction between the two in other courses such as operations management. You may want to recognize this as being beyond the level of breakdown needed for the purpose of this chapter, but at the same time as valid when a finer classification of models is required.

Section 2.8, “How Decisions Are Supported,” gives a wide range of support possibilities for each phase. In teaching this section, which is critical to students’ careers because it tells them what to look for in specific situations, it can help to add perspective to the lists in each subsection by indicating which of the listed tools are more important in each phase, which are less so. For example, expert systems are listed as being able to support all four of the decision phases. Although this is correct, you can add perspective by pointing out (in this case) that they are most useful in the choice phase, secondarily in design and implementation, and relatively less useful in the intelligence stage of a decision.

4 .

Finally, students may have to be reminded (or told for the first time!) that “criterion” is a singular noun whose plural is “criteria.” Saying “The single most important decision criteria is …” is incorrect. Since managers may obtain subconscious clues to a junior staff member’s or job applicant’s competence from his or her ability to use business terms correctly, it’s important for them to learn correct usage when this term comes up in Section 2.5.

ANSWERS TO END OF SECTION REVIEW QUESTIONS

     

Section 2.1 Review Questions 1.

What are some of the key questions to be asked in supporting decision making through DSS? • Will analytics solve the problem?

• Can an existing tool be leveraged?

• Is a tool needed?

2.

What guidelines can be learned from this vignette about developing DSS? • Before building a model, decision makers should develop a good understanding

of the problem that needs to be addressed.

• Coming up with nonmodeling solutions is important because if the problem is due to

conflicting priorities, or the misalignment of incentives or unclear lines of authority or plans, then no DSS can help support the decision.

• A model many not be necessary to address the problem.

• Before developing a new tool, decision makers should explore reuse of existing

tools.

• The goal of model building is to gain better insight into the problem, not just to

generate more numbers.

3.

What lessons should be kept in mind for successful model implementation?

• Implementation plans should be developed along with the model. Successful

implementation results in solving the real problem.

• Including the end users in the development process enhances the decision

makers’ analytical knowledge and capabilities. And by working together, their knowledge and skills complement each other in the final solution and the success of the implementation.

5 .

Section 2.2 Review Questions 1.

What are the various aspects of decision making?

Aspects of decision making that are important to understand if we are to develop effective computer support include the following: • characteristics of decision making, such as groupthink, experimentation, and information overload • decision styles of the decision makers • objectives of the decision makers • supporting disciplines, styles, and how they relate to the personal characteristics of the decision maker, and the nature of group involvement in the decision (if any) • rationality of the decision maker. A decision maker should not simply apply IT tools blindly. Rather, the decision maker gets support through a rational approach that simplifies reality and provides a relatively quick and inexpensive means of considering various alternative courses of action to arrive at the best or a good solution to the problem.

2.

Identify similarities and differences between individual and group decision making.

• • •

•

Individual decision makers need access to data and to experts who can provide advice, while groups additionally need collaboration tools.

There are often conflicting objectives in a group decision-making setting, but not in an individual setting.

Groups can be of variable size and may include people from different departments or from different organizations. Collaborating individuals may therefore have different cognitive styles, personality types, and decision styles. Some clash, whereas others are mutually enhancing.

Consensus can be a difficult political problem in group decision making which is not a problem in individual decision making.

For these and similar reasons, group decision making can be more complicated than individual decision making.

3.

Define decision style and describe why it is important to consider in the decisionmaking process.

Decision style is the manner in which a decision maker thinks and reacts to problems. It is important to consider it because different decision styles require different types of support.

6 .

4.

What are the benefits of mathematical models?

They help with many problems faced by decision makers. Mathematical models can be used for what-if analysis, group decision-making, and quickly collecting and analyzing large data sets. Such models enhance the effectiveness of decision making with imperfect and incomplete information. Different models can be applied to different decision styles.

Section 2.3 Review Questions 1.

List and briefly describe Simon’s four phases of decision making.

Simon’s four phases of decision making are intelligence, design, choice, and implementation.

Intelligence consists of gathering information by examining reality, then identifying and defining the problem. In this phase problem ownership should also be established.

Design consists of determining alternatives and evaluating them. If the evaluation will require construction of a model, that is done in this phase as well.

The choice phase consists of selecting a tentative solution and testing its validity.

Implementation of the decision consists of putting the selected solution into effect.

See Figure 2.1.

2.

What are the impacts of the Web on the phases of decision making?

Simon’s phases of decision making (intelligence, design, choice, and implementation) apply to many areas, including Web application development.

The Web application development framework involves some sort of system lifecycle, which includes all these phases. Of course, the Web provides a wealth of data (mostly unstructured) that can go into the intelligence phase of any decision process. Vendors and service providers operating on the Web also provide some tools that help with design, choice, and implementation tasks.

Section 2.4 Review Questions 1.

What is the difference between a problem and its symptoms?

7 .

Problems arise out of dissatisfaction with the way things are going. It is the result of a difference or gap between what we desire and what is or is not happening. A symptom is how a problem manifests itself.

A familiar personal example is a high temperature (symptom) and an illness (problem). It is necessary to diagnose and treat the underlying illness. Attempting to relieve the temperature works if the illness is one which the body’s defenses can cure, but, can be disastrous in other situations.

A business example: high prices (problem) and high unsold inventory level (symptom). Another is quality variance in a product (symptom) and poorly calibrated or worn-out manufacturing equipment (problem).

2.

Why is it important to classify a problem?

Classifying a problem enables decision makers to use tools that have been developed to deal with problems in that category, perhaps even including a standard solution approach.

3.

What is meant by problem decomposition?

Problem decomposition means dividing a complex problem into subproblems.

This helps solve the problem because the smaller subproblems may be simpler to solve; some of the smaller problems may be structured, facilitating their solution, even though the overall problem as a whole is not; solving the easy subproblems enables decision makers to focus their attention on the remaining difficult parts; and decomposition may facilitate communication among decision makers.

4.

Why is establishing problem ownership so important in the decision-making process?

Problem ownership means having the authority, and taking the responsibility, of solving it. Lack of problem ownership means either that someone is not doing his or her job, or that the problem at hand has yet to be identified as belonging to anyone. In either case, it cannot be solved until someone owns it.

Section 2.5 Review Questions 1.

Define optimization and contrast it with suboptimization.

Optimization refers to the “best.” (There is no such thing as “more” optimal!) To achieve it, all alternatives must be considered, and the optimal one must be the best. Suboptimization is the optimization of a subsystem, without considering its

8 .

impacts on other parts of the overall system. What is optimal for a part of a system (or organization) may not be for the entire system (or organization).

For example, a student spending all 24 final exam study hours on DSS may give him/her the best possible grade in that course, but his/her overall average could be better served by studying six hours for each of four exams. The drop from A+ to B in the DSS course would be more than offset by the improvement from Ds to Bs and Cs in the other three.

2.

Compare the normative and descriptive approaches to decision making.

Normative decision making uses models, or methods that have perhaps previously been derived from models that tell a decision maker what he or she should do.

These prescriptive models are often developed by utilizing optimization methods.

The technology of expert systems, which will be discussed later in the book but which students may have seen elsewhere, derives them from the “rules of thumb” used by recognized experts in the field of the decision.

Descriptive decision making uses models that tell a decision maker “what-if.”

These are usually simulation models.

3.

Define rational decision making. What does it really mean to be a rational decision maker?

Rational decision making follows the economic assumptions of rationality. A rational decision maker exhibits certain assumed behaviors: (1) Humans are economic beings, whose objective is to maximize the attainment of goals; (2) for a decision-making situation, all viable alternative courses of action and their consequences, or at least the probability and the values of the consequences are known; and (3) decision makers have an order or preference that enables them to rank the desirability of all consequences of the analysis (best to worst).

Being a rational decision maker means making decisions according to these assumptions.

4.

Why do people exhibit bounded rationality when problem solving?

Humans in general have limitations that prevent us from being completely rational. We usually simplify things. Individuals’ evaluation scales for the costs and benefits of a decision may be nonlinear and may not follow those of the organization. (For example, a manager may see great personal benefit in not exceeding a budget by even $1, but very little benefit by coming under it by more than a minimal amount. The view from the executive suite would not have such a sharp cut-off. Economic utility theory addresses this area.) Also, individual characteristics may result in a restricted rationality.

9 .

5.

Define scenario. How is a scenario used in decision making?

A scenario is a statement of assumptions about the operating environment of a particular system in a given time. It describes the system's configurations. By changing scenarios and measuring the goal attainment level, it is possible to compare alternatives under different sets of conditions.

6.

Some “errors” in decision making can be attributed to the notion of decision making from the gut. Explain what is meant by this and how such errors can happen.

In general, people have a tendency to measure uncertainty and risk badly. They tend to be overconfident and have an illusion of control in decision making. As a result, decisions in which some part of the future is unknown, which is true of most business decisions, are often made more optimistically than they should be.

In addition, those who decide “from the gut” often do not develop a clear picture of all the details and implications of a situation. This approach can save time if a decision maker’s intuition is attuned to a situation from extensive experience, but can lead to errors if it is applied to unfamiliar situations. An executive who has made successful “gut” decisions in the past may not even recognize that his or her experience does not apply to a new, but superficially similar, situation.

Section 2.6 Review Questions 1.

Explain the difference between a principle of choice and the actual choice phase of decision making.

A principle of choice is a criterion used to describe the acceptability of a solution approach. In other words, it is a basis for deciding whether one approach or another is superior. A principle of choice is general: that is, it applies to many possible decision-making situations.

The choice phase of decision making uses one or more principles of choice, chosen during this decision phase or prior to it, to select an alternative in a specific situation.

2.

Why do some people claim that the choice phase is the point in time when a decision is really made?

Because, in a sense, it is. The decision, choosing one of the available alternatives, is made during this stage. It is, therefore, easy to equate the two.

However, the choice phase as the term is usually used covers more than this single point in time. It also includes the comparisons that lead up to it and the 10 .

assessment of robustness and possible adverse consequences that may lead a decision maker to choose an alternative that is less desirable under ideal conditions but also less likely to lead to disaster in other circumstances.

3.

How can sensitivity analysis help in the choice phase?

Sensitivity analysis determines how an alternative responds to small changes in the input parameters. An alternative that appears best for the nominal set of parameters might produce far worse results for small changes in them. Another alternative, which might not be as attractive if all parameters have their nominal values, might be more robust—that is, it might not degrade as much if they depart from those values. It might therefore be a better choice in practice, since some statistical variation in these values is only to be expected.

Section 2.7 Review Questions 1.

Define implementation.

Implementation is defined as the initiation of a new order of things, the introduction of change; putting a recommended solution to work.

2.

How can a DSS support the implementation of a decision?

A DSS supports the implementation of a decision through communication, explanation, and justification. In a financial decision, for example, a DSS would include not only the detailed financial goals and cash needs for the near term, but would also provide the calculations, intermediate results, and statistics used to determine the aggregate figures. It also conveys to subordinates that the decision maker has thought through the assumptions behind as decision’s goals and is serious about their importance. Finally, it allows people to explain and justify their suggestions and opinions with graphical support.

Section 2.8 Review Questions 1.

Describe how DSS/BI technologies and tools can aid in each phase of decision making.

Intelligence phase: The primary requirement of decision support for the intelligence phase is the ability to scan external and internal information sources for opportunities and problems and to interpret what the scanning discovers. Web tools and sources are extremely useful for environmental scanning.

Decision support/business intelligence technologies can also help. (Automatic) data mining and (manual) online analytic processing support this phase by 11 .

identifying relationships among activities and other factors. Geographic information systems (GIS) can be utilized either as stand alone or integrated with these systems, so that a decision maker can determine opportunities and problems in a spatial sense.

Another aspect of identifying internal problems and opportunities is monitoring operations. Business activity monitoring, business process management, and product life-cycle management provide such capability. Routine and ad-hoc reports can also help: regular reports can be designed to assist in problem finding by comparing expectations with current and projected performance.

Design phase: This phase involves generating alternative courses of action, agreeing on choice criteria and their weights, and forecasting the consequences of various alternatives. Several of these activities can use standard models such as financial and forecasting models. Either standard or special models can generate alternatives for structured problems. OLAP and data mining software are useful in identifying relationships to use in such models. An expert system can assist with qualitative methods as well as with the expertise required in selecting quantitative analysis and forecasting models.

A knowledge management system, if available, can be consulted to determine whether such a problem has been encountered before, or if there are experts on hand to provide quick understanding and answers. CRM systems, revenue management systems, ERP, and SCM software are useful in providing models of business processes that can test assumptions and scenarios. If a problem requires brainstorming to help identify important issues and options, a group DSS may prove helpful.

Choice phase: In addition to providing models that rapidly identify a best or good-enough alternative, a DSS can support the choice phase through what-if and goal-seeking analyses. Different scenarios can be tested for the selected option to reinforce the final decision. A knowledge management system helps identify similar past experiences; CRM, ERP, and SCM systems can test the impact of each choice. If a group makes the decision, a group support system can provide support to lead to consensus.

Implementation phase: DSS can be used in implementation activities such as decision communication, explanation, and justification. Implementation phase DSS benefits are partly due to the vividness and detail of analyses and reports used for these purposes.

All phases of the decision-making process can be supported by improved communication by collaborative computing through GSS and KMS. Computerized systems can facilitate communication by helping people explain and justify their suggestions and opinions.

12 .

Decision implementation can also be supported by expert systems. An ES can be used as an advisory system regarding implementation problems (such as handling resistance to change). Finally, an ES can provide training that may smooth the course of implementation.

Impacts along the value chain are typically identified by BAM, BPM, SCM, and ERP systems. CRM systems report and update internal records based on the impacts of the implementation. These inputs are then used to identify new problems and opportunities—a return to the intelligence phase.

(Note to graders: The text provides additional examples.)

2.

Describe how new technologies can provide decision-making support.

With the development of mobile commerce (m-commerce), more and more personal devices (personal digital assistants, cell phones, tablet computers, laptop computers) can access information sources, and users can respond to systems with information updates, collaboration efforts, and decisions. This can help salespeople, for example, to be more effective by accessing their CRM while on the road. Constant access to corporate data, inventory and otherwise, can only help them in their work. Wireless devices are taking on greater importance in the enterprise, generally by accessing specialized Web servers that provide data and communication directly to the m-commerce device.

Section 2.9 Review Questions 1.

List the key characteristics and capabilities of DSS. These are shown in Figure 2.3, and amplified in the text below: • Support for (mainly) semistructured and unstructured situations, combining human judgment with computerized information.

• Support for all managerial levels, from top executives to line managers.

• Support for individuals and groups.

• Support for interdependent and/or sequential decisions.

• Support all four decision-making phases: intelligence, design, choice, and implementation.

• Support variety of decision-making processes and styles.

• Users can add, delete, combine, change, or rearrange basic elements, or modify them to solve other, similar problems.

• User-friendly interface. (Most new DSS applications use Web-based interfaces.)

• Improves effectiveness of decision making (accuracy, timeliness, quality) rather than its efficiency.

13 .

• • • • •

2.

Decision maker controls all steps of the decision-making process in solving a problem.

End users can develop and modify simple DSS by themselves; larger DSS require IS specialists.

Models are generally utilized to enable experimenting with different strategies.

Access to a variety of data sources, formats, and types, including GIS, multimedia, and object oriented.

Used by an individual decision maker or distributed throughout one or more organizations.

Describe how providing support to a workgroup is different from providing support to group work. Explain why it is important to differentiate these concepts.

A workgroup often works on separate problems within its area of responsibility.

For example, a claim processing workgroup in an insurance company may consist of several people who work on their assigned claims by themselves.

In group work, multiple people cooperate to solve the same problem. Insurance claims processing could be group work if handling a claim required the cooperation of policy analysts, medical experts, vehicle dynamics experts, legal experts, and others.

3.

What kinds of DSS can end users develop in spreadsheets?

Simple ones—this reflects both the limitations of the spreadsheet as a DSS development tool and the limitations of the typical end user in developing complex information systems.

4.

Why is it so important to include a model in a DSS? The modeling capability of a DSS, which is what differentiates DSS from other MSS, is needed to enable experimenting with different decisions under different configurations and assumptions.

Section 2.10 Review Questions 1.

List the DSS classifications of the AIS SIGDSS. • • • • • •

Communications-driven and group DSS (GSS) Data-driven DSS Document-driven DSS Knowledge-driven DSS, data mining, and management ES applications Model-driven DSS Compound DSS, hybrids that combine two or more of these categories 14 .

2.

Define document-driven DSS. A document-driven DSS relies on knowledge coding, analysis, search, and retrieval for decision support. This includes all text-based DSS and most KMS. Document-driven DSS have minimal emphasis on mathematical models.

3.

List the capabilities of institutional DSS and ad hoc DSS. An institutional DSS is planned and developed to handle a recurring decision. It must have the flexibility to deal with that decision in different manifestations, with different data, over time. Such DSS tend to be used at the managerial control and operational levels.

An ad hoc DSS is developed to handle a one-time problem. Such problems typically appear at the strategic and management control levels. Such a DSS need not have the same degree of flexibility as an institutional DSS to deal with variations in the problem. However, problems that were not expected to recur still often do so, or it turns out that the DSS is applicable to other problems as well.

4.

Define the term ready-made DSS. A ready-made DSS is a DSS software product designed to be used, with minimal modifications, by several organizations that have comparable decision-making needs. Such DSS are often designed for a specific industry (e.g., hospitals) or functional area (e.g., finance).

Section 2.11 Review Questions (This section has no review questions.)

ANSWERS TO APPLICATION CASE QUESTIONS FOR DISCUSSION

 

Application Case 2.1: Making Elevators Go Faster!

(This application case has no discussion questions.)

Application Case 2.2: Stations Casinos Wins by Building Customer Relationships Using Its Data 1.

Why is this decision support system classified as a data-focused DSS? A major part of this system involves integration of data from over 500 data sources, incorporated in a Teradata data warehouse. This falls under the category 15 .

of data-driven DSS. Cognos is a tool that provides OLAP cube functionality, which is another characteristic of data driven DSS systems.

2.

What were some of the benefits from implementing this solution?

Enhanced customer segmentation allowing for targeted marketing, reduced promotion costs, improved member retention, and increased profit.

Application Case 2.3: SNAP DSS Helps OneNet Make Telecommunications Rate Decisions (This application case has no discussion questions.)

Application Case 2.4: From a Game Winner to a Doctor!

1.

What is a cognitive system? How can it assist in real-time decision making?

A cognitive system is a knowledge-based DSS model that applies domain knowledge in a particular field (like medicine) and applies human-like reasoning processes, including natural language processing, hypothesis generation, and evidence-based machine learning to assist in problem solving. Watson is a cognitive system that utilizes many AI algorithms and techniques to support medical decision making. Such systems can provide quick answers to complex problems, whereas individual physicians don’t have the time to study all the relevant literature and obtain that level of knowledge.

2.

What is evidence-based decision making?

Evidenced-based decision making is a term often associated with current practices in medicine, and is contrasted with just making decisions based on tradition or conventional wisdom. It involves the use of scientific results and clinical trials, statistical analyses, cost-benefit models, and other DSS-relevant approaches.

3.

What is the role played by Watson in the discussion?

Watson brings the intelligence that made it such a daunting Jeopardy! contestant into the medical world by serving as a decision support system (Dr. Watson).

Watson facilitates evidence-based support for its suggestions and provides a platform for physicians to look at a case from multiple perspectives. Watson also aids insurance providers in detecting fraudulent claims and protecting physicians from malpractice suits.

4.

Does Watson eliminate the need for human decision making?

16 .

No, Watson is used by physicians to support and improve their decision-making capabilities. In this sense, Watson is not operating as an automated decision system (ADS), but more of an expert system (ES).

ANSWERS TO END OF CHAPTER QUESTIONS FOR DISCUSSION 1.

  

Why is intuition still an important aspect of decision making?

Intuition is a critical factor that decision makers use in solving unstructured and semi-structured problems. The best decision makers recognize the tradeoff between the marginal cost of obtaining further information and analysis versus the benefit of making a better decision. (General Patton’s adage, “A good decision today is better than a perfect decision tomorrow,” reflects the application of this principle to his situation.) When decisions must be made quickly, the intuition of a seasoned, excellent decision maker is called for ideally. When an inexperienced or ill-trained decision maker attempts to use intuition, however, disaster can strike.

(See the related answer to Review Question 6 of Section 2.6 above.)

2.

Define efficiency and effectiveness, and compare and contrast the two.

Efficiency refers to the proper utilization of resources, doing things right.

Effectiveness refers to the attainment of goals, doing the right things.

Driving from New York City to Boston by way of Chicago in a hybrid compact that gets 50 miles per gallon (21 km/liter) may be efficient, but is not effective (in terms of reaching the goal, Boston, quickly).

Driving directly, but alone in a large vehicle that gets 12 mpg (5 km/l), may be effective but is not efficient as it uses four times the resources—in some sense, wasting 75 percent of them.

Ideally, one should be both effective and efficient, but if that is not possible a decision maker should focus on effectiveness.

3.

Why is it important to focus on the effectiveness of a decision, not necessarily the efficiency of making a decision?

The effectiveness of a decision affects an organization for as long as the decision matters, and as broadly as the decision scope. The efficiency of making the decision affects only the decision-making process itself. Since the resources involved in making a decision are far less than those affected by its 17 .

implementation, it is more important to make sure the decision is correct than to save resources used in making it.

4.

What are some of the measures of effectiveness in a toy manufacturing plant, a restaurant, an educational institution, and the U.S. Congress?

Toy manufacturing plant: Cost per unit produced, percent of orders delivered on time, quality of finished product (perhaps measured by number of defects per output or the incidence of warranty claims).

Restaurant: Revenue per table per shift, percent of repeat customers, rate of return on investment, rating by professional reviewers, rating by local health inspection authorities.

Educational institution: Students’ performance on national tests, number of students getting jobs in their discipline after graduation, amount of funded research per faculty member, percentage of students successfully finishing and receiving degrees, success of athletic teams.

U.S. Congress: Number of bills passed per year, public image.

Since these are not the only measures for any of the examples, other correct answers are possible.

5.

Even though implementation of a decision involves change, and change management is very difficult, explain how change management has not changed very much in thousands of years. Use specific examples throughout history.

Moses, while attempting to lead the children of Israel away from idol worship, found them rebellious and often desiring to return to slavery in Egypt rather than face the unknown of the desert. When he left them for a few days, they reverted to idolatry with the Golden Calf.

Many of those living in England’s North American colonies in the 1770s preferred the security and known economic benefits of that form of government to the possible benefits, but unknown risks, of independence.

Railroads of the 1950s and 1960s could not adapt to the changes required to see themselves as in the transportation business and therefore lost most of their passengers to airlines and buses.

U.S. automobile companies in the 1970s did not change their products from the large cars, built and bought with the expectation of being traded in frequently, that they were used to. As a result, they lost market share to foreign firms, whose cars offered higher quality and better fuel economy.

18 .

Of course, many other examples are possible.

6.

Your company is considering opening a branch in China. List typical activities in each phase of the decision (intelligence, design, choice, and implementation) of whether to open a branch.

Intelligence: Should we invest in China? What are the possible advantages, disadvantages, risks? What resources would have to be diverted from other activities? When could we start?

Design: Select criteria for assessing the alternatives (e.g., the ROI, market share); create alternatives: invest now, invest later, do not invest; levels and timing of investment; information flow for decision making; prepare a model; how will the choice be made, by whom, and when?

Choice: Get information; final evaluation; sensitivity analysis; solve model.

Implementation: Prepare an implementation plan; deal with resistance to change and necessary approvals and authorizations; conduct training; transfer resources.

These are not the only possible activities for any of the phases; many other correct answers are possible.

7.

You are about to buy a car. Using Simon’s four-phase model, describe your activities at each step.

Intelligence: You recognize that your needs could be better served by a different car, due to a change in your present car (decreasing reliability, growing repair bills) or your situation (changed transportation needs, more money).

Design: Determine parameters that describe the appropriate car to buy. Also determine the criteria (objective function) that indicates the value of certain features in a car. From the multiple criteria, a principle of choice should be developed. The alternatives must be identified as well. This is a multiple-criteria problem. In assessing alternatives, it may be necessary to visit dealers to drive candidate cars and determine actual selling prices.

Choice: Choose the car.

Implementation: Buy the car.

8.

Explain, through an example, the support given to decision makers by computers in each phase of the decision process.

19 .

Intelligence: A computer can scan a large amount of information to identify problems such as a decline in sales.

Design: To boost sales, several advertising proposals may be evaluated. A computerized model that shows the cost of an advertising program and its expected effect on sales can help a decision maker assess their potential.

Choice: Once alternatives are identified and their potential effect forecasted, a choice can be made. The computer can rank choices by cost/benefit ratio, net benefit, or any other desired criterion.

Implementation: A detailed “what-if” analysis can convince management that the proposed alternative is indeed the best. Computer-generated graphics can ease the implementation process.

9.

Some experts believe that the major contribution of DSS is to the implementation of a decision. Why is this so?

Proposed solutions are no good unless they are implemented. Implementation in many cases is the most difficult phase of decision making. However, a specific DSS may be designed to help with any decision phase or phases, in which case the contribution of that DSS is to those phases.

10.

Review the major characteristics and capabilities of DSS. How do each of them relate to the major components of DSS? The characteristics and capabilities of DSS are listed in Figure 2.3 as follows.

Their relationship to the major components of DSS is noted after each one.

• Semistructured and unstructured problems: The nature of all DSS components reflects this.

• Support managers at all levels: A given DSS may be oriented more toward one level than others. This will be reflected in its user interface.

• Support individuals and groups: DSS designed to support groups must include communication features in their user interface component.

• Interdependent or sequential decisions: This is controlled by the decision maker.

• Support all four decision phases: intelligence, design, choice, and implementation. The DSS must incorporate models, and optionally knowledge-based capabilities, appropriate to each phase. The data and user interface will in most cases be common to all phases.

• Support variety of decision processes and styles: This relates primarily to the user interface. It also relates to the models (and knowledge-based system, if present), because decision makers with different styles will want different levels of analytical support.

20 .

• • • • • • • •

11.

Adaptable and flexible: Primarily a function of the user interface, secondarily reflected in the variety of models and data sources included Interactive ease of use: Relates to the user interface Effectiveness, not efficiency: This relates largely to the way all DSS components were developed and the priorities used in the development process.

Humans control the process: Relates to the user interface Ease of development by end users: Relates to the tools used to build models and to the end-user query facility of the database Modeling and analysis: Describes the models in the DSS Data access: Describes the database management capabilities of the DSS Stand-alone, integrated and/or Web-based: Reflects the way all components were developed

List some internal data and external data that could be found in a DSS for a university’s admission office.

Internal data: Capacities of various majors for new students, historical success rates of applicants with specific characteristics, amount of money available for different forms of financial aid, team needs for specific types of athletes, agreed cutoff criteria for factors such as SAT scores.

External data: SAT scores and high school academic records of individual applicants, average grades given at various secondary schools

12.

Why does a DSS need a DBMS, a model management system, and a user interface, but not necessarily a knowledge-based management system?

The database management system is required to store and retrieve the variety of data required by a DSS. The model-management system is required to provide the capabilities to analyze the data. The user interface provides a mechanism for the user to be able to interact with the DSS. All these capabilities are necessary; none can be dispensed with. The knowledge-based management system, by contrast, provides knowledge or expertise that might also be provided by an experienced user or, possibly trading off decision quality for system cost and complexity, dispensed with entirely.

13.

What are the benefits and the limitations of the AIS SIGDSS classification for DSS? The benefits of any classification are that, by grouping items that share common characteristics, one can benefit by knowing the common characteristics of each type rather than having to learn the characteristics of each item individually. The AIS SIGDSS DSS classification provides these benefits in a concise way by focusing on the key factor that drives each type of DSS. 21 .

The limitations of this classification are that any broad grouping of items must of necessity ignore or gloss over differences among items within each category.

Thus, this classification ignores issues such as whether a DSS is custom-built or ready-made, whether it is institutional or ad hoc, and other differences that can be important in many situations.

14.

Search for a ready-made DSS. What type of industry is its market? Explain why it is a ready-made DSS. The answer to this question will depend on the ready-made DSS that each student finds.

ANSWERS TO END OF CHAPTER APPLICATION CASE QUESTIONS 1.

 

Explain why solving the empty container logistics problem contributes to cost savings for CSAV. The container fleet of 700,000 is valued at $2 billion, second only to vessel fuel in terms of cost. Empty containers are wasted space, and reducing this has a large impact on spending and operational efficiency.

2.

What are some of the qualitative benefits of the optimization model for the empty container movements?

It allows system activities in all regional centers to be well coordinated while still maintaining operational flexibility and creativity. Forecasting errors were reduced with this model, allowing CSAV’s depot to maintain lower safety stocks. The generation of intelligent information from historical transactional data helped increase efficiency of operations. The MC Network Flow model helps optimize the repositioning of empty containers.

3.

What are some of the key benefits of the forecasting model in the ECO system implemented by CSAV? Reducing forecasting errors, allow for lower safety stock levels.

4.

Perform an online search to determine how other shipping companies handle the empty container problem. Do you think the ECO system would directly benefit those companies?

Many companies can be found on the Web with reference to the empty container problem. In addition to CSAV, a Google search finds: Magellan, Maersk, and TransBaltic (to name a few) as companies grappling with the empty container 22 .

problem. Two big issues with empty containers are the cost (waste of space), and the problem of repositioning the empty containers on a ship. Also, since containers are usually leased, this raises the question of which to keep on board.

There are many web sources describing strategies, heuristics, and intelligent systems that tackle these complex problems, so ECO is part of an industry-wide effort in applying intelligent systems to the empty container problem.

5.

Besides shipping logistics, can you think of any other domain where such a system would be useful in reducing cost?

Many industries and services include complex logistics, including problems of scheduling, inventory management, routing, and work flow. These problems often require managing limited resources of various kinds, with the danger that sometimes resources will be underutilized and other times overutilized. The models and reasoning algorithms of a system like ECO can be applied to these problems as well.

23 .

Chapter 3: Data Warehousing Learning Objectives for Chapter 3 1.

2.

3.

4.

5.

6.

Understand the basic definitions and concepts of data warehouses Understand data warehousing architectures Describe the processes used in developing and managing data warehouses Explain data warehousing operations Explain the role of data warehouses in decision support Explain data integration and the extraction, transformation, and load (ETL) processes 7. Describe real-time (active) data warehousing 8. Understand data warehouse administration and security issues

CHAPTER OVERVIEW The concept of data warehousing has been around since the late 1980s. This chapter provides the foundation for an important type of database, called a data warehouse, which is primarily used for decision support and provides improved analytical capabilities. We discuss data warehousing in the following sections:

CHAPTER OUTLINE 3.1

OPENING VIGNETTE: ISLE OF CAPRI CASINOS IS WINNING WITH ENTERPRISE DATA WAREHOUSE  Questions for the Opening Vignette A. WHAT WE CAN LEARN FROM THIS VIGNETTE

3.2

DATA WAREHOUSING DEFINITIONS AND CONCEPTS A. WHAT IS A DATA WAREHOUSE? B. A HISTORICAL PERSPECTIVE TO DATA WAREHOUSING C. CHARACTERISTICS OF DATA WAREHOUSING D. DATA MARTS E. OPERATIONAL DATA STORES F. ENTERPRISE DATA WAREHOUSES (EDW) 1 .

⧫

G.

Application Case 3.1: A Better Data Plan: Well-Established TELCOs Leverage Data Warehousing and Analytics to Stay on Top in a Competitive Industry

METADATA  Section 3.2 Review Questions

3.3

DATA WAREHOUSING PROCESS OVERVIEW ⧫ Application Case 3.2: Data Warehousing Helps MultiCare Save More Lives  Section 3.3 Review Questions

3.4

DATA WAREHOUSING ARCHITECTURES A. ALTERNATIVE DATA WAREHOUSING ARCHITECTURES B. WHICH ARCHITECTURE IS THE BEST?  Section 3.4 Review Questions

3.5

DATA INTEGRATION AND THE EXTRACTION, TRANSFORMATION, AND LOAD (ETL) PROCESSES A. DATA INTEGRATION ⧫ Application Case 3.3: BP Lubricants Achieves BIGS Success B. EXTRACTION, TRANSFORMATION, AND LOAD  Section 3.5 Review Questions

3.6

DATA WAREHOUSE DEVELOPMENT ⧫ Application Case 3.4: Things Go Better with Coke’s Data Warehouse A. DATA WAREHOUSE DEVELOPMENT APPROACHES 1.

The Inmon Model: The EDW Approach 2.

The Kimball Model: The Data Mart Approach 3.

Which Model Is Best? ⧫ Application Case 3.5: Starwood Hotels & Resorts Manages Hotel Profitability with Data Warehousing B. ADDITIONAL DATA WAREHOUSE DEVELOPMENT CONSIDERATIONS ⧫ Technology Insights 3.1: Hosted Data Warehouses C. REPRESENTATION OF DATA IN DATA WAREHOUSE D. ANALYSIS OF DATA IN THE DATA WAREHOUSE E. OLAP VERSUS OLTP F. OLAP OPERATIONS 1.

Variations of OLAP ⧫ Technology Insights 3.2: Hands-On Data Warehousing with MicroStrategy  Section 3.6 Review Questions

3.7

DATA WAREHOUSING IMPLEMENTATION ISSUES 2 .

⧫

A.

Application Case 3.6: EDW Helps Connect State Agencies in Michigan MASSIVE DATA WAREHOUSES AND SCALABILITY  Section 3.7 Review Questions

3.8

REAL-TIME DATA WAREHOUSING ⧫ Application Case 3.7: Egg Plc Fries the Competition in Near Real Time ⧫ Technology Insights 3.3: The Real-Time Realities of Active Data Warehousing  Section 3.8 Review Questions

3.9

DATA WAREHOUSE ADMINISTRATION, SECURITY ISSUES, AND FUTURE TRENDS ⧫ Technology Insights 3.4: Ambeo Delivers Proven DataAccess Auditing Solution A. THE FUTURE OF DATA WAREHOUSING  Section 3.9 Review Questions

3.10

RESOURCES, LINKS, AND THE TERADATA UNIVERSITY NETWORK CONNECTION A. RESOURCES AND LINKS B. CASES C. VENDORS, PRODUCTS, AND DEMOS D. PERIODICALS E. ADDITIONAL REFERENCES F. THE TERADATA UNIVERSITY NETWORK (TUN) CONNECTION Chapter Highlights Key Terms Questions for Discussion Exercises Teradata University and Other Hands-On Exercises Team Assignments and Role-Playing Projects Internet Exercises ⧫ End of Chapter Application Case: Continental Airlines Flies High with Its Real-Time Data Warehouse  Questions for the Case References

TEACHING TIPS/ADDITIONAL INFORMATION

       

Expect to spend some time on this chapter, as understanding its content is crucial and many of the concepts are not intuitive to students whose major computer exposure has been at the personal level. You should prepare yourself with some technical examples of how things work, such as screen shots from some of the tools mentioned in the chapter, 3 .

as most of the technical discussion here is at a more conceptual level. The business discussion is tangible, but the technical part is less so. It will be up to you to connect it to the students.

Section 3.2 defines several fundamental concepts. Students must understand that these are not definitions for the sake of definitions (every textbook has some of those, this one is no exception) but are important to anyone working in the field. The characteristics of a data warehouse in its extended definition, for example, are given not because some researcher thought they’d be theoretically nice, but, more specifically, because they’ve turned out to be important in practice. So, it’s important to really know what each of them is about. (The concept of an operational data store should already be familiar to students, though perhaps not by that name. They should know what metadata are from their use with general databases, too.)

Section 3.3 is brief. It can be covered quickly. It’s important to understand what each step does, but the details of things like ETL will be covered later in this chapter.

Section 3.4, by contrast, is seven pages long—though a lot of that is diagrams.

The key point here is that there are a lot of options; the correct one can be chosen only when something is known about the characteristics of the application, and in most cases more than one will work reasonably well.

Section 3.5 discusses ETL, a process that never shows up outside a data warehouse context and that students may therefore never have heard of before this course. The need for ETL arises because databases aren’t all in the same format; chances are that few, if any, of the source databases are in the format of the DW—and even if the formats match, most databases contain dirty data. This is a good place to show screen shots of ETL tools so students will really appreciate how they work.

Section 3.6, another long one, is on the development process. Students will have studied the development and implementation of information systems in general in their introductory MIS course and probably gone deeper into it in Systems Analysis and other courses. Here, it helps to focus on what’s different about data warehouses. For example, while training is needed for both conventional systems and data warehouses, the training needed for a DW is at a higher level: more information literacy than computer literacy. A systems analyst who is used to the correct type of training for ERP or SCM systems, and who attempts to replicate that experience for a DW user community, is courting disaster.

This section covers more than the database development suggested by its title. It also discusses the internal structure of a data warehouse database. It first focuses on the star schema and then describes multidimensional analytical queries. Students who’ve studied database management—likely to be most of the class in a senior-level business school course for MIS concentrators—will want tangible information here. You can provide this with examples showing how a simple set of historical sales data with several dimensions could be stored in an operational database and in a data warehouse organized by the star schema.

If you have provided your students with a good foundation up to this point, Sections 3.7 and 3.8 should go quickly. Section 3.7 raises some issues to prepare for in implementing a DW, including the importance of scalability. In Section 3.8, the business case for real-time DWs is well presented, and the technology introduces no new conceptual difficulties.

4 .

Section 3.9 returns to issues that students will be familiar with from a general database management context. The need for “establishing effective corporate and security policies and procedures,” for instance, is not unique to data warehouses. Therefore, you may, again, want to focus on what’s different about them. Answers to Question for Discussion 5 at the end of the chapter can lead to a good class discussion of this.

ANSWERS TO END OF SECTION REVIEW QUESTIONS

     

Section 3.1 Review Questions 1.

Why is it important for Isle to have an EDW? In the gaming industry, companies distinguish themselves based on their customer relationships. An enterprise data warehouse (EDW) gathers and provides the data needed to tell Isle of Capri what customers respond to, so the casinos can adapt their offerings. The information provided by the EDW lets Isle deepen its understanding of customers, so it can efficiently give them more of the kinds of entertainment they are looking for.

2.

What were the business challenges or opportunities that Isle was facing?

Isle of Capri Casinos is one of the largest publicly traded gaming companies in the United States, but the gaming business is competitive. Other casinos offer essentially the same games, so Isle must find ways to make its entertainment and hospitality atmosphere one that exceeds customer expectations. Before Isle implemented the EDW, casino managers had to wait to review monthly data until the second week of the following month. The time lag made it difficult for casinos to identify what actions were appealing to customers in time to respond. Adding to these business challenges, Hurricane Katrina set back initial efforts to set up a data warehouse at the southeastern company.

3.

What was the process Isle followed to realize EDW? Comment on the potential challenges Isle might have had going through the process of EDW development.

Isle of Capri brought in two expert suppliers. Teradata provided the core solution; IBM Cognos provided expertise in business intelligence. Isle hired a management team that understood how EDW could support decision making at Isle. That team would be able to help Isle’s managers with the challenge of understanding how they can frame queries and follow-up questions to figure out ways to improve the business. Most likely, the potential of using detailed, up-to-the-minute data would be unfamiliar to many of these managers.

4.

What were the benefits of implementing an EDW at Isle? Can you think of other potential benefits that were not listed in the case?

5 .

The implementation of EDW brought several benefits related to the timeliness and detail of the data that became available. Instead of five week-old monthly reports, managers can now study a variety of daily, weekly, and monthly reports.

The reports segment data by particular properties and customer groups, so managers can zero in on particular problems and successes, easily making comparisons among properties. Managers can submit queries about data sets and receive information within minutes. In this way, managers can find out how particular promotions are affecting customer behavior at particular casinos. The EDW also connects data about casino activity with data about customers’ use of hotels and efforts by Isle’s hosts. This, too, helps the company target promotions and offer customers incentives they value. Even decisions as detailed as where to locate slot machines can be adjusted to boost profits based on data from the EDW. 5.

Why do you think large enterprises like Isle in the gaming industry can succeed without having a capable data warehouse/business intelligence infrastructure?

In the past, businesses in the gaming industry could succeed without a capable data warehouse/business intelligence infrastructure because their managers knew as much about customers as their competitors’ management knew. They were all testing ideas for promotion or entertainment and responding to customer behavior at about the same pace. However, when one company such as Isle begins to respond to daily, property-by-property data, choosing the marketing and entertainment options that deliver exactly what most profitably lures customers, competitors without that capability will soon begin to suffer.

Section 3.2 Review Questions 1.

What is a data warehouse?

A data warehouse is defined in this section as “a pool of data produced to support decision making.” This focuses on the essentials, leaving out characteristics that may vary from one DW to another but are not essential to the basic concept.

The same paragraph gives another definition: “a subject-oriented, integrated, time-variant, nonvolatile collection of data in support of management’s decisionmaking process.” This definition adds more specifics, but in every case appropriately: it is hard, if not impossible, to conceive of a data warehouse that would not be subject-oriented, integrated, etc.

2.

How does a data warehouse differ from a database?

Technically a data warehouse is a database, albeit with certain characteristics to facilitate its role in decision support. Specifically, however, it is (see previous question) an “integrated, time-variant, nonvolatile, subject-oriented repository of detail and summary data used for decision support and business analytics within 6 .

an organization.” These characteristics, which are discussed further in the section just after the definition, are not necessarily true of databases in general—though each could apply individually to a given one.

As a practical matter most databases are highly normalized, in part to avoid update anomalies. Data warehouses are highly denormalized for performance reasons. This is acceptable because their content is never updated, just added to.

Historical data are static.

3.

What is an ODS? Operational Data Store is the database from which a business operates on an ongoing basis.

4.

Differentiate among a data mart, an ODS, and an EDW. An ODS (Operational Data Store) is the database from which a business operates on an ongoing basis.

Both an EDW and a data mart are data warehouses. An EDW (Enterprise Data Warehouse) is an all-encompassing DW that covers all subject areas of interest to the entire organization. A data mart is a smaller DW designed around one problem, organizational function, topic, or other suitable focus area.

5.

Explain the importance of metadata.

Metadata, “data about data,” are the means through which applications and users access the content of a data warehouse, through which its security is managed, and through which organizational management manages, in the true sense of the word, its information assets. Most database management systems would be unable to function without at least some metadata. Indeed, the use of metadata, which enable data access through names and logical relationships rather than physical locations, is fundamental to the very concept of a DBMS. Metadata are essential to any database, not just a data warehouse. (See answer to Review Question 2 of this section above.)

Section 3.3 Review Questions 1.

Describe the data warehousing process.

The data warehousing process consists of the following steps: 1.

Data are imported from various internal and external sources 2.

Data are cleansed and organized consistently with the organization’s needs 3. a.

Data are loaded into the enterprise data warehouse, or 7 .

4.

5.

2.

Describe the major components of a data warehouse.

• • • • • •

3.

b.

Data are loaded into data marts. a.

If desired, data marts are created as subsets of the EDW, or b.

The data marts are consolidated into the EDW Analyses are performed as needed

Data sources. Data are sourced from operational systems and possibly from external data sources.

Data extraction and transformation. Data are extracted and properly transformed using custom-written or commercial software called ETL. Data loading. Data are loaded into a staging area, where they are transformed and cleansed. The data are then ready to load into the data warehouse.

Comprehensive database. This is the EDW that supports decision analysis by providing relevant summarized and detailed information.

Metadata. Metadata are maintained for access by IT personnel and users.

Metadata include rules for organizing data summaries that are easy to index and search.

Middleware tools. Middleware tools enable access to the data warehouse from a variety of front-end applications.

Identify and discuss the role of middleware tools.

Middleware tools enable access to the data warehouse. Power users such as analysts may write their own SQL queries. Others may access data through a managed query environment. There are many front-end applications that business users can use to interact with data stored in the data repositories, including data mining, OLAP, reporting tools, and data visualization tools. All these have their own data access requirements. Those may not match with how a given data warehouse must be accessed. Middleware translates between the two.

Section 3.4 Review Questions 1.

What are the key similarities and differences between a two-tiered architecture and a three-tiered architecture?

Both provide the same user visibility through a client system that accesses a DSS/BI application remotely. The difference is behind the scenes and is invisible to the user: in a two-tiered architecture, the application and data warehouse reside on the same machine; in a three-tiered architecture, they are on separate machines.

2.

How has the Web influenced data warehouse design?

Primarily by making Web-based data warehousing possible.

8 .

3.

List the alternative data warehousing architectures discussed in this section.

• • • • •

4.

Independent data marts architecture Data mart bus architecture with linked dimensional data marts Hub-and-spoke architecture (corporate information factory) Centralized data warehouse architecture Federated architecture

What issues should be considered when deciding which architecture to use in developing a data warehouse? List the 10 most important factors.

1.

2.

3.

4.

5.

6.

7.

8.

9.

10.

Information interdependence between organizational units Upper management’s information needs Urgency of need for a data warehouse Nature of end-user tasks Constraints on resources Strategic view of the data warehouse prior to implementation Compatibility with existing systems Perceived ability of the in-house IT staff Technical issues Social/political factors

(This list from the text, while clearly intended by the authors as the answer to this review question, does not explicitly say that these are the ten most important factors. Students may choose others.)

5.

Which data warehousing architecture is the best? Why? See Table 3.1 Average Assessment Scores for the Success of the Architectures.

What is interesting is the similarity of the averages for the bus, hub-and-spoke, and centralized architectures. The differences are sufficiently small that no claims can be made for a particular architecture’s superiority over the others, at least based on a simple comparison of these success measures.

Section 3.5 Review Questions 1.

Describe data integration.

Data integration is an umbrella term that covers three processes that combine to move data from multiple sources into a data warehouse: accessing the data, combining different views of the data, and capturing changes to the data.

2.

Describe the three steps of the ETL process.

9 .

Extraction: selecting data from one or more sources and reading the selected data.

Transformation: converting data from their original form to whatever form the DW needs. This step often also includes cleansing of the data to remove as many errors as possible.

Load: putting the converted (transformed) data into the DW. 3.

Why is the ETL process so important for data warehousing efforts?

Since ETL is the process through which data are loaded into a data warehouse, a DW could not exist without it. The ETL process also contributes to the quality of the data in a DW.

Section 3.6 Review Questions 1.

List the benefits of data warehouses.

Direct benefits include: • Allowing end users to perform extensive analysis in numerous ways.

• A consolidated view of corporate data (i.e., a single version of the truth).

• Better and more timely information. A data warehouse permits information processing to be offloaded from costly operational systems onto low-cost servers; therefore, end-user information requests can be processed more quickly.

• Enhanced system performance. A data warehouse frees production processing because some operational system reporting requirements are moved to DSS. • Simplification of data access.

Indirect benefits arise when end users take advantage of these direct benefits.

2.

List several criteria for selecting a data warehouse vendor, and describe why they are important.

Six important criteria are: financial strength, ERP linkages, qualified consultants, market share, industry experience, and established partnerships. These are important to indicate that a vendor is likely to be in business for the long term, to have the support capabilities its customers need, and to provide products that interoperate with other products the potential user has or may obtain.

One could add others, such as product functionality (Does it do what we need?), vendor strategic vision (Does their direction make sense for our future plans and/or is it consistent with industry trends?) and quality of customer references (What do their existing customers think of them?).

10 .

3.

What is OLAP and how does it differ from OLTP? Data stored in a data warehouse can be analyzed using techniques referred to as OLAP. OLAP is one of the most commonly used data analysis techniques in data warehouses. OLAP is an approach to quickly answer ad hoc questions that require data analysis.

OLTP is concerned with the capture and storage of data. OLAP is concerned with the analysis of that data.

4.

What is a cube? What do drill down, roll up, and slice and dice mean?

The main operational structure in OLAP is based on a concept called cube. A cube in OLAP is a multidimensional data structure (actual or virtual) that allows fast analysis of data.

Using OLAP, an analyst can navigate through the database and screen for a particular subset of the data (and its progression over time) by changing the data’s orientations and defining analytical calculations. These types of user-initiated navigation of data through the specification of slices (via rotations) and drill down/up (via aggregation and disaggregation) are sometimes called “slice and dice.” Commonly used OLAP operations include slice and dice, drill down, roll up, and pivot.

5.

•

Slice: A slice is a subset of a multidimensional array (usually a twodimensional representation) corresponding to a single value set for one (or more) of the dimensions not in the subset.

•

Dice: The dice operation is a slice on more than two dimensions of a data cube.

•

Drill Down/Up: Drilling down or up is a specific OLAP technique whereby the user navigates among levels of data ranging from the most summarized (up) to the most detailed (down).

What are ROLAP, MOLAP, and HOLAP? How do they differ from OLAP? ROLAP stands for Relational Online Analytical Processing. ROLAP is an alternative to the MOLAP (Multidimensional OLAP) technology. While both ROLAP and MOLAP analytic tools are designed to allow analysis of data through the use of a multidimensional data model, ROLAP differs in that it does not require the pre-computation and storage of information. Instead, ROLAP tools access the data in a relational database and generate SQL queries to calculate information at the appropriate level when an end user requests it. MOLAP is an alternative to the ROLAP technology. MOLAP differs from ROLAP significantly 11 .

in that it requires the pre-computation and storage of information in the cube— the operation known as preprocessing. MOLAP stores this data in an optimized multidimensional array storage, rather than in a relational database.

HOLAP (Hybrid Online Analytical Processing) is a combination of ROLAP and MOLAP. HOLAP allows storing part of the data in a MOLAP store and another part of the data in a ROLAP store. The degree of control that the cube designer has over this partitioning varies from product to product.

All of these are variations of OLAP.

Section 3.7 Review Questions 1.

What are the major DW implementation tasks that can be performed in parallel?

Reeves (2009) and Solomon (2005) provided some guidelines regarding the critical questions that must be asked, some risks that should be weighted, and some processes that can be followed to help ensure a successful data warehouse implementation. They compiled a list of 11 major tasks that could be performed in parallel: Establishment of service-level agreements and data-refresh requirements Identification of data sources and their governance policies Data quality planning Data model design ETL tool selection Relational database software and platform selection Data transport Data conversion Reconciliation process Purge and archive planning End-user support

2.

List and discuss the most pronounced DW implementation guidelines.

•

Senior management must support development of the data warehouse. The DW needs a project champion at a high position in the organization chart.

12 .

• •

•

3.

4.

Benefits of a DW project may be difficult to measure, so management support makes it more likely the project will receive funding.

Web-based data warehouses may need special security requirements.

These ensure that only authorized users have access to the data.

Users should participate in the development process. Their participation is essential for data modeling and access modeling. User participation ensures that the DW includes the needed data and that decision makers can retrieve the data they need.

DW implementation requires certain skills from members of the development team: in-depth knowledge of database technology and the development tools used.

When developing a successful data warehouse, what are the most important risks and issues to consider and potentially avoid?

•

Starting with the wrong sponsorship chain

•

Setting expectations that you cannot meet

•

Engaging in politically naive behavior

•

Loading the data warehouse with information just because it is available

•

Believing that data warehousing database design is the same as transactional database design

•

Choosing a data warehouse manager who is technology oriented rather than user oriented

•

Focusing on traditional internal record-oriented data and ignoring the value of external data and of text, images, and, perhaps, sound and video

•

Delivering data with overlapping and confusing definitions

•

Believing promises of performance, capacity, and scalability

•

Believing that your problems are over when the data warehouse is up and running

•

Focusing on ad hoc data mining and periodic reporting instead of alerts

What is scalability? How does it apply to DW? Scalability refers to the degree to which a system can adjust to changes in demand without major additional changes or investments. DW scalability issues are the amount of data in the warehouse, how quickly the warehouse is expected to grow, 13 .

the number of concurrent users, and the complexity of user queries. A data warehouse must scale both horizontally and vertically. The warehouse will grow as a function of data growth and the need to expand the warehouse to support new business functionality. Data growth may be a result of the addition of current cycle data (e.g., this month’s results) and/or historical data.

Section 3.8 Review Questions 1.

What is an RDW? A real-time data warehouse, in which decision-making data are updated on an ongoing basis as business transactions occur; same as an active data warehouse (ADW).

2.

List the benefits of an RDW. The RDW extends the benefits of data warehousing, in general, down into tactical, and perhaps operational, decision making. It empowers people who interact directly with customers and suppliers by providing them with information to make decisions. It can then be extended to customers and suppliers themselves, thus affecting almost all aspects of customer service, SCM, logistics, and beyond.

It can also facilitate e-business activities, as when sales outlets such as overstock.com (cited in the text) use historical data to price new close-outs.

3.

What are the major differences between a traditional data warehouse and an RDW? 1.

2.

3.

4.

5.

6.

7.

4.

A traditional data warehouse (TDW) is used for strategic decisions (and sometimes tactical); an RDW for strategic and tactical (sometimes operational) ones.

The results of using a TDW can be hard to measure; results of using an RDW are measured by operational data.

Acceptable TDW refresh rates range from daily to monthly; RDW data must be up to the minute.

TDW summaries are often appropriate; RDWs must supply detailed data.

Small user community at upper organizational levels means a TDW supports few concurrent users; an RDW must support many, perhaps over a thousand.

TDWs typically use restrictive reporting to confirm or check patterns, often predefined summary tables; RDWs need flexible, ad hoc reporting.

TDW user community generally consists of power users, knowledge workers, managers, other internal users; RDWs are used by operational staff, call centers, perhaps external users.

List some of the drivers for RDW. 14 .

• • •

•

A business often cannot afford to wait a whole day for its operational data to load into the data warehouse for analysis.

Traditional data warehouses have captured snapshots of an organization’s fixed states instead of incremental real-time data showing every state change and almost analogous patterns over time.

With a traditional hub-and-spoke architecture, retaining the metadata in sync is difficult. It is also costly to develop, maintain, and secure many systems as opposed to one huge data warehouse so that data are centralized for BI/BA tools.

In cases of huge nightly batch loads, the necessary ETL setup and processing power for large nightly data warehouse loading might be very high, and the processes might take too long. An EAI with real-time data collection can reduce or eliminate the nightly batch processes.

Section 3.9 Review Questions 1.

What steps can an organization take to ensure the security and confidentiality of customer data in its data warehouse?

Effective security in a data warehouse should focus on four main areas: Step 1. Establishing effective corporate and security policies and procedures. An effective security policy should start at the top and be communicated to everyone in the organization.

Step 2. Implementing logical security procedures and techniques to restrict access.

This includes user authentication, access controls, and encryption.

Step 3. Limiting physical access to the data center environment.

Step 4. Establishing an effective internal control review process for security and privacy.

2.

What skills should a DWA possess? Why? • • • •

3.

Familiarity with high-performance hardware, software, and networking technologies, since the data warehouse is based on those Solid business insight, to understand the purpose of the DW and its business justification Familiarity with business decision-making processes to understand how the DW will be used Excellent communication skills, to communicate with the rest of the organization

What recent technologies may shape the future of data warehousing? Why? Following are some of the recently popularized concepts and technologies that will play a significant role in defining the future of data warehousing.

15 .

Sourcing: Acquisition of data from diverse and dispersed sources •

Web, social media, and Big Data

•

Open source software

•

SaaS (software as a service) “The Extended ASP Model”

•

Cloud computing

Infrastructure: Architectural—hardware and software—enhancements •

Columnar (a new way to score and access data in the database)

•

Real-time data warehousing

•

Data warehouse appliances (all-in-one solutions to DW)

•

Data management technologies and practices

•

In-database processing technology (putting the algorithms where the data is)

•

In-memory storage technology (moving the data in the memory for faster processing)

•

New database management systems

•

Advanced analytics

As the world of business becomes more global and complex, the need for business intelligence and data warehousing tools also becomes more prominent. The fast improving information technology tools and techniques seem to be moving in the right direction to address the needs of the future business intelligence systems.

Section 3.10 Review Questions (This section has no review questions.)

ANSWERS TO APPLICATION CASE QUESTIONS FOR DISCUSSION

 

Application Case 3.1: A Better Data Plan: Well-Established TELCOs Leverage Data Warehousing and Analytics to Stay on Top in a Competitive Industry 16 .

1.

What are the main challenges for TELCOs? To stay competitive, TELCOs must continuously refine everything from customer service to plan pricing. The major challenges faced by both entrenched and new companies in this industry include: retaining customers, decreasing costs, finetuning pricing models, improving customer satisfaction, acquiring new customers, and understanding the role of social media in customer loyalty.

2.

How can data warehousing and data analytics help TELCOs in overcoming their challenges?

Highly targeted data analytics play an ever more critical role in helping carriers secure or improve their standing in an increasingly competitive marketplace.

Argentina’s Telefónica de Argentina used analytics for its “traceability project,” which tracked the factors involved in customer churn, a big problem among phone service carriers. France’s Bouygues Telecom used BI technologies to facilitate cost reduction through automation via its Teradata-based marketing operations management system, which automates marketing/communications collateral production. Pakistan’s Mobilink uses BI to help acquire customers and grow their subscriber network, largely aided by social networking.

3.

Why do you think TELCOs are well suited to take full advantage of data analytics?

TELCOs control the telecommunications infrastructure, and acquire much usage data as a result. They have the technical expertise to create, deploy, and refine plans to address their business challenges. The industry and mobile technology have expanded and improved over the years, which provides a strong foundation on which to build intelligent solutions. The data analytics solutions that have been created to meet these challenges have also improved drastically over the past few years, placing TELCOs in a good position to capitalize on their technological advantages.

Application Case 3.2: Data Warehousing Helps Multi-Care Save More Lives 1.

What do you think is the role of data warehousing in healthcare systems?

This question seeks opinions, so accept any reasonable answer. The case focuses on the role of data warehousing in improving patient care, which can literally save lives. Data on patient needs, hospital practices, and patient outcomes can help health care systems identify the practices that lead to the best patient outcomes.

Of course, organizations also can use data warehousing to improve efficiency and profits.

17 .

2.

How did MultiCare use data warehousing to improve health outcomes?

MultiCare organized and simplified data from multiple sources across the continuum of care, so the Adaptive Data Warehouse became a single source that unified decision makers across functions. It addressed the serious problem of septicemia by defining a standard of care for septic patients, identify patients with conditions signaling they were likely to take a sudden turn for the worse, and efficiently implement the standard of care. Building on the first-year success of the effort, MultiCare intended to expand the application of data warehousing in similar ways to treat heart failure, manage the performance of the emergency department, and improve inpatient throughput.

Application Case 3.3: BP Lubricants Achieve BIGS Success 1.

What is BIGS at BP Lubricants?

BIGS is the Business Intelligence and Global Standards program, a strategic initiative for management information and business intelligence. Its purpose is to deliver globally consistent and transparent management information across functions.

2.

What were the challenge, the proposed solution, and the obtained results with BIGS? The challenge facing BP Lubricants was that it had been involved in merger activity, which meant it had data held in disparate source systems. To be more agile and prepare for growth, it wanted a unified system. The proposed solution was Kalido, an adaptive enterprise data warehousing solution. The system integrates and stores information from multiple source systems to provide consolidated views for marketing, sales, and finance. The obtained results are that BIGS helps the business identify a multitude of business opportunities to improve profits and lower costs. Typical benefits include improved consistency and transparency of business data, faster and more flexible reporting, and greater ability to respond intelligently to new business opportunities.

Application Case 3.4: Things Go Better with Coke’s Data Warehouse 1.

How did Coca-Cola in Japan use data warehousing to improve its business processes?

The Hokuriku Coca-Cola Bottling Co. (HCCBC) hired Teradata Corp. to implement a data warehouse and analytical software to collect historical data and near-real-time data wirelessly transmitted from each vending machine. The data

18 .

tells the company what is selling, and it sends alerts when a product sells out, a customer is short-changed, or a machine is malfunctioning.

2.

What were the results of their enterprise active data warehouse implementation?

In a pilot test of the system, the accurate demand forecasts and rapid problem identification led to an immediate 10 percent increase in sales. More accurate machine servicing contributed to a 42 percent decline in costs while improving the efficiency of the salespeople who service the machines. Based on the results, HCCBC intended to expand the use of the active data warehouse to cover all the vending machines in Japan.

Application Case 3.5: Starwood Hotels & Resorts Manages Hotel Profitability with Data Warehousing 1.

How big and complex are the business operations of Starwood Hotels & Resorts?

The business operations are very large and complex. Starwood is one of the world’s leading hotel and leisure companies, with 1,112 properties in nearly 100 countries. The company has 154,000 employees at properties that it owns and manages. It also operates a customer loyalty program.

2.

How did Starwood Hotels & Resorts use data warehousing for better profitability?

With data warehousing, Starwood Hotels can prepare operational reports in just four to six hours. With real-time feeds, transactions can be posted immediately to the data warehouse, and users can access the changes in just 5 to 10 minutes, or 288 times faster than before. The accelerated access to data improves profits because hotel managers can make sure rooms are available for premier customers and improve the ability to manage room occupancy rates. Marketing campaigns also have become more efficient as managers analyze results in days or weeks instead of months.

3.

What were the challenge, the proposed solution, and the obtained results?

The challenge was that the company had grown significantly, especially in the Asia/Pacific region. This greatly increased the need for business critical information about the hotels and their customers. Obtaining reports from the central reservation system could take up to 18 hours, and with delays in obtaining data, managers had difficulty adjusting rates to increase revenues. The proposed solution was an Oracle Exadata system that provides near-real-time information.

As a result of Starwood implementing the system, managers have accelerated access to data so they can do same-day or next-day analyses, as well as additional reporting tools. The company now operates more efficiently and profitably, and customer service has improved.

19 .

Application Case 3.6: EDW Helps Connect State Agencies in Michigan 1.

Why would a state invest in a large and expensive IT infrastructure (such as an EDW)?

Like a business, a state government can operate more efficiently and make better decisions if it has access to current data. State officials can use the EDW to improve efficiency and better serve the state’s residents.

2.

What are the size and complexity of EDW used by state agencies in Michigan?

Michigan’s EDW has almost 10,000 users in five major departments, 20 agencies, and more than 100 bureaus. Just in the Department of Human Services, the EDW contributes to nearly every function, including accurate delivery of and accounting for benefits to almost 2.5 million clients receiving public assistance.

3.

What were the challenges, the proposed solution, and the obtained results of the EDW? The challenge is not directly stated, but students should be aware that state governments struggle to balance budgets and satisfy a wide variety of needs. They also should recognize that a state operates many kinds of departments, so it likely could benefit from a unified source of information. The proposed solution was an enterprise (statewide) electronic data warehouse linking employees across departments. The obtained results include financial benefits worth $1 million per business day. Savings come from operational efficiencies, avoidance of sanctions, improved client outcomes, integrity benefits awarded to well-performing programs, and recovery of inappropriate benefits payments. The data warehouse has yielded a 15:1 cost-effectiveness ratio and improvements in the accurate delivery and accounting of benefits.

Application Case 3.7: Egg Plc Fries the Competition in Near Real Time 1.

What kind of business is Egg plc in? What is the competitive landscape?

Egg plc is an online bank that provides banking, insurance, investments, and mortgages to more than 3.6 million customers through its Internet site. Although the case does not discuss the competition, it mentions that Egg was once the world’s largest online bank. Students may note that the competitive landscape for online banking has changed considerably over the past few years, with most banks now offering at least some services online. Also accept any other reasonable description of the competitive landscape.

20 .

2.

How did Egg plc use near-real-time data warehousing for competitive advantage?

Near-real-time data access gave employees and customers access to data. Within the company, the data helped with the construction of sales and marketing campaigns. It allowed faster decision making about specific customers and customer classes. An interesting basis for classroom discussion might be the fact that Egg did not maintain a competitive advantage for long, as it has gone out of business. Setting up a data warehouse can give the organization an initial advantage over companies slower to adopt the technology, but how might a company maintain that advantage, using analytics?

ANSWERS TO END OF CHAPTER QUESTIONS FOR DISCUSSION 1.

  

Compare data integration and ETL. How are they related?

Data integration consists of three processes that integrate data from multiple sources into a data warehouse: accessing the data, combining different views of the data, and capturing changes to the data. It makes data available to ETL tools and, through the three processes of ETL, to the analysis tools of the data warehousing environment.

2.

What is a data warehouse and what are its benefits? Why is Web accessibility important with a data warehouse?

A data warehouse can be defined as “a pool of data produced to support decision making.” This focuses on the essentials, leaving out characteristics that may vary from one DW to another but are not essential to the basic concept.

The same paragraph gives another definition: “a subject-oriented, integrated, time-variant, nonvolatile collection of data in support of management’s decisionmaking process.” This definition adds more specifics, but in every case appropriately: it is hard, if not impossible, to conceive of a data warehouse that would not be subject-oriented, integrated, etc.

The benefits of a data warehouse are that it provides decision-making information, organized in a way that facilitates the types of access required for that purpose and supported by a wide range of software designed to work with it.

Web accessibility of a data warehouse is important because many analysis applications are Web-based, because users often access data over the Web (or over an intranet using the same tools) and because data from the Web may feed the DW.

21 .

3.

A data mart can replace a data warehouse or complement it. Compare and discuss these options.

For a data mart to replace a data warehouse, it must make the DW unnecessary.

This would mean that all the analyses for which the DW would be used can instead be satisfied by a DM (or perhaps a combination of several DMs). If this is so, it can be much less expensive, in terms of development and computer resources, to use multiple DMs (let alone one DM!) instead of an overall DW. In other situations, a data mart can be used for some analyses which would in its absence use the DW, but not all of them. For those, the smaller DM is more efficient—quite possibly, enough more efficient as to justify the cost of having a DM in addition to a DW. Here the DM complements the DW.

4.

Discuss the major drivers and benefits of data warehousing to end users.

Major drivers include: • Increased competition and pace of business, leading to increased need for good decisions quickly • Successful pioneering experiences with data warehouses, leading to their wider user acceptance • Decreasing hardware costs, making terabyte databases with masses of historical data economically feasible for more firms • Increased availability of software to manage a large data warehouse • Increased availability of analysis tools making DWs potentially more useful • Increased computer literacy of decision makers, making them more likely to use these tools

5.

List the differences and/or similarities between the roles of a database administrator and a data warehouse administrator.

Since a data warehouse is a specific type of database designed for a specific application area, a data warehouse administrator has all the roles of a database administrator—plus others. One new role is advising on decision support uses of the DW, for which a DWA needs to understand decision-making processes.

Beyond that, the issue is more a need for additional skills in the same roles as a DBA—e.g., understanding high-performance hardware to deal with the large size of a DW—than it is one of additional roles. (See Review Question 2, Section 5.8, for list of skills.)

6.

Describe how data integration can lead to higher levels of data quality.

22 .

A question involving the word “higher” (or any other comparative, for that matter) requires asking “higher than what?” In this case, we can take it to mean “higher than we would have for the same data, but without a formal data integration process.”

Without a data integration process to combine data in a planned and structured manner, data might be combined incorrectly. That could lead to misunderstood data (a measurement in meters taken as being in feet) and to inconsistent data (data from one source applying to calendar months, data from another to fourweek or five-week fiscal months). These are aspects of low-quality data which can be avoided, or at least reduced, by data integration.

7.

Compare the Kimball and Inmon approaches toward data warehouse development. Identify when each one is most effective.

Inmon’s approach starts with an enterprise data warehouse, creating data marts as subsets if appropriate. It is most effective when there is a recognized need for an EDW, an executive “champion” of the project, and a willingness to invest in a data warehousing infrastructure before it will show results.

Kimball’s approach starts with data marts, consolidating them into an EDW later if appropriate. It is most effective when it is desired to provide a “proof of concept” implementation before embarking on a full-scale EDW project or when a well-defined area with the greatest benefits can be identified.

8.

Discuss security concerns involved in building a data warehouse.

Security and privacy concerns are important in building a data warehouse: 1.

Laws and regulations, in the U.S. and elsewhere, require certain safeguards on databases that contain the type of information typically found in a DW. 2.

The large amount of valuable corporate data in a data warehouse can make it an attractive target.

3.

The need to allow a wide variety of unplanned queries in a DW makes it impractical to restrict end user access to specific carefully constrained screens, one way to limit potential violations.

9.

Investigate current data warehouse development implementation through offshoring. Write a report about it. In class, debate the issue in terms of the benefits and costs, as well as social factors.

Open-ended answer to the report; it is impossible to predict what the debate will bring.

23 .

A student’s position on this issue is related to his/her feelings on the relationship of national economies to the global economy. It can be argued that offshoring improves the global economy while potentially harming one or more of the national economies involved—such as the student’s own. U.S. students may see primarily the damage they perceive it does to their national economy (and to their own career prospects), but students in India may take a different view. The economic, political, and philosophical issues can be pursued well beyond what is practical in a DSS course.

If you feel students are too nationalistic on this issue, you can ask them if they feel the same way about a Massachusetts or California bank processing checks in Alabama to reduce labor costs. (This example uses U.S. territories, but similar issues exist in any country large enough to have regional economic differences.)

ANSWERS TO END OF CHAPTER APPLICATION CASE QUESTIONS 1.

 

Describe the benefits of implementing the Continental Go Forward strategy.

This strategy consisted of a number of interrelated, concurrent actions. The first version of the overall strategy had the benefit of restoring Continental (CO) to profitability and giving it first-place rankings by many airline industry metrics.

The second phase of the strategy led to savings of $41 million and a reduction of $7 million in fraud in the first year alone. Its revenue increased by over $500 million in six years. A data warehouse played a critical role in the second phase.

2.

Explain why it is important for an airline to use a real-time data warehouse.

It’s important for an airline to use a real-time data warehouse because many airline decisions cannot be made with week-old, or even day-old, data.

An example is frequent flyer award availability on a given flight. Airlines limit these so as not to give away too many seats that would otherwise be sold. Award seat allocation is usually automated. Travelers can check availability online.

American Airlines (and probably others) offers expanded award availability to the 30,000 or so people who fly at least 100,000 miles per year with them. When one of these wants an award seat that is not available online, he or she can request it by phone. The agent must decide whether or not to make it available. If the flight is selling slowly, the traveler gets the seat, even if the computer hasn’t allocated it for award. If it is likely to sell out, the seat isn’t offered, even to this select group.

To make this decision, telephone agents (and the Yield Management staff, which agents can consult) need up-to-the-minute, or at least up-to-the-hour, information.

3.

Identify the major differences between the traditional data warehouse and a realtime data warehouse, as was implemented at Continental.

24 .

A traditional data warehouse moves data from operational databases to the DW on a scheduled basis, typically daily or weekly. This provides consistent data for analyses performed during one update cycle, but does not make current information available for decisions that require it. A real-time DW, as was implemented at CO, moves data into the DW on an hourly or even more frequent basis.

4.

What strategic advantage can Continental derive from the real-time system as opposed to a traditional information system?

By having real-time data available through its data warehouse, CO can make decisions using up-to-date information. While data warehousing applications, which focus on long-term decisions, aren’t affected much by the last hour’s, day’s, or even week’s data, lower-level short-term decisions are. As the use of the DW is extended to these decisions and down in the organization, current data become necessary. By having real-time (or near-real-time) data in the system, CO obtains a strategic advantage by making better decisions.

25 .

Chapter 4: Business Reporting, Visual Analytics, and Business Performance Management

Learning Objectives for Chapter 4 1.

2.

3.

4.

5.

6.

7.

8.

9.

Define business reporting and understand its historical evolution Recognize the need for and the power of business reporting Understand the importance of data/information visualization Learn different types of visualization techniques Appreciate the value that visual analytics brings to BI/BA Know the capabilities and limitations of dashboards Understand the nature of business performance management (BPM) Learn the closed-loop BPM methodology Describe the basic elements of the balanced scorecard

CHAPTER OVERVIEW

              

A report is a communication artifact prepared with the specific intention of relaying information in a presentable form. If it concerns business matters, then it is called a business report. Business reporting is an essential part of the business intelligence movement toward improving managerial decision making. Nowadays, these reports are more visually oriented, often using colors and graphical icons that collectively look like a dashboard to enhance the information content. Business reporting and business performance management (BPM) are both enablers of business intelligence and analytics.

As a decision support tool, BPM is more than just a reporting technology. It is an integrated set of processes, methodologies, metrics, and applications designed to drive the overall financial and operational performance of an enterprise. It helps enterprises translate their strategies and objectives into plans, monitor performance against those plans, analyze variations between actual results and planned results, and adjust their objectives and actions in response to this analysis.

This chapter starts with examining the need for and the power of business reporting. With the emergence of analytics, business reporting evolved into dashboards and visual analytics, which, compared to traditional descriptive reporting, are much more predictive and prescriptive. Coverage of dashboards and visual analytics is followed by a comprehensive introduction to BPM. As you will see and appreciate, BPM and visual

1 .

analytics have a symbiotic relationship (over scorecards and dashboards) where they benefit from each other’s strengths.

CHAPTER OUTLINE 4.1

OPENING VIGNETTE: SELF-SERVICE REPORTING ENVIRONMENT SAVES MILLIONS FOR CORPORATE CUSTOMERS  Questions for the Opening Vignette A. WHAT WE CAN LEARN FROM THIS VIGNETTE

4.2

BUSINESS REPORTING DEFINITIONS AND CONCEPTS A. WHAT IS A BUSINESS REPORT? ⧫ Application Case 4.1: Delta Lloyd Group Assures Accuracy and Efficiency in Financial Reporting 1.

Metric Measurement Reports 2.

Dashboard-Type Reports 3.

Balanced Scorecard-Type Reports B. COMPONENTS OF THE BUSINESS REPORTING SYSTEM ⧫ Application Case 4.2: Flood of Paper Ends at FEMA  Section 4.2 Review Questions

4.3

DATA AND INFORMATION VISUALIZATION ⧫ Application Case 4.3: Tableau Saves Blastrac Thousands of Dollars with Simplified Information Sharing A. A BRIEF HISTORY OF DATA VISUALIZATION ⧫ Application Case 4.4: TIBCO Spotfire Provides DanaFarber Cancer Institute with Unprecedented Insight into Cancer Vaccine Clinical Trials  Section 4.3 Review Questions

4.4

DIFFERENT TYPES OF CHARTS AND GRAPHS A. BASIC CHARTS AND GRAPHS 1.

Line Chart 2.

Bar Chart 3.

Pie Chart 4.

Scatter Plot 5.

Bubble Chart B. SPECIALIZED CHARTS AND GRAPHS 1.

Histogram 2.

Gantt Chart 3.

Pert Chart 4.

Geographic Map 5.

Bullet 6.

Heat Map 7.

Highlight Table 8.

Tree Map 2 .



Section 4.4 Review Questions

4.5

THE EMERGENCE OF DATA VISUALIZATION AND VISUAL ANALYTICS ⧫ Technology Insights 4.1: Gartner Magic Quadrant for Business Intelligence and Analytics Platforms A. VISUAL ANALYTICS ⧫ Technology Insights 4.2: Telling Great Stories with Data and Visualization B. HIGH-POWERED VISUAL ANALYTICS ENVIRONMENTS  Section 4.5 Review Questions

4.6

PERFORMANCE DASHBOARDS ⧫ Application Case 4.5: Dallas Cowboys Score Big with Tableau and Teknion A. DASHBOARD DESIGN ⧫ Application Case 4.6: Saudi Telecom Company Excels with Information Visualization B. WHAT TO LOOK FOR IN A DASHBOARD C. BEST PRACTICES IN DASHBOARD DESIGN D. BENCHMARK KEY PERFORMANCE INDICATORS WITH INDUSTRY STANDARDS E. WRAP THE DASHBOARD METRICS WITH CONTEXTUAL METADATA F. VALIDATE THE DASHBOARD DESIGN BY A USABILITY SPECIALIST G. PRIORITIZE AND RANK ALERTS/EXCEPTIONS STREAMED TO THE DASHBOARD H. ENRICH DASHBOARD WITH BUSINESS USERS’ COMMENTS I. PRESENT INFORMATION IN THREE DIFFERENT LEVELS J. PICK THE RIGHT VISUAL CONSTRUCT USING DASHBOARD DESIGN PRINCIPLES K. PROVIDE FOR GUIDED ANALYTICS  Section 4.6 Review Questions

4.7

BUSINESS PERFORMANCE MANAGEMENT A. CLOSED-LOOP BPM CYCLE ⧫ Application Case 4.7: IBM Cognos Express Helps Mace for Faster and Better Business Reporting  Section 4.7 Review Questions

4.8

PERFORMANCE MANAGEMENT A. KEY PERFORMANCE INDICATOR (KPI) B. PERFORMANCE MEASUREMENT SYSTEM  Section 4.8 Review Questions

3 .

4.9

BALANCED SCORECARDS A. THE FOUR PERSPECTIVES 1.

The Customer Perspective 2.

The Financial Perspective 3.

The Learning and Growth Perspective 4.

The Internal Business Process Perspective B. THE MEANING OF BALANCE IN BSC C. DASHBOARDS VERSUS SCORECARDS  Section 4.9 Review Questions

4.10

SIX SIGMA AS A PERFORMANCE MEASUREMENT SYSTEM A. THE DMAIC PERFORMANCE MODEL B. BALANCED SCORECARD VERSUS SIX SIGMA C. EFFECTIVE PERFORMANCE MANAGEMENT ⧫ Application Case 4.8: Expedia.com’s Customer Satisfaction Scorecard  Section 4.10 Review Questions

Chapter Highlights Key Terms Questions for Discussion Exercises Teradata University and Other Hands-On Exercises Team Assignments and Role-Playing Projects ⧫ End of Chapter Application Case: Smart Business Reporting Helps Healthcare Providers Deliver Better Care  Questions for the Case References TEACHING TIPS/ADDITIONAL INFORMATION

       

This chapter is different from those that have gone before or will come in the future. It discusses largely management issues, with secondary attention (though not no attention!) to the technologies used to support them. Management students will take to such things as the BPM processes or Six Sigma like ducks to water; in fact, they may have already studied some of them in other courses. Technically oriented students may wonder why you’re wasting their time.

The fact is that a great deal of BI work today is focused on BPM and the other areas covered in this chapter. They are today’s version of what was once called an “executive information system,” highly evolved but conceptually similar. Other than the technology having changed, so has the way these systems are used. The old EIS was pretty much independent of everything else that went on in the company; it could be used, or not used, according to the whims of each executive. Today’s BPM software is integrated into the planning structure of the organization. Using it is not optional. As regards your students: understanding that structure isn’t optional either if they are to work with this kind of software.

4 .

If you happen to have mostly a technical background yourself, you may find some of the concepts in this chapter a bit alien and difficult to teach. If it’s practical in your situation, you may be able to prevail on one of your more business-oriented colleagues to take a class or two at this point.

From a pedagogic point of view, this chapter is more unified than some of the others because it covers a subject area which is itself inherently more unified. You won’t have to add as many transitions from one topic to the next; they’re already there. Students will probably find that it flows better, too. Compared to the earlier chapters, you (and they) will probably also find that the Questions for Discussion at the end involve more hard work!

Section 4.2, the first section after the opening vignette, introduces business reporting concepts. The role of reporting is shown in the overall context of information technology, business functions, and managerial decision making. Three main categories of reports are described (metric management, dashboard-type, and balanced scorecard).

You may find that this breakdown is different from what you’ve read in previous texts.

For example, reporting systems have often been delineated between static, ad hoc (custom), and exception reports. But in this chapter, the notion of “static” reports is deemphasized, and reports are generally thought of as highly interactive and graphical entities.

Section 4.3 gives a history of data and information visualization. Students will be interested to see how ancient the ideas behind visualization are. Spend some time talking about Minard’s graphic of Napoleon’s Russian campaign (Figure 4.3). Students can see how color coding, sizes of visual entities, and geographic overlays can combine to form a rich information visualization. Later, when discussing different types of charts and graphs from section 4.4, you can compare Minard’s visualization with the GapMinder example of Figure 4.4. Here, size also plays a role (compare the size of Minard’s band depicting army size with the size of bubbles depicting country populations). Another thing to do in this lecture would be to actually go to gapminder.com and watch the motion chart in action.

Section 4.5 discusses visual analytics, and distinguishes this topic from simple information visualization. Note that a sizable emphasis of visual analytics (as opposed to information visualization) is on its utility for prescriptive analytics (not just descriptive).

Keep in mind that prescriptive analytics topics aren’t discussed in depth until chapter 5, so students might not have enough context to truly appreciate the topics of this section until later. Another theme in this section is the importance of storytelling. Business reporting should tell compelling stories.

Section 4.6 explores dashboards, which leads naturally into section 4.7’s discussion of BPM. The basic BPM methodology breaks down into four processes, which are discussed in 4.7. Once students understand the processes, the rest of the chapter looks at specific aspects of them in more depth. In many cases these discussions are connected back to the four processes of BPM. In others, you might want to make the connection explicit yourself. Section 4.8 introduces performance measurement. This is followed in Sections 4.9 and 4.10 with discussions of Balanced Scorecard (BSC) and Six Sigma, respectively. It’s useful to compare these two and emphasize the strategic vs. operational nature of the two performance management approaches.

5 .

There are eight application cases in this chapter, and all bring some real-world examples that you can use to reinforce the concepts covered in this chapter. Many of the applications involve utilizing multiple reporting and performance management approaches and technologies.

ANSWERS TO END OF SECTION REVIEW QUESTIONS

     

Section 4.1 Review Questions 1.

What does Travel and Transport, Inc., do?

Travel and Transport, Inc., is the sixth largest travel management company in the United States, with more than 700 employee-owners located nationwide. The company has extensive experience in multiple verticals, including travel management, loyalty solutions programs, meeting and incentive planning, and leisure travel services.

2.

Describe the complexity and the competitive nature of the business environment in which Travel and Transport, Inc., functions.

The travel industry is a very crowded marketplace. With travel arrangements often made on an ad hoc basis, customers find it difficult to analyze costs or instate optimal purchase agreements.

3.

What were the main business challenges?

In the field of employee travel services, the ability to effectively communicate a value proposition to existing and potential customers is critical to winning and retaining business. With travel arrangements often made on an ad hoc basis, customers find it difficult to analyze costs or instate optimal purchase agreements.

Travel and Transport wanted to overcome these challenges by implementing an integrated reporting and analysis system to enhance relationships with existing clients, while providing the kind of value-added services that would attract new prospects.

4.

What was the solution? How was it implemented?

Travel and Transport implemented Information Builders’ WebFOCUS business intelligence (BI) platform (called eTTek Review) as the foundation of a dynamic customer self-service BI environment. This dashboard-driven expensemanagement application helps more than 800 external clients, and more than 200 internal employees, to plan, track, analyze, and budget their travel expenses more efficiently, and generate more precise forecasts. The approach was to create an online system that allows clients to access information directly, so they won’t have to rely on the IT department to run reports for them.

6 .

5.

Why do you think a multi-vendor, multi-tool solution was implemented?

Travel and Transport decided to combine multiple vendor tools, including Adobe Flex, WebFOCUS, DB2, and Global Distribution Systems (GDS). A combination of vendors and tools was required for their solution; each vendor brings a unique technological contribution to the mix. Adobe helps with the reporting features, WEBFOCUS contributes online BI capabilities, DB2 serves as the database backend, and GDS brings the domain expertise (i.e., travel agency knowledge) into the system.

6.

List and comment on at least three main benefits of the implemented system. Can you think of other potential benefits that are not mentioned in the case?

The implemented system benefits customers by enabling them to plan, track, analyze, and budget their travel expenses more efficiently. Customers are getting the best pricing from airlines, hotels, and car rental companies. For employees, including customer service representatives, the system helps with forecasting and quickly responding to client needs. Metrics derived from actionable data are key to helping Travel and Transport negotiate with their suppliers, enforce their travel policy, and save their company money.

Section 4.2 Review Questions 1.

What is a report? What are they used for?

A report is any communication artifact prepared with the specific intention of conveying information in a presentable form to whoever needs it, whenever and wherever they may need it. It is usually a document that contains information (usually driven from data and personal experiences) organized in a narrative, graphic, and/or tabular form, prepared periodically (recurring) or on an asrequired (ad hoc) basis, referring to specific time periods, events, occurrences, or subjects.

2.

What is a business report? What are the main characteristics of a good business report?

A business report is a written document that contains information regarding business matters. Business reporting (also called enterprise reporting) is an essential part of the larger drive toward improved managerial decision making and organizational knowledge management. The foundation of these reports is various sources of data coming from both inside and outside the organization.

Creation of these reports involves ETL (extract, transform, and load) procedures in coordination with a data warehouse and then using one or more reporting tools.

While reports can be distributed in print form or via e-mail, they are typically accessed via a corporate intranet. Primary characteristics of a good business report include clarity, brevity, completeness, and correctness.

7 .

3.

Describe the cyclic process of management and comment on the role of business reports.

The cyclic process of management, as illustrated in Figure 4.1, involves these steps: data acquisition leads to information generation which leads to decision making which leads to business process management. Perhaps the most critical task in this cyclic process is the reporting (i.e., information generation)— converting data from different sources into actionable information.

4.

List and describe the three major categories of business reports.

There are a wide variety of business reports, which for managerial purposes can be grouped into three major categories: metric management reports, dashboardtype reports, and balanced scorecard-type reports. Metric management reports involve outcome-oriented metrics based on service level agreements and/or key performance indicators. Dashboard-type reports present a range of performance indicators on one page, with both static/predefined elements and customizable widgets and views. Balanced scorecard reports present an integrated view of a company’s health and include financial, customer, business process, and learning/growth perspectives.

5.

What are the main components of a business reporting system?

A business reporting system includes several components. One is the online transaction processing system (ERP, POS, etc.) that records transactions. A second is a data supply that takes recorded events and transactions and delivers them to the reporting system. Next comes an ETL component that ensures quality and performs necessary transformations prior to loading the data into a data store.

Then there is the data storage itself (such as a data warehouse). Business logic converts the data into the reporting outputs. Publication distributes or hosts the reports for end users. And finally assurance provides a quality control check on the reports and their dissemination.

Section 4.3 Review Questions 1. What is data visualization? Why is it needed?

Data visualization, perhaps more appropriately called “information visualization,” is the use of visual representations to explore, make sense of, and communicate data. It is closely related to the fields of information graphics, scientific visualization, and statistical graphics. What is portrayed in visualizations is the information (aggregations, summarizations, and contextualization) and not the data. Companies and individuals increasingly rely on data to make good decisions. Because data is so voluminous, there is a need for visual tools that help people understand it.

8 .

2.

What are the historical roots of data visualization?

Predecessors to data visualization date back to the second century AD. Today’s most popular visual forms date back a few centuries. Geographical exploration, mathematics, and popularized history spurred the creation of early maps, graphs, and timelines as far back as the 1600s. The now familiar line and bar charts date back to the late 1700s. Charles Joseph Minard used visualizations to graphically portray the losses suffered by Napoleon’s army in the Russian campaign of 1812.

The 1900s saw the rise of a more formal, empirical attitude toward visualization, which tended to focus on aspects such as color, value scales, and labeling. In the 2000s the Internet has emerged as a new medium for visualization, and added interactivity to previously static graphics.

3.

Carefully analyze Charles Joseph Minard’s graphical portrayal of Napoleon’s march. Identify and comment on all of the information dimensions captured in this ancient diagram.

In this graphic Minard managed to simultaneously represent several data dimensions, including the size of the army, direction of movement, geographic locations, outside temperature, etc. He did this in an artistic and informative manner. The background of the image is a map depicting the location of battles.

There is a thick lighter band that shows the size of Napoleon’s army at each position, and a dark lower one that depicts the retreat. A line at the bottom depicts temperatures at each position in time and space.

4.

Who is Edward Tufte? Why do you think we should know about his work?

Edward Tufte is a statistician whose website chronicles many historical data visualizations, including Minard’s graphic of Napoleon’s defeat. His work can bring insights into how to follow best practices for information visualization.

5.

What do you think the “next big thing” is in data visualization?

The future of data/information visualization is very hard to predict. We can only extrapolate from what has already been invented: more three-dimensional visualization, more immersive experience with multidimensional data in a virtual reality environment, and holographic visualization of information. There is a pretty good chance that we will see something that we have never seen in the information visualization realm invented before the end of this decade.

9 .

Section 4.4 Review Questions 1.

Why do you think there are large numbers of different types of charts and graphs?

Different types of charts are appropriate for conveying different types of information. Line graphs are good for time-series data. Bar charts are good for depicting nominal or numerical data that can be easily categorized. Pie charts should be used for depicting proportions. Scatter plots and bubble charts are good for illustrating relationships between two or three variables (bubble charts add a dimension via the size of the dot). Histograms are like bar charts, except they depict frequency distributions. Gantt charts and PERT charts are good at illustrating project timelines and task dependencies. Geographic maps, of course, show geographic information. Bullet graphs show progress toward a goal. Heat maps and highlight tables illustrate the comparison of continuous values across two categories using color. Tree maps are good for showing hierarchical information. Even though these charts and graphs cover a major part of what is commonly used in information visualization, they by no means cover it all.

Nowadays, one can find many other specialized graphs and charts that serve a specific purpose.

2.

What are the main differences among line, bar, and pie charts? When should you choose to use one over the other?

Line graphs are good for time-series data. Bar charts are good for depicting nominal or numerical data that can be easily categorized. Pie charts should be used for depicting proportions. You shouldn’t use pie charts if the number of categories is very large.

3.

Why would you use a geographic map? What other types of charts can be combined with a geographic map?

Geographic maps are useful when the data set includes any kind of location data, including addresses, postal codes, state names or abbreviations, country names, latitude/longitude, or some type of custom geographic encoding. Maps can be used in conjunction with other charts and graphs. For instance, one can use maps to show distribution of customer service requests by product type (depicted in pie charts) by geographic locations.

4.

Find two more charts that are not covered in this section, and comment on their usability.

A concept map is a diagram that shows relationships between concepts, usually showing specific ideas and information as boxes and using arrows to connect them. Concept maps are often used by designers and engineers to organize ideas.

10 .

Another type of chart is an organization chart (or org-chart). This is a hierarchical, tree-structured chart that shows how an organization is structured and how its parts and jobs are related.

A motion chart is like a bubble chart in that it depicts data on dimensions of the xaxis, y-axis, size, and color of bubbles. In addition, however, it is also animated, so that bubbles move and resize themselves over time. The gapminder example from Figure 4.4 is an example of a motion chart.

Section 4.5 Review Questions 1.

What are the reasons for the recent emergence of visual analytics?

The growth of visual analytics correlates with the growth of analytics in general.

More BI and analytics vendors are becoming aware that their customers require quick and preferably interactive visualizations, not just for their normal reporting systems, but also to illustrate predictive and prescriptive decision-making information. Many of the information visualization vendors are adding the capabilities to call themselves visual analytics solution providers. Conversely, analytics solution providers such as SAS are embedding their analytics capabilities into a high-performance data visualization environment that they call visual analytics.

2.

Look at Gartner’s Magic Quadrant for Business Intelligence and Analytics Platforms. What do you see? Discuss and justify your observations.

What we can see from Gartner’s Magic Quadrant is that the vast majority of the “challengers” in the visual analytics space are considered to be “niche players,” while all of the “leaders” are “visionaries.” Most of the leaders are either relatively recently founded information visualization companies (e.g., Tableau Software, QlikTech, or Tibco Spotfire) or are well-established, large analytics companies (e.g., SAS, IBM, Microsoft, SAP, or MicroStrategy) that are increasingly focusing their efforts in information visualization and visual analytics. This chart, and the Gartner report from which it comes, shows that data discovery/visualization has become a mainstream architecture, perhaps surpassing the OLAP/semantic offerings of traditional BI vendors. This is further evidenced by the change of name of the magic quadrant from “BI” to “BI and Analytics.”

3.

What is the difference between information visualization and visual analytics?

Visual analytics is the combination of visualization and predictive analytics.

While information visualization is aimed at answering “what happened” and “what is happening” and is closely associated with business intelligence (routine reports, scorecards, and dashboards), visual analytics is aimed at answering “why is it happening,” “what is more likely to happen,” and is usually associated with business analytics (forecasting, segmentation, and correlation analysis).

11 .

4.

Why should storytelling be a part of your reporting and data visualization?

The central idea of business reporting is to tell a story. Everyone who has data to analyze has stories to tell, whether it’s diagnosing the reasons for manufacturing defects, selling a new idea in a way that captures the imagination of your target audience, or informing colleagues about a particular customer service improvement program. Stories bring life to data and facts. They can help you make sense and order out of a disparate collection of facts. They make it easier to remember key points and can paint a vivid picture of what the future can look like. Stories also create interactivity—people put themselves into stories and can relate to the situation. People will be much more engaged and receptive if information is presented to them in a story format.

5.

What is a high-powered visual analytics environment? Why do we need it?

A high-powered visualization environment is one in which high-performance, inmemory solutions are applied to exploring massive amounts of data in a very short time (almost instantaneously). Due to the increasing demand for visual analytics coupled with fast-growing data volumes, there is an ever-growing need to invest in highly efficient visualization systems. SAS Visual Analytics is an example of such an environment. These systems help to empower larger numbers of users, solve complex problems more quickly, and improve collaboration and information sharing. By enabling end-users, IT staff are freed up. In addition, these tools allow for growth at a self-determined pace.

Section 4.6 Review Questions 1.

What is a performance dashboard? Why are they so popular for BI software tools?

Performance dashboards provide visual displays of important information that is consolidated and arranged on a single screen so that information can be digested at a single glance and easily drilled in and further explored. They are common components of most, if not all, performance management systems, performance measurement systems, BPM software suites, and BI platforms. Dashboards pack a lot of information into a single screen, which is one reason for their popularity.

2.

What are the graphical widgets commonly used in dashboards? Why? Dashboards can include many kinds of visual widgets, including charts, performance bars, sparklines, gauges, meters, stoplights, geographic maps, etc.

These help to highlight, at a glance, the data and exceptions that require action. A picture tells a thousand words, and through the use of many graphical widgets, a dashboard can convey a wealth of information to decision makers in a short time.

12 .

3.

List and describe the three layers of information portrayed on dashboards.

The three layers of information found in most dashboards are: 1. Monitoring. Graphical, abstracted data to monitor key performance metrics.

2. Analysis. Summarized dimensional data to analyze the root cause of problems.

3. Management. Detailed operational data that identify what actions to take to resolve a problem.

4.

What are the common characteristics of dashboards and other information visuals?

All well-designed dashboards share some common characteristics. They use visual components (e.g., charts, performance bars, sparklines, gauges, meters, stoplights) to highlight, at a glance, the data and exceptions that require action.

They are transparent to the user, meaning that they require minimal training and are extremely easy to use. They combine data from a variety of systems into a single, summarized, unified view of the business. They enable drill-down or drillthrough to underlying data sources or reports, providing more detail about the underlying comparative and evaluative context. They present a dynamic, realworld view with timely data refreshes, enabling the end user to stay up to date with any recent changes in the business. And they require little, if any, customized coding to implement, deploy, and maintain.

5.

What are the best practices in dashboard design?

• • • • • • •

Benchmark key performance indicators with industry standards.

Wrap the dashboard metrics with contextual metadata (e.g. data source, data currency, refresh schedule).

Prioritize and rank alerts/exceptions streamed to the dashboard.

Enrich the dashboard with business users’ comments.

Present information in three different levels (visual dashboard, static report, and self-service cube).

Pick the right visual construct using dashboard design principles.

Provide for guided analytics.

Section 4.7 Review Questions 1.

What is business performance management? How does it relate to BI? Business performance management (BPM) refers to the business processes, methodologies, metrics, and technologies used by enterprises to measure, monitor, and manage business performance. It is also known as corporate 13 .

performance management (CPM), enterprise performance management (EPM), and strategic enterprise management (SEM). It can be considered to be a type of BI tool/technique. The most significant differentiator of BPM from any other BI tools and practices is its strategy focus. BPM encompasses a closed-loop set of processes that link strategy to execution in order to optimize business performance.

2.

What are the three key components of a BPM system?

According to Colbert (2009), a BPM encompasses three key components. The first is a set of integrated, closed-loop management and analytic processes (supported by technology) that addresses financial as well as operational activities. The second involves tools for businesses to define strategic goals and then measure and manage performance against those goals. And the third component involves a core set of processes, including financial and operational planning, consolidation and reporting, modeling, analysis, and monitoring of key performance indicators (KPIs), linked to organizational strategy.

3.

List and briefly describe the four phases of the BPM cycle.

The BPM cycle contains four main phases. First is to strategize. This involves answering the question, “Where do we want to go?”, and involves a high-level, long-term plan. Missions, visions, and objectives are key components of this phase. The second phase is to plan, which answers the question, “How do we get there?” Key elements here are a detailed operational plan and a financial plan including budget. The next phase is to monitor and analyze, which answers the question, “How are we doing?” Here is where KPIs, dashboards, reporting, and analytics are helpful. Finally come action and adjustment, based on comparing our analysis results against our plans. Sometimes this means changing the way we operate, and sometimes it means adjusting our strategy.

4.

Why is strategy the most important part of a BPM implementation?

Strategy is the art and the science of crafting decisions that help businesses achieve their goals. More specifically, it is the process of identifying and stating the organization’s mission, vision, and objectives. Business strategy provides an overall direction to the enterprise, which is why it is so important.

Section 4.8 Review Questions 1.

What is a performance management system? Why do we need one?

The purpose of a performance management system is to (a) identify and articulate the strategic mission, goals and objectives of an organization, and (b) assist managers in tracking the implementations of business strategy by comparing actual results against these strategic goals and objectives. The latter task is 14 .

accomplished by a performance measurement system, which can be considered a subset of the overall performance management system. A performance measurement system typically comprises systematic methods of setting business goals together with periodic feedback reports that indicate progress against goals.

This is a key and necessary element of the BPM process.

2.

What are the most distinguishing features of KPIs? The key features described in the book are strategy, targets, ranges, encodings, time frames, and benchmarks. KPIs embody strategic objectives and measure performance against specific targets, based on specified ranges of values.

Encodings provide visual cues (e.g., color) to indicate how close or far from a target we are on a particular metric. Benchmarks provide something to compare against.

3.

List and briefly define four of the most commonly cited operational areas for KPIs. • • • •

4.

Customer performance. Metrics for customer satisfaction, speed and accuracy of issue resolution, and customer retention.

Service performance. Metrics for service-call resolution rates, service renewal rates, service-level agreements, delivery performance, and return rates.

Sales operations. New pipeline accounts, sales meetings secured, conversion of inquiries to leads, and average call closure time.

Sales plan/forecast. Metrics for price-to-purchase accuracy, purchase order-tofulfillment ratio, quantity earned, forecast-to-plan ratio, and total closed contracts.

What is a performance measurement system? How does it work?

A performance measurement system is one component of a performance management system. The most popular performance measurement systems in use are some variant of Kaplan and Norton’s balanced scorecard (BSC).

Section 4.9 Review Questions 1.

What is a balanced scorecard (BSC)? Where did it come from?

Balanced Scorecard was first developed by Kaplan and Norton in their 1992 Harvard Business Review article, “The Balanced Scorecard: Measures That Drive Performance.” It is a performance management system whose key feature is that it does not rely solely on financial measures of success. Over the past few years, BSC has become a generic term that is used to represent virtually every type of scorecard application and implementation, but it is intended to emphasize a strategic focus.

15 .

2.

What are the four perspectives that BSC suggests us to use to view organizational performance?

The four perspectives are: customer, financial, internal business processes, and learning and growth. If customers are not satisfied, they will eventually find other suppliers that will meet their needs. Poor performance from this perspective is thus a leading indicator of future decline, even though the current financial picture may look good. Timely and accurate funding data will always be a priority, and managers will do whatever is necessary to provide it. This should include risk analysis. In the current climate of rapid technological change, it is becoming necessary for knowledge workers to be in a continuous learning and growing mode. Metrics based on this perspective allow the managers to know how well their internal business processes and functions are running, and whether the outcomes of these processes (i.e., products and services) meet and exceed the customer requirements (the mission).

3.

Why do we need to define separate objectives, measures, targets, and initiatives for each of these four BSC perspectives?

BSC is designed to overcome the limitations of systems that are financially focused. An organization’s vision and strategy should recognize the interrelation between financial and nonfinancial objectives, measures, targets, and initiatives.

Therefore, nonfinancial objectives form a simple causal chain with “learning and growth” driving “internal business process” change, which produces “customer” outcomes that are responsible for reaching a company’s “financial” objectives.

4.

What is the meaning of and motivation for balance in BSC? In BSC, the term “balance” arises because the combined set of measures is supposed to encompass indicators that are financial and nonfinancial, leading and lagging, internal and external, quantitative and qualitative, and both short term and long term.

5.

What are the differences and commonalities between dashboards and scorecards?

These terms are often used interchangeably, and they share many common features. The main difference is that scorecards are used by executives, managers, and staff to monitor strategic alignment and success with strategic objectives and targets. By contrast, dashboards are used at the operational and tactical levels.

Managers, supervisors, and operators use operational dashboards to monitor detailed operational performance on a weekly, daily, or even hourly basis.

16 .

Section 4.10 Review Questions 1.

What is Six Sigma? How is it used as a performance measurement system?

Most companies use Six Sigma as a process improvement methodology that enables them to scrutinize their processes, pinpoint problems, and apply remedies.

It's not used much as a performance management or measurement methodology.

As a performance tool, it is aimed at reducing the number of defects in a business process to as close to zero DPMO (defects per million opportunities) as possible.

2.

What is DMAIC? List and briefly describe the steps involved in DMAIC. DMAIC is a closed loop performance improvement model that involves the following steps: define, measure, analyze, improve, and control. First, you define the goals, objectives, and boundaries of the improvement activity. Next, you measure the existing system, in order to monitor its performance against the goals.

Then, you analyze the system to identify ways to eliminate the gap between the current performance of the system or process and the desired goal. This leads to improvement, which involves initiating actions to reduce these gaps. Finally, control involves modifying compensation and incentive systems, policies, procedures, manufacturing resource planning, budgets, operation instructions, or other management systems.

3.

Compare BSC and Six Sigma as two competing performance measurement systems.

BSC is focused on improving overall strategy, whereas Six Sigma is focused on improving processes. BSC gives a longer-term view of the organization, whereas Six Sigma gives a snapshot at a particular point in time of its operational effectiveness. BSC focuses on long-term growth, whereas Six Sigma emphasizes current profitability. These are a few of the differences between the two. Some companies choose to give a more holistic performance assessment by combining elements of both approaches.

4.

What are the ingredients for an effective performance management system?

Effective performance management/measurement should focus on key factors. It should mix past, present, and future. Also, it should balance the needs of shareholders, employees, partners, suppliers, and other stakeholders. Performance measures should start at the top and flow to the bottom, and should involve targets that are based on research and reality rather than arbitrary.

17 .

ANSWERS TO APPLICATION CASE QUESTIONS FOR DISCUSSION

 

Application Case 4.1: Delta Lloyd Group Assures Accuracy and Efficiency in Financial Reporting 1.

How did Delta Lloyd Group improve accuracy and efficiency in financial reporting?

They switched from a manual, spreadsheet-based process to utilizing IBM Cognos for their financial reporting.

2.

What were the challenges, the proposed solution, and the obtained results?

The most challenging part of the reporting process is the “last mile”—the stage at which the consolidated figures are cited, formatted, and described to form the final text of the report. They used to do this with Microsoft Excel and manual processes, which is very time consuming and error-prone. Accuracy is essential in financial reporting, since errors could lead to penalties, reputational damage, and even a negative impact on the company’s stock price. The solution was to implement IBM Cognos Financial Statement Reporting (FSR). This allowed the workload to be divided up, so that staff were able complete the report with 25% less overtime. It also allowed more flexibility; staff members could update figures automatically across the whole report which reduced the scope for manual error inherent in spreadsheet-based processes and activities.

3.

Why is it important for Delta Lloyd Group to comply with industry regulations?

Since Delta Lloyd Group is publicly listed on the NYSE Euronext Amsterdam, it is obliged to produce annual and half-year reports. Their subsidiaries must also produce reports to fulfill local legal requirements for banking and insurance purposes. They must provide reports to meet international requirements, such as the IFRS (International Financial Reporting Standards).

Application Case 4.2: Flood of Paper Ends at FEMA 1.

What is FEMA and what does it do?

The Federal Emergency Management Agency (FEMA) is a U.S. federal agency that coordinates disaster response when the President declares a national disaster.

2.

What are the main challenges that FEMA faces?

18 .

When a disaster occurs, FEMA is inundated with a huge amount of paperwork to sift through in order to administer the National Flood Insurance Program (NFIP).

Sifting through this paperwork is very cumbersome and labor-intensive.

3.

How did FEMA improve its inefficient reporting practices?

The solution was to implement a system based on WebFOCUS software from Information Builders. As a result, FEMA staff can now browse insurance data posted on NFIP’s BureauNet intranet site, select just the information they want to see, and get an onscreen report or download the data as a spreadsheet. This also allows them to create custom reports without having to go through their IT provider, CSC. The first major test of this system was Tropical Storm Allison, and BureauNet worked very well. It also has been able to scale up to handle increased demand.

Application Case 4.3: Tableau Saves Blastrac Thousands of Dollars with Simplified Information Sharing 1.

How did Blastrac achieve significant cost saving in reporting and information sharing?

They switched from a manual, Excel-based approach of reporting to using Tableau Software for data visualization and reporting.

2.

What were the challenge, the proposed solution, and the obtained results?

Blastrac depends on the creation and distribution of reports across the organization to make business decisions, but had no consistent reporting method in place. This caused them to spend a lot of time extracting data from the multiple ERP systems, loading it into several Excel spreadsheets, creating filtering capabilities, and establishing predefined pivot tables. This led to many errors. An enterprise BI system would have been too costly. So, they deployed two data visualization tools from Tableau Software (Tableau Desktop to create reports and Tableau Reader to view and filter them). As a result, only one hour per week is now required to create reports. Also, updates to these reports happen automatically through Tableau. This has resulted in a 95 percent increase in productivity.

Application Case 4.4: TIBCO Spotfire Provides Dana-Farber Cancer Institute with Unprecedented Insight into Cancer Vaccine Clinical Trials 1.

How did Dana-Farber Cancer Institute use TIBCO Spotfire to enhance information reporting and visualization?

19 .

TIBCO Spotfire was used to conduct data exploration and discovery regarding data from clinical trials that had been recorded in an XML-based data registry.

This automated process was much more effective and efficient than manual, spreadsheet-based data analysis.

2.

What were the challenges, the proposed solution, and the obtained results?

Dana Farber needs to study data from many sources in order to understand the relationships underlying the data and identify trends and patterns from clinical vaccine trials. This data was obtained from 645 clinical trials related to cancer vaccines. The data was extracted in XML from the ClinicalTrials.gov website.

There is great inconsistency and redundancy in this data registry. It was too difficult and time consuming to analyze information from the multiple data sources separately using spreadsheets. The solution was to use TIBCO Spotfire DXP (Spotfire) computational and visual analysis tool for data exploration and discovery. This solution provides the Dana-Farber and the research community at large with a better understanding of the cancer vaccine clinical trials landscape and enables rapid insight into the hotspots of cancer vaccine activity, as well as into the identification of neglected cancers.

Application Case 4.5: Dallas Cowboys Score Big with Tableau and Teknion 1.

How did the Dallas Cowboys use information visualization?

They incorporated Tableau and Teknion to assist with visualizing and understanding their merchandising activities, involving the complete supply chain from manufacture to end customer.

2.

What were the challenge, the proposed solution, and the obtained results?

The Dallas Cowboys Merchandising Division needed more visibility into their data so they could run more profitably. Microsoft was selected as the baseline platform that had been selected was not sufficient for the task. Tableau and Teknion together provided real-time reporting and dashboard capabilities that provided the necessary visualization functionality to meet and exceed the Cowboys’ requirements. Now, for the first time, the Dallas Cowboys are able to monitor their complete merchandising activities from manufacture to end customer and see not only what is happening across the life cycle, but drill down even further into why it is happening.

Application Case 4.6: Saudi Telecom Company Excels with Information Visualization

20 .

1.

Why do you think telecommunication companies are among the prime users of information visualization tools?

Telecommunications companies typically have millions of customers, contacting them constantly for billing, payment, network usage, and support. All of this information has to be monitored in a timely fashion.

2.

How did Saudi Telecom use information visualization?

STC used visualization to monitor information regarding network statistics, service analytics, and customer calls. Part of this requires prioritizing and contextualizing the data, identifying the relevant metrics, and viewing KPIs.

3.

What were their challenges, the proposed solution, and the obtained results?

STC needed to identify the relevant metrics, properly visualize them, and provide them to the right people, often with time-sensitive information. But executives didn’t have the ability to see key performance indicators in a timely fashion. They would have to contact the technical teams to get status reports, which often came too late and to be of benefit for preventing problems before they happen. The solution was to contract with Dundas business intelligence consultants to refine the telecommunication dashboards and improve their functionality. This led to engagement on an enterprise-wide, mission-critical project to transform their data center and create a more proactive monitoring environment. Dundas’ information visualization tools allowed STC to see trends and correct issues before they became problems. This resulted in a decrease in the amount of service tickets by 55 percent.

Application Case 4.7: IBM Cognos Express Helps Mace for Faster and Better Business Reporting 1.

What was the reporting challenge Mace was facing? Do you think this is an unusual challenge specific to Mace?

Many of Mace’s international projects are contracted on a fixed-price basis, and their success depends on the company’s ability to control costs and maintain profitability. Until recently, the only way for senior managers to gain a full understanding of international operations was a monthly report, based on a complex spreadsheet which drew data from numerous different accounting systems in subsidiaries around the world. This spreadsheet had about 40 tabs and hundreds of cross-links, which meant it was very easy to introduce errors, and the lack of a standardized approach was affecting accuracy and consistency.

21 .

This is not a unique problem for Mace. Many businesses face similar issues, especially those with an international component who contract on fixed-price basis like Mace.

2.

What was the approach for a potential solution?

The solution was to use IBM Cognos Express as the foundation of their international reporting platform; they were already using Cognos for their domestic business. The implementation was completed within six months.

3.

What were the results obtained in the short term, and what were the future plans?

The resulting system produces much more useful and dynamic reports, including drill-down capabilities. Reporting is automated and easily customizable. Mace is now in a better position to monitor performance and maintain profitability while ensuring that their projects are delivered on time and within budget. In the future, Mace is working to unite all its UK and international subsidiaries into this single financial reporting and budgeting system. The company also plans to continue working with IBM to explore other possible benefits of Cognos for facilitating standardization and financial control.

Application Case 4.8: Expedia.com’s Customer Satisfaction Scorecard 1.

Who are the customers for Expedia.com? Why is customer satisfaction a very important part of their business?

Expedia, Inc., is the parent company to some of the world’s leading travel companies (hotels.com, hotwire.com, expedia.com). Therefore, it acts as an online travel agent for consumers who want to travel. Because Expedia.com is an online business, the customer’s shopping experience is critical to Expedia’s revenues.

The online shopping experience can make or break an online business. Obviously, the travel experience is also critical to Expedia’s success.

2.

How did Expedia.com improve customer satisfaction with scorecards?

Scorecards give Expedia much better insight into customer satisfaction by providing staff, managers and executives instant visualization and reporting of patterns and trends. Through KPIs, the scorecard also allows for comparison of the customer patterns and trends to Expedia’s corporate goals and objectives.

3.

What were the challenges, the proposed solution, and the obtained results?

Because the customer experience is critical, all customer issues need to be tracked, monitored, and resolved as quickly as possible. But Expedia had no uniform way of measuring satisfaction, of analyzing the drivers of satisfaction, or 22 .

of determining the impact of satisfaction on the company’s profitability or overall business objectives. However, there was plenty of data to work with. It took a business analyst two to three weeks every month to pull and aggregate the data, leaving virtually no time for analysis. Expedia’s solution was to start by identifying key drivers of customer satisfaction. From these drivers, the customer satisfaction group constructed a scorecard application. They came up with 10 to 12 objectives that linked directly to Expedia’s corporate initiatives, which translated into more than 200 KPIs. This involved three steps: deciding how to measure satisfaction, setting performance targets, and putting the data into context. All of this was applied to their data warehouse (which they called DSS Factory). As a result, managers and executives have a transparent view of how well actions are aligning with the strategy, with the ability to drill down into the data underlying any of the trends or patterns observed. This would have taken weeks to do previously. Other business units are also benefitting.

ANSWERS TO END OF CHAPTER QUESTIONS FOR DISCUSSION

  

1. What are the best practices in business reporting? How can we make our reports stand out?

Business reporting is about good story-telling. Here are some best practices for telling compelling stories with your business report: Think of your analysis as a story—use a story structure. Be authentic—your story will flow. Be visual—think of yourself as a film editor. Make it easy for your audience and you. Invite and direct discussion.

Business reports are often presented in the form of a dashboard or scorecard.

Following are some best practices of dashboard design: • Benchmark key performance indicators with industry standards.

• Wrap the dashboard metrics with contextual metadata (e.g. data source, data currency, refresh schedule).

• Prioritize and rank alerts/exceptions streamed to the dashboard.

• Enrich the dashboard with business users’ comments.

• Present information in three different levels (visual dashboard, static report, and self-service cube).

• Pick the right visual construct using dashboard design principles.

• Provide for guided analytics.

2.

Why has information visualization become a centerpiece in the business intelligence and analytics business? Is there a difference between information visualization and visual analytics?

A picture tells a thousand words. Hence, information visualization is key to quickly providing understanding of complex information. This has been true for hundreds of years, as evidenced by Minard’s graphical depiction of Napoleon’s 23 .

defeat. In an increasingly complex information age, visualization is even more important. Since BI and analytics are all about making sense of high volume, fastpaced, in interrelated data, it is natural that information visualization would be a centerpiece. BI companies are rapidly adding information visualization and visual analytics into their products and services.

Information visualization and visual analytics are related but distinct concepts.

Visual analytics is the combination of visualization and predictive analytics.

While information visualization is aimed at answering “what happened” and “what is happening” and is closely associated with business intelligence (routine reports, scorecards, and dashboards), visual analytics is aimed at answering “why is it happening,” “what is more likely to happen,” and is usually associated with business analytics (forecasting, segmentation, and correlation analysis).

3.

Do you think performance dashboards are here to stay? Or are they about to be outdated? What do you think will be the next big wave in business intelligence and analytics?

Like all technology, performance dashboards will evolve and morph over time.

Dashboards are not a new concept. Their roots can be traced at least to the EIS of the.1980s. Today, dashboards are ubiquitous. Currently they are used as reporting tools, largely for the purpose of evaluating a company’s performance, and are often tied with business performance methodologies like Balanced Scorecard or Six Sigma. The specific visualizations in a dashboard (charts, graphs, and maps) will evolve over time, and the technology platforms may change. For example, with virtual reality, holographic and immersive technologies, the user experience may change considerably. But the purpose of a dashboard to monitor company performance and respond to warnings and events will continue. (Answers will vary by student.)

4.

SAP uses the term strategic enterprise management (SEM), Cognos uses the term corporate performance management (CPM), and Hyperion uses the term business performance management (BPM). Are they referring to the same basic ideas?

Provide evidence to support your answer.

In the business and trade literature, business performance management (BPM) has a number of names, including corporate performance management (CPM), enterprise performance management (EPM), and strategic enterprise management (SEM). CPM was coined by the market analyst firm Gartner (gartner.com). EPM is a term associated with Oracle’s (oracle.com) offering by the same name. SEM is the term that SAP (sap.com) uses. All of these terms refer to the same general idea. They all encompass a closed-loop set of processes that link strategy to execution in order to optimize business performance.

5.

BPM encompasses five basic processes: strategize, plan, monitor, act, and adjust.

Select one of these processes and discuss the types of software tools and applications that are available to support it. Figure 4.10 provides some hints. Also, 24 .

refer to Bain & Company’s list of management tools for assistance (bain.com/management_tools/home.asp).

(Students will select a single process. Below is one paragraph for each of the first three processes; Act/Adjust is usually something that involves human decision making as a result of the previous three phases.)

Strategize: One type of software for the strategize process involves applications that assist with the development of strategy maps. These applications are often components of a BSC software. Strategy map software enables users to build diagrams that tie objectives to different perspectives (for example, the BSC perspectives of customer, finance, process, and learning/growth). There are many products on the market that assist with strategy map formulation, including SmartDraw, Corporater, and StrategyKpi. Plan: The planning process involves creating budgets, project plans, forecasts, models, initiatives, and targets. Budgeting software can be as simple as Excel spreadsheets, or can involve accounting packages such as Quicken or Quickbooks or Peachtree. Here is a link to an article describing various options for budgeting software: http://www.forbes.com/sites/moneywisewomen/2012/01/03/budgetingsoftware-options/. Planning software will include project management tools such as Microsoft Project. Forecasting tools typically involve predictive analytics, and there are many products that combine budgeting and forecasting together, as described in this article: http://www.softwareadvice.com/accounting/budgetingforecasting-software-comparison/.

Monitor/Analyze: Monitoring and analyzing typically involves dashboards, scorecards, and visual analytics tools. Dashboards provide graphical visuals of various types. Gartner’s Magic Quadrant shows several companies that either specialize in dashboards and information visualization (e.g., Tableau Software, QlikTech, and Tibco Spotfire) or incorporate these tools into broader BI and analytics offerings (e.g., SAS, IBM, Microsoft, SAP, and MicroStrategy) 6.

Select a public company of interest. Using the company’s 2013 annual report, create three strategic financial objectives for 2014. For each objective, specify a strategic goal or target. The goals should be consistent with the company’s 2013 financial performance.

(This question will obviously vary significantly per student. A good approach to answering this question is to do what Figure 4.12 does, by combining a strategy map with a balanced scorecard.)

25 .

7.

Netflix’s strategy of moving to online video downloads has been widely discussed in a number of articles that can be found online. What are the basic objectives of Netflix’s strategy now? What are some of the major assumptions underlying the strategy? Given what you know about discovery-driven planning, do these assumptions seem reasonable?

(Student answers will vary. The following answer is based primarily on the following sources: https://lpelin.expressions.syr.edu/trf483/files/2013/05/final_strategic_report.pdf, http://ir.netflix.com/long-term-view.cfm, http://www.cnbc.com/id/100905886, and http://www.strategy-business.com/article/cs00003?pg=all) Netflix started as a mail-order DVD rental business, in which customers would pay for each DVD they rented. This was sort of like the Blockbuster model, except that it was done remotely. Netflix did this without a brick-and-mortar store. Instead, customers would order videos online and receive them in the characteristic red envelope. Also, Netflix eschewed fines and late fees, which gave them a jump up in customer loyalty.

As Web technologies evolved and ISP speeds improved, Netflix shifted its primary delivery to online streaming. The DVD side of the business has declined significantly, and in fact they attempted to split that off into a separate company called Qwikster, a move that irritated many of its customers and even led to a lawsuit.

Another major shift with Netflix was to begin developing their own content. The award-winning series House of Cards is an example. The delivery of this show was also innovative; rather than a week-by-week delivery, they made a whole season available at once, giving customers the ability to view the show at their own pace.

Netflix’s 2013 strategy document includes a SWOT analysis, which sheds light on the underlying assumptions driving their top decision makers’ thinking. The major challenges they face are (a) high cost and debt for acquiring content from providers, (b) significant losses in their physical media delivery (DVD mailings), (c) potentially higher ISP costs, and (d) competitors such as Amazon and Hulu.

By contrast, their strengths and opportunities include (a) strong brand identity, (b) success with their early attempts at providing original content, and (c) a rising stock price.

Netflix has taken advantage of Web analytics to track customer behavior patterns.

In this way they are using discovery-driven planning. To illustrate, here is a quote from their 2014 strategic plan: “Another area of focus is personalized recommendations and merchandising, which drives what content we feature on a member’s initial screen. Google search is an example of a ranking system, where 26 .

results are automatically computed to show Google’s estimate of the most relevant answer to the query. For Netflix, the member’s opening screen is the personalized ranking of what we think will be most relevant for that specific member selected from our diverse catalog. By analyzing terabytes of data from every recent click, view, repeat view, early abandon, page views and other data, we are able to generate a personalized display with the content most likely to please. Our aim is to keep inventing and tuning algorithms to generate higher satisfaction, viewing, and retention.”

Clearly, Netflix is embracing analytics in order to refine their marketing efforts as well as their overall business strategy. This seems to help them stay agile and competitive as top players in the entertainment industry.

8.

In recent years, the Beyond Budgeting Round Table (BBRT; bbrt.org) has called into question traditional budgeting practices. A number of articles on the Web discuss the BBRT’s position. In the BBRT’s view, what is wrong with today’s budgeting practices? What does the BBRT recommend as a substitute?

BBRT’s “beyond budgeting” is an idea that budgeting is an inherently flawed process and that companies should move beyond budgeting. Companies should instead employ a range of techniques, such as rolling forecasts, to replace traditional budgeting. In this sense, “beyond budgeting” is considered to be more adaptive and less centralized than traditional budgeting.

BBRT advocates claim that this helps alleviate several drawbacks of traditional budgeting, which are time consuming, expensive, and rarely focus on strategy.

Traditional budgets limit flexibility and often discourage change, and focus too much on reducing costs rather than creating value.

With BBRT, “budget” does not refer solely to money expenditures, but to the entire performance management process. In this sense it shares a common philosophy with BSC. The BBRT approach involves setting clear priorities and boundaries, with empowered teams that take responsibility for value creating systems.

(NOTE: This information comes from http://www.cimaglobal.com/Documents/ImportedDocuments/cid_tg_beyond_bud geting_oct07.pdf)

9.

Distinguish between performance management and performance measurement.

There is a difference between a performance measurement system and a performance management system. The latter encompasses the former. That is, any performance management system has a performance measurement system, but not the other way around. If you were to ask, most companies today would claim that they have a performance measurement system but not necessarily a performance management system, even though a performance measurement system has very 27 .

little, if any, use without the overarching structure of the performance management system. Probably the best-known and most widely used performance management system is the balanced scorecard (BSC).

The purpose of a performance measurement system is to assist managers in tracking the implementations of business strategy by comparing actual results against strategic goals and objectives. A performance measurement system typically comprises systematic methods of setting business goals together with periodic feedback reports that indicate progress against goals. This is a key and necessary element of the BPM process.

10.

Create a measure for some strategic objective of interest (you can use one of the objectives formulated in discussion question 6). For the selected measurement, complete the measurement template found in Table W4.2.1 in the online file for this chapter.

(This question will vary significantly per student. A good approach to answering this question is to do what Figure 4.12 does, by combining a strategy map with a balanced scorecard. Here is an example.)

Such an objective for a company entering a brand new market might be a market share increase from 20 to 30 percent. For that objective, a possible measurement template might look like this:

Measurement Name Purpose

Property What is it called, and is it understandable?

Why is it being used, and what actions is it encouraging?

Relates to

What strategies does it support, and what other measures is it linked to?

Metric/formula

What is the actual measurement?

Target level(s) Frequency

What is the desirable level of performance?

How often is the measurement made and reported?

Where do the data come from?

Source of data Who measures

Who acts on the data (owner)

What are the name and function of the person responsible for collecting, collating, and analyzing the data?

Who—name and function—is responsible for ensuring the performance levels?

28 .

Explanation Market share increase; yes Used to measure improvement in our competitive position in our industry; encourages sales Supports strategy of expanding market penetration to obtain new customers while market is still in early growth stages; linked to shipments Ratio of sales revenue to total market size using public industry data 30 percent Monthly Our data: accounting; industry data: IDC John Doe, Business Analyst

Jane Roe, VP of Sales

Measurement What do they do

Property How will the owner use the data, and what actions can he or she take to improve performance?

Efficiency

Are all the data that are being collected used?

Are there easier ways to collect the data?

Is there any duplication among the measures?

Can the amount of data being collected be reduced through sampling?

Can the frequency of collection or reporting be reduced?

Effectiveness

11.

Can the measure be eliminated because it is no longer being used?

Are there new tools that can be used to produce the measure in new ways?

Are all key processes adequately measured?

Are proper process measures in place to keep performance within acceptable limits?

Is everyone getting the information they need?

Does everyone understand the measures so they can interpret the results?

Are users taking action as a result of the measures?

Can the measures be easily used?

Are the measures sufficiently timely and accurate?

Does everyone have access to the measures they need?

Is the relative impact of different measures clear to users?

Have the measures been changed to reflect changes in processes?

Explanation If the data do not show an increase in market share tending toward at least 30 percent, she will use this as evidence of a need to develop new sales programs to achieve it They will be No, as far as we know No No; it must be collected for other reasons in any case No; monthly collection frequency is required for other reasons No; it is being used No Yes Yes Yes Yes Yes Yes Yes Yes Yes N/A, processes have not changed

Using the four perspectives of the BSC, create a strategy for a hypothetical company. Express the strategy as a series of strategic objectives. Produce a strategy map depicting the linkages among the objectives.

(This answer will vary significantly depending on the hypothetical company and industry that the student selects. The key here is for students to discuss the company in terms of customer, financial, process, and learning/growth perspectives. For each of these perspectives, students should come up with objectives, linked together in a strategy map. These objectives can then be

29 .

translated into measures and targets, and finally action plans. Figure 4.12 is a useful guide for students to follow.)

One example of a hypothetical company is Microsoft’s Adventure Works sample, found at http://technet.microsoft.com/en-us/library/ms124825(v=sql.100).aspx.

Adventure Works is a fictional bicycle manufacturing company. In their case, the primary business process will be the manufacture and sale of bicycles, and many measures can be incorporated, including measures of waste and scrap. Adventure Works sells primarily to retail stores and secondarily to individual customers through its online store (they do not have brick-and-mortar stores of their own).

Questions of customer retention therefore have two sub-perspectives (retail stores and individuals). Many factors can be brought to bear, including sales/marketing, supply chain and purchasing, and human resource concerns. Adventure Works is a very useful case for a BI class because it is used extensively in SqlServer’s sample database and data warehouse, as well as including many online tutorials for a wide range of data integration, OLAP, and data mining topics that are relevant for this textbook.

12.

Compare and contrast the DMAIC model with the closed-loop processes of BPM. The DMAIC model (the performance improvement model used in the Six Sigma methodology) and BPM are both closed-loop models. However, their purposes are different: BPM is for overall monitoring of an organization, while the DMAIC model is focused on improving performance of a process or area. Their emphasis and the resulting processes therefore differ. BPM has four processes (strategize, plan, monitor, act and adjust) while the DMAIC model has five (define, measure, analyze, improve, control). Specific differences between them include: • • •

13.

BPM takes a more strategic approach. It begins with strategizing. DMAIC takes strategic objectives as given, and continues from there.

The Plan process of BPM corresponds to the Define process of the DMAIC model.

The DMAIC model, as fits its purpose, goes into more detail in the later (implementation) stages of the improvement process, which BPM lumps under “act and adjust.”

Select two companies that you are familiar with. What terms do they use to describe their BPM initiatives and software suites? Compare and contrast their offerings in terms of BPM applications and functionality.

(This will vary widely based on students’ experience and preference. Students should be able to relate the topics from sections 4.7-4.10. BPM phases of Strategize, Plan, Monitor/Analyze, and Act/Adjust should be applied to each company (as illustrated in Figure 4.10). Students who can compare the two companies in terms of both BSC and Six Sigma will have demonstrated a deep understanding of BPM. By tying in specific applications such as strategy maps,

30 .

dashboards, and scorecards, students will show that they can speak to the technological implementations of the BPM systems they describe.)

ANSWERS TO END OF CHAPTER APPLICATION CASE QUESTIONS 1.

 

What is Premier? What does it do?

Premier is a healthcare services company that serves more than 2,600 U.S. hospitals and 84,000-plus other healthcare sites. It exists to help its members improve the cost and quality of the care they provide the communities they serve.

Premier also assists its members to prepare for and stay ahead of health reform, including accountable care and other new models of care delivery and reimbursement.

2.

What were the main challenges for Premier to achieve its vision?

Premier had developed a series of “siloed” applications, making it difficult for members to connect different data sources and metrics and see the “big picture” of how to drive healthcare transformation. These platforms and associated software systems also lacked the scalability required to support the massive transaction volumes that were needed. The company’s existing technical infrastructure could not support their needs for better integration.

3.

What was the solution provided by IBM and other partners?

Premier “re-platformed” their IT architecture to enable the sharing and analysis of data from its thousands of member organizations. The new data architecture and infrastructure uses IBM software and hardware, including a data model called IBM Healthcare Provider Data Warehouse (HCPDW), based on IBM DB2 pureScale. The organization’s IT division is building its new infrastructure from the ground up. This includes replacing its existing x86 servers from a variety of hardware vendors with IBM POWER7 processor-based systems. Data from IBM DB2 database software is loaded to an IBM Netezza data warehouse so that members can conduct advanced and rapid analytics.

The new platform is designed to deliver trusted information in the right context at the right time to users based on their roles. Using the new platform, Premier members will be able to use the portal to access the integrated system for various clinical, business, and compliance-related applications. This helps healthcare organizations provide their staff with accurate and timely information to support evidence-based, patient-centric, and accountable care.

31 .

4.

What were the results? Can you think of other benefits coming from such an integrated system?

Potential benefits for saving lives, helping people enjoy healthier lives, and reducing healthcare costs are enormous. In one Premier project, 157 participating hospitals saved an estimated 24,800 lives while reducing healthcare spending by $2.85 billion. Over the next 5 years, Premier plans to provide its members with many new applications in support of healthcare reform and other legislative requirements. As capabilities are added, the SaaS model will enable the platform to support its commitment to keep all 2,600 hospital members on the same page, and even expand its user community.

32 .

Chapter 5: Data Mining

Learning Objectives for Chapter 5 1.

2.

3.

4.

5.

6.

7.

8.

Describe data mining as an enabling technology for business analytics Understand the objectives and benefits of data mining Become familiar with the wide range of applications of data mining Learn the standardized data mining processes Understand the steps involved in data preprocessing for data mining Learn different methods and algorithms of data mining Build awareness of the existing data mining software tools Understand the privacy issues, pitfalls, and myths of data mining

CHAPTER OVERVIEW Generally speaking, data mining is a way to develop intelligence (i.e., actionable information or knowledge) from data that an organization collects, organizes, and stores.

A wide range of data mining techniques are being used by organizations to gain a better understanding of their customers and their own operations and to solve complex organizational problems. In this chapter, we study data mining as an enabling technology for business analytics, learn about the standard processes of conducting data mining projects, understand and build expertise in the use of major data mining techniques, develop awareness of the existing software tools, and explore privacy issues, and common myths and pitfalls that are often associated with data mining.

CHAPTER OUTLINE 5.1

OPENING VIGNETTE: CABELA’S REELS IN MORE CUSTOMERS WITH ADVANCED ANALYTICS AND DATA MINING  Questions for the Opening Vignette A. WHAT WE CAN LEARN FROM THIS VIGNETTE

1 .

5.2

DATA MINING CONCEPTS AND APPLICATIONS ⧫ Application Case 5.1: Smarter Insurance: Infinity P&C Improves Customer Service and Combats Fraud with Predictive Analytics A. DEFINITIONS, CHARACTERISTICS, AND BENEFITS ⧫ Technology Insights 5.1: A Simple Taxonomy of Data ⧫ Application Case 5.2: Harnessing Analytics to Combat Crime: Predictive Analytics Helps Memphis Police Department Pinpoint Crime and Focus Police Resources B. HOW DATA MINING WORKS 1.

Prediction 2.

Classification 3.

Clustering 4.

Associations 5.

Visualization and Time-Series Forecasting C. DATA MINING VERSUS STATISTICS  Section 5.2 Review Questions

5.3

DATA MINING APPLICATIONS ⧫ Application Case 5.3: A Mine on Terrorist Funding  Section 5.3 Review Questions

5.4

DATA MINING PROCESS A, Step 1: Business Understanding B. Step 2: Data Understanding C. Step 3: Data Preparation D. Step 4: Model Building ⧫ Application Case 5.4: Data Mining in Cancer Research E. Step 5: Testing and Evaluation F. Step 6: Deployment G. OTHER DATA MINING STANDARDIZED PROCESSES AND METHODOLOGIES  Section 5.4 Review Questions

5.5

DATA MINING METHODS A. CLASSIFICATION B. ESTIMATING THE TRUE ACCURACY OF CLASSIFICATION MODELS 1.

Simple Split 2. k-Fold Cross-Validation 3.

Additional Classification Assessment Methodologies 4.

Classification Techniques 5.

Decision Trees ⧫ Application Case 5.5: 2degrees Gets a 1275 Percent Boost in Churn Identification C. CLUSTER ANALYSIS FOR DATA MINING 2 .

D.

1.

Determining the Optimal Number of Clusters 2.

Analysis Methods 3.

K-Means Clustering Algorithm ASSOCIATION RULE MINING 1.

Apriori Algorithm  Section 5.5 Review Questions

5.6

DATA MINING SOFTWARE TOOLS ⧫ Application Case 5.6: Data Mining Goes to Hollywood: Predicting Financial Success of Movies  Section 5.6 Review Questions

5.7

DATA MINING PRIVACY ISSUES, MYTHS, AND BLUNDERS A. DATA MINING AND PRIVACY ISSUES ⧫ Application Case 5.7: Predicting Customer Buying Patterns—The Target Story B. DATA MINING MYTHS AND BLUNDERS  Section 5.7 Review Questions

Chapter Highlights Key Terms Questions for Discussion Exercises Teradata University Network (TUN) and Other Hands-on Exercises Team Assignments and Role-Playing Projects Internet Exercises ⧫ End of Chapter Application Case: Macys.com Enhances Its Customers’ Shopping Experience with Analytics  Questions for the Case References

TEACHING TIPS/ADDITIONAL INFORMATION

       

Data mining was mentioned briefly in previous chapters. Students have encountered the term several times but will only have a general idea of what it means. By the time they finish this chapter, students will have a comprehensive understanding of key data mining topics and applications.

The order of sections in this chapter follows a logical sequence to make teaching this topic easy. After the opening vignette, Section 5.2 discusses the general concepts.

Section 5.3 covers a wide range of applications. Section 5.4 focuses on the data mining process. Section 5.5 describes the most popular data mining methods and explains their representative techniques. Section 5.6 covers software vendors, commercial tools, and open source tools for modeling and data visualization. Finally, section 5.7 covers privacy concerns.

This chapter contains seven application cases that can help the material come 3 .

alive for students. The Internet exercises at the end are also helpful, especially the ones that send students to vendor sites for case studies and white papers. It can be a good class or group project for students to go to several different vendor sites and compare what they find on each.

ANSWERS TO END OF SECTION REVIEW QUESTIONS

     

Section 5.1 Review Questions 1.

Why should retailers, especially omni-channel retailers, pay extra attention to advanced analytics and data mining?

Utilizing large and information-rich transactional and customer data (that they collect on a daily basis) to optimize their business processes is not a choice for large-scale retailers anymore, but a necessity to stay competitive.

2.

What are the top challenges for multi-channel retailers? Can you think of other industry segments that face similar problems?

The retail industry is amongst the most challenging because of the change that they have to deal with constantly. Understanding customer needs, wants, likes, and dislikes is an ongoing challenge. As the volume and complexity of data increase, so does the time spent on preparing and analyzing it.

Prior to the integration of SAS and Teradata, data for modeling and scoring customers was stored in a data mart. This process required a large amount of time to construct, bringing together disparate data sources and keeping statisticians from working on analytics.

3.

What are the sources of data that retailers such as Cabela’s use for their data mining projects?

Cabela’s uses large and information-rich transactional and customer data (that they collect on a daily basis). In addition, through Web mining they track clickstream patterns of customers shopping online.

4.

What does it mean to have “a single view of the customer”? How can it be accomplished?

Having a single view of the customer means treating the customer as a single entity across whichever channels the customer utilizes. Shopping channels include brick-and-mortar, television, catalog, and e-commerce (through computers and mobile devices). Achieving this single view helps to better focus marketing efforts and drive increased sales.

4 .

5.

What type of analytics help did Cabela’s get from their efforts? Can you think of any other potential benefits of analytics for large-scale retailers like Cabela’s?

Cabela’s has long relied on SAS statistics and data mining tools to help analyze the data it gathers from sales transactions, market research, and demographic data associated with its large database of customers. Using SAS data mining tools, Cabela’s analysts create predictive models to optimize customer selection for all customer contacts. Cabela’s uses these prediction scores to maximize marketing spending across channels and within each customer’s personal contact strategy.

These efforts have allowed Cabela’s to continue its growth in a profitable manner.

In addition, dismantling the information silos, and integration of SAS and Teradata, enabled them to create “a holistic view of the customer.” Since this works so well for the sales/customer side of the business, it could also work in other areas as well. Supply chain is one example. Analytics could help produce a “holistic view of the vendor” as well.

6.

What was the reason for Cabela’s to bring together SAS and Teradata, the two leading vendors in the analytics marketplace?

Cabela’s was already using both for different elements of their business. Each of the two systems was producing actionable analysis of data. But by being separate, too much time was required to construct data marts, bringing together disparate data sources and keeping statisticians from working on analytics. Now, with the integration of the two systems, statisticians can leverage the power of SAS using the Teradata warehouse as one source of information.

7.

What is in-database analytics, and why would you need it?

In-database analytics refers to the practice of applying analytics directly to a database or data warehouse rather than the traditional practice of first transforming into the analytics application’s data format. The time it takes to transform production data into a data warehouse format can be very long. Indatabase analytics eliminates this need.

Section 5.2 Review Questions 1.

Define data mining. Why are there many different names and definitions for data mining?

Data mining is the process through which previously unknown patterns in data were discovered. Another definition would be “a process that uses statistical, mathematical, and artificial learning techniques to extract and identify useful information and subsequent knowledge from large sets of data.” This includes most types of automated data analysis. A third definition: Data mining is the process of finding mathematical patterns from (usually) large sets of data; these can be rules, affinities, correlations, trends, or prediction models.

5 .

Data mining has many definitions because it’s been stretched beyond those limits by some software vendors to include most forms of data analysis in order to increase sales using the popularity of data mining.

2.

What recent factors have increased the popularity of data mining?

Following are some of the most pronounced reasons: • More intense competition at the global scale driven by customers’ ever-changing needs and wants in an increasingly saturated marketplace.

• General recognition of the untapped value hidden in large data sources.

• Consolidation and integration of database records, which enables a single view of customers, vendors, transactions, etc.

• Consolidation of databases and other data repositories into a single location in the form of a data warehouse.

• The exponential increase in data processing and storage technologies.

• Significant reduction in the cost of hardware and software for data storage and processing.

• Movement toward the de-massification (conversion of information resources into nonphysical form) of business practices.

3.

Is data mining a new discipline? Explain.

Although the term data mining is relatively new, the ideas behind it are not. Many of the techniques used in data mining have their roots in traditional statistical analysis and artificial intelligence work done since the early part of 1980s. New or increased use of data mining applications makes it seem like data mining is a new discipline.

In general, data mining seeks to identify four major types of patterns: Associations, Predictions, Clusters and Sequential relationships. These types of patterns have been manually extracted from data by humans for centuries, but the increasing volume of data in modern times has created a need for more automatic approaches. As datasets have grown in size and complexity, direct manual data analysis has increasingly been augmented with indirect, automatic data processing tools that use sophisticated methodologies, methods, and algorithms. The manifestation of such evolution of automated and semiautomated means of processing large datasets is now commonly referred to as data mining.

4.

What are some major data mining methods and algorithms?

Generally speaking, data mining tasks can be classified into three main categories: prediction, association, and clustering. Based on the way in which the patterns are extracted from the historical data, the learning algorithms of data mining methods can be classified as either supervised or unsupervised. With supervised learning algorithms, the training data includes both the descriptive attributes (i.e., 6 .

independent variables or decision variables) as well as the class attribute (i.e., output variable or result variable). In contrast, with unsupervised learning the training data includes only the descriptive attributes. Figure 5.3 (p. 198) shows a simple taxonomy for data mining tasks, along with the learning methods, and popular algorithms for each of the data mining tasks.

5.

What are the key differences between the major data mining methods?

Prediction: the act of telling about the future. It differs from simple guessing by taking into account the experiences, opinions, and other relevant information in conducting the task of foretelling. A term that is commonly associated with prediction is forecasting. Even though many believe that these two terms are synonymous, there is a subtle but critical difference between the two. Whereas prediction is largely experience and opinion based, forecasting is data and model based. That is, in order of increasing reliability, one might list the relevant terms as guessing, predicting, and forecasting, respectively. In data mining terminology, prediction and forecasting are used synonymously, and the term prediction is used as the common representation of the act.

Classification: analyzing the historical behavior of groups of entities with similar characteristics, to predict the future behavior of a new entity from its similarity to those groups Clustering: finding groups of entities with similar characteristics Association: establishing relationships among items that occur together Sequence discovery: finding time-based associations Visualization: presenting results obtained through one or more of the other methods Regression: a statistical estimation technique based on fitting a curve defined by a mathematical equation of known type but unknown parameters to existing data Forecasting: estimating a future data value based on past data values.

Section 5.3 Review Questions 1.

What are the major application areas for data mining?

Applications are listed near the beginning of this section (pp. 201-203): CRM, banking, retailing and logistics, manufacturing and production, brokerage and securities trading, insurance, computer hardware and software, government and defense, travel, healthcare, medicine, entertainment, homeland security and law enforcement, and sports.

2.

Identify at least five specific applications of data mining and list five common characteristics of these applications.

7 .

This question expands on the prior question by asking for common characteristics.

Several such applications and their characteristics are listed on pp. 201-203.

3.

What do you think is the most prominent application area for data mining? Why? Students’ answers will differ depending on which of the applications (most likely banking, retailing and logistics, manufacturing and production, government, healthcare, medicine, or homeland security) they think is most in need of greater certainty. Their reasons for selection should relate to the application area’s need for better certainty and the ability to pay for the investments in data mining.

4.

Can you think of other application areas for data mining not discussed in this section? Explain.

Students should be able to identify an area that can benefit from greater prediction or certainty. Answers will vary depending on their creativity.

Section 5.4 Review Questions 1.

What are the major data mining processes?

Similar to other information systems initiatives, a data mining project must follow a systematic project management process to be successful. Several data mining processes have been proposed: CRISP-DM, SEMMA, and KDD.

2.

Why do you think the early phases (understanding of the business and understanding of the data) take the longest in data mining projects?

Students should explain that the early steps are the most unstructured phases because they involve learning. Those phases (learning/understanding) cannot be automated. Extra time and effort are needed upfront because any mistake in understanding the business or data will most likely result in a failed BI project.

3.

List and briefly define the phases in the CRISP-DM process.

CRISP-DM provides a systematic and orderly way to conduct data mining projects. This process has six steps. First, an understanding of the data and an understanding of the business issues to be addressed are developed concurrently.

Next, data are prepared for modeling; are modeled; model results are evaluated; and the models can be employed for regular use.

4.

What are the main data preprocessing steps? Briefly describe each step and provide relevant examples.

8 .

Data preprocessing is essential to any successful data mining study. Good data leads to good information; good information leads to good decisions. Data preprocessing includes four main steps (listed in Table 5.1 on page 209): • data consolidation: access, collect, select and filter data • data cleaning: handle missing data, reduce noise, fix errors • data transformation: normalize the data, aggregate data, construct new attributes • data reduction: reduce number of attributes and records; balance skewed data 5.

How does CRISP-DM differ from SEMMA? The main difference between CRISP-DM and SEMMA is that CRISP-DM takes a more comprehensive approach—including understanding of the business and the relevant data—to data mining projects, whereas SEMMA implicitly assumes that the data mining project’s goals and objectives along with the appropriate data sources have been identified and understood.

Section 5.5 Review Questions 1.

Identify at least three of the main data mining methods.

Classification learns patterns from past data (a set of information—traits, variables, features—on characteristics of the previously labeled items, objects, or events) in order to place new instances (with unknown labels) into their respective groups or classes. The objective of classification is to analyze the historical data stored in a database and automatically generate a model that can predict future behavior.

Cluster analysis is an exploratory data analysis tool for solving classification problems. The objective is to sort cases (e.g., people, things, events) into groups, or clusters, so that the degree of association is strong among members of the same cluster and weak among members of different clusters.

Association rule mining is a popular data mining method that is commonly used as an example to explain what data mining is and what it can do to a technologically less savvy audience. Association rule mining aims to find interesting relationships (affinities) between variables (items) in large databases.

2.

Give examples of situations in which classification would be an appropriate data mining technique. Give examples of situations in which regression would be an appropriate data mining technique.

Students’ answers will differ, but should be based on the following issues.

Classification is for prediction that can be based on historical data and 9 .

relationships, such as predicting the weather, product demand, or a student’s success in a university. If what is being predicted is a class label (e.g., “sunny,” “rainy,” or “cloudy”) the prediction problem is called a classification, whereas if it is a numeric value (e.g., temperature such as 68°F), the prediction problem is called a regression.

3.

List and briefly define at least two classification techniques.

• Decision tree analysis. Decision tree analysis (a machine-learning technique) is arguably the most popular classification technique in the data mining arena.

• Statistical analysis. Statistical classification techniques include logistic regression and discriminant analysis, both of which make the assumptions that the relationships between the input and output variables are linear in nature, the data is normally distributed, and the variables are not correlated and are independent of each other.

• Case-based reasoning. This approach uses historical cases to recognize commonalities in order to assign a new case into the most probable category.

• Bayesian classifiers. This approach uses probability theory to build classification models based on the past occurrences that are capable of placing a new instance into a most probable class (or category).

• Genetic algorithms. The use of the analogy of natural evolution to build directed search-based mechanisms to classify data samples.

• Rough sets. This method takes into account the partial membership of class labels to predefined categories in building models (collection of rules) for classification problems.

4.

What are some of the criteria for comparing and selecting the best classification technique?

• The amount and availability of historical data • The types of data, categorical, interval, ration, etc.

• What is being predicted -- class or numeric value • The purpose or objective

5.

Briefly describe the general algorithm used in decision trees.

A general algorithm for building a decision tree is as follows: 1. Create a root node and assign all of the training data to it.

2. Select the best splitting attribute.

3. Add a branch to the root node for each value of the split. Split the data into mutually exclusive (nonoverlapping) subsets along the lines of the specific split and mode to the branches.

4. Repeat steps 2 and 3 for each and every leaf node until the stopping criteria is reached (e.g., the node is dominated by a single class label).

6.

Define Gini index. What does it measure?

10 .

The Gini index and information gain (entropy) are two popular ways to determine branching choices in a decision tree. The Gini index measures the purity of a sample. If everything in a sample belongs to one class, the Gini index value is zero.

7.

Give examples of situations in which cluster analysis would be an appropriate data mining technique.

Cluster algorithms are used when the data records do not have predefined class identifiers (i.e., it is not known to what class a particular record belongs).

8.

What is the major difference between cluster analysis and classification?

Classification methods learn from previous examples containing inputs and the resulting class labels, and once properly trained they are able to classify future cases. Clustering partitions pattern records into natural segments or clusters.

9.

What are some of the methods for cluster analysis?

The most commonly used clustering algorithms are k-means and self-organizing maps.

10.

Give examples of situations in which association would be an appropriate data mining technique.

Association rule mining is appropriate to use when the objective is to discover two or more items (or events or concepts) that go together. Students’ answers will differ.

Section 5.6 Review Questions 1.

What are the most popular commercial data mining tools?

Examples of these vendors include IBM (IBM SPSS Modeler), SAS (Enterprise Miner), StatSoft (Statistica Data Miner), KXEN (Infinite Insight), Salford (CART, MARS, TreeNet, RandomForest), Angoss (KnowledgeSTUDIO, KnowledgeSeeker), and Megaputer (PolyAnalyst). Most of the more popular tools are developed by the largest statistical software companies (SPSS, SAS, and StatSoft).

2.

Why do you think the most popular tools are developed by statistics companies?

Data mining techniques involved the use of statistical analysis and modeling. So it’s a natural extension of their business offerings.

11 .

3.

What are the most popular free data mining tools?

Probably the most popular free and open source data mining tool is Weka. Others include RapidMiner and Microsoft’s SQL Server.

4.

What are the main differences between commercial and free data mining software tools?

The main difference between commercial tools, such as Enterprise Miner and Statistica, and free tools, such as Weka and RapidMiner, is computational efficiency. The same data mining task involving a rather large dataset may take a whole lot longer to complete with the free software, and in some cases it may not even be feasible (i.e., crashing due to the inefficient use of computer memory).

5.

What would be your top five selection criteria for a data mining tool? Explain.

Students’ answers will differ. Criteria they are likely to mention include cost, user-interface, ease-of-use, computational efficiency, hardware compatibility, type of business problem, vendor support, and vendor reputation.

Section 5.7 Review Questions 1.

What are the privacy issues in data mining?

Data that is collected, stored, and analyzed in data mining often contains information about real people. This includes identification, demographic, financial, personal, and behavioral information. Most of these data can be accessed through some third-party data providers. In order to maintain the privacy and protection of individuals’ rights, data mining professionals have ethical (and often legal) obligations.

2.

How do you think the discussion between privacy and data mining will progress?

Why? As technology advances and more information about people becomes easier to get, the privacy debate will adjust accordingly. People’s expectations about privacy will become tempered by their desires for the benefits of data mining, from individualized customer service to higher security. As with all issues of social import, the privacy issue will include social discourse, legal and legislative decisions, and corporate decisions. The fact that companies often choose to selfregulate (e.g., by ensuring their data is de-identified) implies that we may as a society be able to find a happy medium between privacy and data mining.

(Answers will vary by student.)

3.

What are the most common myths about data mining?

12 .

Data mining provides instant, crystal-ball predictions.

Data mining is not yet viable for business applications.

Data mining requires a separate, dedicated database.

Only those with advanced degrees can do data mining.

Data mining is only for large firms that have lots of customer data.

4.

What do you think are the reasons for these myths about data mining?

Students’ answers will differ. Some answers might relate to fear of analytics, fear of the unknown, or fear of looking dumb.

5.

What are the most common data mining mistakes/blunders? How can they be minimized and/or eliminated?

• •

Selecting the wrong problem for data mining.

Ignoring what your sponsor thinks data mining is and what it really can and cannot do.

• Leaving insufficient time for data preparation. It takes more effort than one often expects.

• Looking only at aggregated results and not at individual records.

• Being sloppy about keeping track of the mining procedure and results.

• Ignoring suspicious findings and quickly moving on.

• Running mining algorithms repeatedly and blindly. It is important to think hard enough about the next stage of data analysis. Data mining is a very hands-on activity.

• Believing everything you are told about data.

• Believing everything you are told about your own data mining analysis.

• Measuring your results differently from the way your sponsor measures them.

Ways to minimize these risks are basically the reverse of these items.

ANSWERS TO APPLICATION CASE QUESTIONS FOR DISCUSSION

 

Application Case 5.1: Smarter Insurance: Infinity P&C Improves Customer Service and Combats Fraud with Predictive Analytics 1.

How did Infinity P&C improve customer service with data mining?

One out of five claims is fraudulent. Rather than putting all five customers through an investigatory process, SPSS helps Infinity ‘fast-track’ four of them and close their cases within a matter of days. This results in much happier customers, contributes to a more efficient workflow with improved cycle times, and improves retention due to an overall better claims experience.

13 .

2.

What were the challenges, the proposed solution, and the obtained results?

Infinity P&C faces a significant challenge in recognizing fraudulent claims. Fraud represents a $20 billion exposure to the insurance industry and in certain venues could be an element in around 40 percent of claims.

The solution involved using IBM SPSS predictive analytics tools. Based on “red flag” claims, they used this to develop rules for rating and identifying potential frauds. A key benefit of the IBM SPSS system is its ability to continually analyze and score these claims, which helps ensure that we get the claim to the right adjuster at the right time.

As a result of implementing the IBM SPSS analytics tools, Infinity P&C has doubled the accuracy of its fraud identification, contributing to a return on investment of 403 percent per a Nucleus Research study. With SPSS, Infinity P&C has reduced SIU referral time from an average of 45–60 days to approximately 1–3 days. Predictive analytics also helped with subrogation, the process of collecting damages from the at-fault driver’s insurance company.

3.

What was their implementation strategy? Why is it important to produce results as early as possible in data mining studies?

The Special Investigative Unit (SIU) started with low hanging fruit, applying data mining for constructing discovery rules from “red flag” claims. They then leveraged the company’s credit-rating approach toward coming up with a similar scoring mechanism of claims (higher scores indicate greater chance of fraud).

They added rules to the SPSS rule-base to help flag potential subrogation claims.

This incremental approach to implementation works well with a tool like SPSS.

Application Case 5.2: Harnessing Analytics to Combat Crime: Predictive Analytics Helps Memphis Police Department Pinpoint Crime and Focus Police Resources 1.

How did the Memphis Police Department use data mining to better combat crime?

They started mining MPD’s crime data banks to help zero in on where and when criminals were hitting hardest. This began as a pilot program called Operation Blue CRUSH, or Crime Reduction Utilizing Statistical History. Their use of data mining enabled them to focus police resources intelligently by putting them in the right place, on the right day, at the right time. Today, the MPD is continuing to explore new ways to exploit statistical analysis in its crime-fighting mission.

2.

What were the challenges, the proposed solution, and the obtained results?

Crime across the metro area was surging, and city leaders were growing impatient. The solution, a project called Blue CRUSH, involves IBM SPSS 14 .

Modeler, which enables officers to unlock the intelligence hidden in the department’s huge digital library of crime records and police reports going back nearly a decade. This has put a serious dent in Memphis area crime. Since the program was launched, the number of Part One crimes—a category of serious offenses including homicide, rape, aggravated assault, auto theft, and larceny— has plummeted, dropping 27 percent from 2006 to 2010.

Application Case 5.3: A Mine on Terrorist Funding 1.

How can data mining be used to fight terrorism? Comment on what else can be done beyond what is covered in this short application case.

The application case discusses use of data mining to detect money laundering and other forms of terrorist financing. Other applications could be to track the behavior and movement of potential terrorists, as well as text mining emails, blogs, and social media threads.

2.

Do you think that, although data mining is essential for fighting terrorist cells, it also jeopardizes individuals’ rights to privacy?

Yes, because it inevitably involves tracking personal and financial data of individuals. (As an opinion question, students’ answers will vary.)

Application Case 5.4: Data Mining in Cancer Research 1.

How can data mining be used for ultimately curing illnesses like cancer?

Even though cancer research has traditionally been clinical and biological in nature, in recent years data-driven analytic studies have become a common complement. In medical domains where data- and analytics-driven research have been applied successfully, novel research directions have been identified to further advance the clinical and biological studies. Using data mining techniques, researchers are able to identify novel patterns, paving the road toward a cancerfree society.

2.

What do you think are the promises and major challenges for data miners in contributing to medical and biological research endeavors?

According to the American Cancer Society, half of all men and one-third of all women in the United States will develop cancer during their lifetimes; approximately 1.5 million new cancer cases will be diagnosed in 2013. Cancer is the second most common cause of death in the United States and in the world, exceeded only by cardiovascular disease. Data mining shows tremendous promise for helping to understand cancer, leading to better treatment and saved lives. Data mining is not meant to replace medical professionals and researchers, but to 15 .

complement their invaluable efforts to provide data-driven new research directions and to ultimately save more lives. Without the cooperation and feedback from the medical experts, data mining results are not of much use. The patterns found via data mining methods should be evaluated by medical professionals who have years of experience in the problem domain to decide whether they are logical, actionable, and novel to warrant new research directions.

Application Case 5.5: 2degrees Gets a 1275 Percent Boost in Churn Identification 1.

What does 2degrees do? Why is it important for 2degrees to accurately identify churn?

2degrees is New Zealand’s fastest growing mobile telecommunications company.

Customer churn (customers leaving) is an all-too-common problem in the mobile telecommunications industry. So, it makes sense that they would be interested in identifying customers most at risk of churning.

2.

What were the challenges, the proposed solution, and the obtained results?

Their main challenge was to identify the most likely churn customers. For this, they used 11Ants Customer Churn Analyzer. This tool allowed them to get up and running quickly and very economically. They could test the water and determine what the ROI was likely to be for predictive analytics. A carefully controlled experiment was run over a period of 3 months. Customers identified as churners were 1275 percent more likely to be churners than customers chosen at random.

The benefits to 2degrees and their customers included identification and appropriateness, so that likely churners would get more intense marketing than unlikely churners.

3.

How can data mining help in identifying customer churn? How do some companies do it without using data mining tools and techniques?

Identifying customers most at risk of churning is done by analyzing data such as time on network, days since last top-up, activation channel, whether the customer ported their number or not, customer plan, and outbound calling behaviors over the preceding 90 days. Companies without data mining cannot score customers as often; previously 2degrees would do this once per month, but now they can do it daily if they want.

Application Case 5.6: Data Mining Goes to Hollywood: Predicting Financial Success of Movies 1.

Why is it important for Hollywood professionals to predict the financial success of movies?

16 .

The movie industry is the “land of hunches and wild guesses” due to the difficulty associated with forecasting product demand, making the movie business in Hollywood a risky endeavor. If Hollywood could better predict financial success, this would mitigate some of the financial risk.

2.

How can data mining be used to predict the financial success of movies before the start of their production process?

The way Sharda and Delen did it was to use data from movies between 1998 and 2005 as training data, and movies of 2006 as test data. They applied individual and ensemble prediction models, and were able to identify significant variables impacting financial success. They also showed that by using sensitivity analysis, decision makers can predict with fairly high accuracy how much value a specific actor (or a specific release date, or the addition of more technical effects, etc.) brings to the financial success of a film, making the underlying system an invaluable decision aid.

3.

How do you think Hollywood performed, and perhaps is still performing, this task without the help of data mining tools and techniques?

Most is done by gut feel and trial-and-error. This may keep the movie business as a financially risky endeavor, but also allows for creativity. Sometimes uncertainty is a good thing.

Application Case 5.7: Predicting Customer Buying Patterns—The Target Story 1.

What do you think about data mining and its implications concerning privacy?

What is the threshold between knowledge discovery and privacy infringement?

There is a tradeoff between knowledge discovery and privacy rights. Retailers should be sensitive about this when targeting their advertising based on data mining results, especially regarding topics that could be embarrassing to their customers. Otherwise they risk offending these customers, which could hurt their bottom line. (Answers will vary by student.)

2.

Did Target go too far? Did they do anything illegal? What do you think they should have done? What do you think they should do now (quit these types of practices)?

Target might have made a tactical mistake, but they certainly didn’t do anything illegal. They did not use any information that violates customer privacy; rather, they used transactional data that most every other retail chain is collecting and storing (and perhaps analyzing) about their customers. Indeed, even the father apologized when realizing his daughter was actually pregnant. The fact is, we live in a world of massive data, and we are all as consumers leaving traces of our buying behavior for anyone to see. (Answers will vary by student.)

17 .

ANSWERS TO END OF CHAPTER QUESTIONS FOR DISCUSSION 1.

  

Define data mining. Why are there many names and definitions for data mining?

Data mining is the process through which previously unknown patterns in data were discovered. Another definition would be “a process that uses statistical, mathematical, and artificial learning techniques to extract and identify useful information and subsequent knowledge from large sets of data.” This includes most types of automated data analysis. A third definition: Data mining is the process of finding mathematical patterns from (usually) large sets of data; these can be rules, affinities, correlations, trends, or prediction models.

Data mining has many definitions because it’s been stretched beyond those limits by some software vendors to include most forms of data analysis in order to increase sales using the popularity of data mining.

2.

What are the main reasons for the recent popularity of data mining?

Following are some of the most pronounced reasons: • More intense competition at the global scale driven by customers’ ever-changing needs and wants in an increasingly saturated marketplace.

• General recognition of the untapped value hidden in large data sources.

• Consolidation and integration of database records, which enables a single view of customers, vendors, transactions, etc.

• Consolidation of databases and other data repositories into a single location in the form of a data warehouse.

• The exponential increase in data processing and storage technologies.

• Significant reduction in the cost of hardware and software for data storage and processing.

• Movement toward the de-massification (conversion of information resources into nonphysical form) of business practices.

3.

Discuss what an organization should consider before making a decision to purchase data mining software.

Before making a decision to purchase data mining software, organizations should consider the standard criteria to use when investing in any major software: cost/benefit analysis, people with the expertise to use the software and perform the analyses, availability of historical data, a business need for the data mining software.

4.

Distinguish data mining from other analytical tools and techniques.

18 .

Students can view the answer in Figure 5.1 (p. 193), which shows that data mining is a composite or blend of multiple disciplines or analytical tools and techniques.

5.

Discuss the main data mining methods. What are the fundamental differences among them?

Three broad categories of data mining methods are prediction (classification or regression), clustering, and association.

Prediction is the act of telling about the future. It differs from simple guessing by taking into account the experiences, opinions, and other relevant information in conducting the task of foretelling. A term that is commonly associated with prediction is forecasting. Even though many believe that these two terms are synonymous, there is a subtle but critical difference between the two. Whereas prediction is largely experience and opinion based, forecasting is data and model based.

Classification is analyzing the historical behavior of groups of entities with similar characteristics, to predict the future behavior of a new entity from its similarity to those groups.

Clustering is finding groups of entities with similar characteristics.

Association is establishing relationships among items that occur together.

The fundamental differences are:

6.

•

Prediction (classification or regression) predicts future cases or conditions based on historical data.

•

Clustering partitions pattern records into natural segments or clusters.

Each segment’s members share similar characteristics.

•

Association is used to discover two or more items (or events or concepts) that go together.

What are the main data mining application areas? Discuss the commonalities of these areas that make them a prospect for data mining studies.

Applications are listed near the beginning of section 5.3: CRM, banking, retailing and logistics, manufacturing and production, brokerage and securities trading, insurance, computer hardware and software, government and defense, travel, healthcare, medicine, entertainment, homeland security and law enforcement, and sports.

19 .

The commonalities are the need for predictions and forecasting for planning purposes and to support decision making.

7.

Why do we need a standardized data mining process? What are the most commonly used data mining processes?

In order to systematically carry out data mining projects, a general process is usually followed. Similar to other information systems initiatives, a data mining project must follow a systematic project management process to be successful.

Several data mining processes have been proposed: CRISP-DM, SEMMA, and KDD.

8.

Discuss the differences between the two most commonly used data mining processes.

The main difference between CRISP-DM and SEMMA is that CRISP-DM takes a more comprehensive approach—including understanding of the business and the relevant data—to data mining projects, whereas SEMMA implicitly assumes that the data mining project’s goals and objectives along with the appropriate data sources have been identified and understood.

9.

Are data mining processes a mere sequential set of activities? Explain.

No. Even though these steps are sequential in nature, there is usually a great deal of backtracking. Because the data mining is driven by experience and experimentation, depending on the problem situation and the knowledge/experience of the analyst, the whole process can be very iterative (i.e., one should expect to go back and forth through the steps quite a few times) and time consuming. Because latter steps are built on the outcome of the former ones, one should pay extra attention to the earlier steps in order not to put the whole study on an incorrect path from the onset.

10.

Why do we need data preprocessing? What are the main tasks and relevant techniques used in data preprocessing?

Data preprocessing is essential to any successful data mining study. Good data leads to good information; good information leads to good decisions. Data preprocessing includes four main steps (listed in Table 5.1 on page 209): • data consolidation: access, collect, select and filter data • data cleaning: handle missing data, reduce noise, fix errors • data transformation: normalize the data, aggregate data, construct new attributes • data reduction: reduce number of attributes and records; balance skewed data

11.

Discuss the reasoning behind the assessment of classification models.

20 .

The model-building step also encompasses the assessment and comparative analysis of the various models built. Because there is not a universally known best method or algorithm for a data mining task, one should use a variety of viable model types along with a well-defined experimentation and assessment strategy to identify the “best” method for a given purpose.

12.

What is the main difference between classification and clustering? Explain using concrete examples.

Classification learns patterns from past data (a set of information—traits, variables, features—on characteristics of the previously labeled items, objects, or events) in order to place new instances (with unknown labels) into their respective groups or classes. The objective of classification is to analyze the historical data stored in a database and automatically generate a model that can predict future behavior. Classifying customer-types as likely to buy or not buy is an example.

Cluster analysis is an exploratory data analysis tool for solving classification problems. The objective is to sort cases (e.g., people, things, events) into groups, or clusters, so that the degree of association is strong among members of the same cluster and weak among members of different clusters. Customers can be grouped according to demographics.

13.

Moving beyond the chapter discussion, where else can association be used?

Students’ answers will vary.

14.

What are the privacy issues with data mining? Do you think they are substantiated?

Data that is collected, stored, and analyzed in data mining often contains information about real people. This includes identification, demographic, financial, personal, and behavioral information. Most of these data can be accessed through some third-party data providers. In order to maintain the privacy and protection of individuals’ rights, data mining professionals have ethical (and often legal) obligations. As time goes on, this will continue to be a public debate.

As technology advances and more information about people becomes easier to get, the privacy debate will adjust accordingly. People’s expectations about privacy will become tempered by their desires for the benefits of data mining, from individualized customer service to higher security. As with all issues of social import, the privacy issue will include social discourse, legal and legislative decisions, and corporate decisions. The fact that companies often choose to selfregulate (e.g., by ensuring their data is de-identified) implies that we may as a society be able to find a happy medium between privacy and data mining.

21 .

15.

What are the most common myths and mistakes about data mining?

• • • • •

Data mining provides instant, crystal-ball predictions.

Data mining is not yet viable for business applications.

Data mining requires a separate, dedicated database.

Only those with advanced degrees can do data mining.

Data mining is only for large firms that have lots of customer data.

22 .

Chapter 6: Techniques for Predictive Modeling

Learning Objectives for Chapter 6 1.

2.

3.

4.

5.

6.

7.

Understand the concept and definitions of artificial neural networks (ANN) Learn the different types of ANN architectures Know how learning happens in ANN Understand the concept and structure of support vector machines (SVM) Learn the advantages and disadvantages of SVM compared to ANN Understand the concept and formulation of k-nearest neighbor algorithm (kNN) Learn the advantages and disadvantages of kNN compared to ANN and SVM

CHAPTER OVERVIEW Predictive modeling is perhaps the most commonly practiced branch in data mining. It allows decision makers to estimate what the future holds by means of learning from the past. In this chapter, we study the internal structures, capabilities/limitations, and applications of the most popular predictive modeling techniques, such as artificial neural networks, support vector machines, and k-nearest neighbor. These techniques are capable of addressing both classification- and regression-type prediction problems. Often, they are applied to complex prediction problems where other techniques are not capable of producing satisfactory results. In addition to these three (that are covered in this chapter), other notable prediction modeling techniques include regression (linear or nonlinear), logistic regression (for classification-type prediction problems), naive Bayes (probabilistically oriented classification modeling), and different types of decision trees (covered in Chapter 5).

CHAPTER OUTLINE 6.1

OPENING VIGNETTE: PREDICTING MODELING HELPS BETTER UNDERSTAND AND MANAGE COMPLEX MEDICAL PROCEDURES  Questions for the Opening Vignette A. WHAT WE CAN LEARN FROM THIS VIGNETTE

1 .

6.2

BASIC CONCEPTS OF NEURAL NETWORKS A. BIOLOGICAL AND ARTIFICIAL NEURAL NETWORKS ⧫ Technology Insights 6.1: The Relationship Between Biological and Artificial Neural Networks ⧫ Application Case 6.1: Neural Networks Are Helping to Save Lives in the Mining Industry B. ELEMENTS OF ANN 1.

Processing Elements 2.

Network Structure C. NETWORK INFORMATION PROCESSING 1.

Input 2.

Outputs 3.

Connection Weights 4.

Summation Function 5.

Transformation (Transfer) Function 6.

Hidden Layers D. NEURAL NETWORK ARCHITECTURES 1.

Kohonen’s Self-Organizing Feature Maps 2.

Hopfield Networks ⧫ Application Case 6.2: Predictive Modeling Is Powering the Power Generators  Section 6.2 Review Questions

6.3

DEVELOPING NEURAL NETWORK-BASED SYSTEMS A. THE GENERAL ANN LEARNING PROCESS B. BACKPROPAGATION  Section 6.3 Review Questions

6.4

ILLUMINATING THE BLACK BOX OF ANN WITH SENSITIVITY ANALYSIS ⧫ Application Case 6.3: Sensitivity Analysis Reveals Injury Severity Factors in Traffic Accidents  Section 6.4 Review Questions

6.5

SUPPORT VECTOR MACHINES ⧫ Application Case 6.4: Managing Student Retention with Predictive Modeling A. MATHEMATICAL FORMULATION OF SVMS B. PRIMAL FORM C. DUAL FORM D. SOFT MARGIN E. NONLINEAR CLASSIFICATION F. KERNEL TRICK  Section 6.5 Review Questions

6.6

A PROCESS-BASED APPROACH TO THE USE OF SVM 2 .

A.

6.7

1.

Numericizing the Data 2.

Normalizing the Data 3.

Select the Kernel Type and Kernel Parameters 4.

Deploy the Model SUPPORT VECTOR MACHINES VERSUS ARTIFICIAL NEURAL NETWORKS  Section 6.6 Review Questions

NEAREST NEIGHBOR METHOD FOR PROTECTION A. SIMILARITY MEASURE: THE DISTANCE METRIC B. PARAMETER SELECTION 1.

Cross-Validation ⧫ Application Case 6.5: Efficient Image Recognition and Categorization with kNN  Section 6.7 Review Questions

Chapter Highlights Key Terms Questions for Discussion Exercises Teradata University Network (TUN) and Other Hands-On Exercises Team Assignments and Role-Playing Projects Internet Exercises ⧫ End of Chapter Application Case: Coors Improves Beer Flavors with Neural Networks  Questions for the Case References TEACHING TIPS/ADDITIONAL INFORMATION

       

Predictive models grew out of research in pattern recognition and classification.

They form an important element of the new analytics movement. You can discuss these in the “descriptive, predictive, prescriptive” framework. This chapter and chapter 5 focus on the predictive.

This chapter is pretty technical, and gives an in-depth overview of artificial neural networks (ANN), support vector machines (SVM), and the k-nearest neighbor algorithms. There is some rather dense mathematics and detailed design/development specifications, which can be daunting for students. If you would rather use your finite time on other topics, it can be skipped without interfering with students’ ability to understand the chapters that follow.

Glossing over this material probably won’t do the students much good. You should try to go in depth if possible. If you find your time limited, you might want to cover one of the three approaches in more depth. The easiest to explain is probably kNN. But the most interesting for students is probably neural networks, since this is the

3 .

approach that emulates the human brain and that is the most frequently depicted in popular media (for example, consider Data’s “brain” in the Star Trek series).

Where the book refers to artificial neurons “more or less resembling the structure of” their biological counterparts in Section 6.2 (the first “content” section of the chapter), students must understand that this does not mean a physical resemblance. An artificial neuron is a silicon construct, usually embodied in a software model. The ANN equivalent of sending a signal to another biological neuron is placing data in a shared location or sending a software message. The structural resemblance is functional, not physical. This may seem obvious to an instructor, but is often not equally obvious to students seeing the concept for the first time.

Be sure to discuss some common and comparative features between the three approaches. Bring up the distinction between “classification” and “regression” problems, as well as what it means to “train” a system in a supervised learning environment. Stress the importance in all of these models of picking the right parameters, and discuss approaches (such as cross-validation) for this. Students should be able to associate “hyperplanes” with SVM, “similarity measures” with nearest-neighbor, and “backpropagation” with neural networks. Also, a distinction should be clearly made between feedforward and recurrent neural networks. Even if students can’t implement the algorithms, knowing the terminology will help them conceptualize the different approaches. Also, point out that in many of the application cases, multiple approaches can be used. In fact, the opening vignette explicitly espouses the value of using and comparing several algorithms for the accuracy and efficiency.

ANSWERS TO END OF SECTION REVIEW QUESTIONS

     

Section 6.1 Review Questions 1.

Why is it important to study medical procedures? What is the value in predicting outcomes?

Demand for healthcare services is increasing because of the aging population, but the supply side is having problems keeping up with the level and quality of service. Healthcare systems ought to significantly improve their operational effectiveness (doing the right thing, such as diagnosing and treating accurately) and efficiency (doing it the right way, such as using the least amount of resources and time). Clinical decision support systems that use the outcome of data mining studies can support healthcare managers and/or medical professionals in making accurate and timely decisions to optimally allocate resources in order to increase the quantity and quality of medical services.

2.

What factors do you think are the most important in better understanding and managing healthcare? Consider both managerial and clinical aspects of healthcare.

4 .

Healthcare systems ought to significantly improve their operational effectiveness (doing the right thing, such as diagnosing and treating accurately) and efficiency (doing it the right way, such as using the least amount of resources and time).

Effectiveness is probably more of a clinical concern, while efficiency is more of a managerial concern.

3.

What would be the impact of predictive modeling on healthcare and medicine?

Can predictive modeling replace managerial or medical personnel?

Clinical decision support systems that use the outcome of data mining studies (such as the ones presented in this case study) are shown to be useful and reasonably accurate predictors, especially if used in combination. These are not meant to replace healthcare managers and/or medical professionals. Rather, they intend to support them in making accurate and timely decisions to optimally allocate resources in order to increase the quantity and quality of medical services. There still is a long way to go before we can see these decision aids being used extensively in healthcare practices. Among others, there are behavioral, ethical, and political reasons for this resistance to adoption. Maybe the need and the government incentives for better healthcare systems will expedite the adoption.

4.

What were the outcomes of the study? Who can use these results? How can the results be implemented?

The main outcome of this study was to show the efficacy of data mining in predicting the outcome and in analyzing the prognostic factors of complex medical procedures such as CABG surgery. The study showed that using a number of prediction methods (as opposed to only one) in a competitive experimental setting has the potential to produce better predictive as well as explanatory results. SVM, ANN, and both C5 and CART decision trees were used.

5.

Search the Internet to locate two additional cases where predictive modeling is used to understand and manage complex medical procedures.

Open-ended question, answer will be determined by the search results each student selects.

Section 6.2 Review Questions 1.

What is an ANN? An ANN (artificial neural network) is a computer program that models a biological neural network to learn to recognize patterns, and then recognize them in new data.

5 .

2.

Explain the following terms: neuron, axon, and synapse.

A neuron is defined in the book as one of the cells in the human brain. More generally, and as students may have learned in biology courses, it is an electrically excitable cell in the nervous system of an animal.

An axon is the component of a neuron that sends signals to other cells.

A synapse is a junction between two neurons through which signals pass from one neuron to the next, perhaps being altered en route.

3.

How do weights function in an ANN? Weights define the impact that a given input has on a neuron in the next layer. As such, they embody what the network has learned so far. As a network learns, its weights are adjusted.

4.

What is the role of the summation and transformation function?

The summation function determines the total input to a neuron by calculating the weighted sum of its individual input values. Its output is input to the transformation (transfer) function. The transformation (or transfer) function determines the output of a neuron from its input (i.e., output of the summation function).

Many possible transformation functions exist. Most are sigmoid functions. (A sigmoid function is a non-decreasing function whose value is in the range 0–1.) A simple transformation function produces an output of 0 for inputs up to a threshold value, an output of 1 from that value up.

5.

What are the most common ANN architectures? How do they differ from each other?

The most common ANN architectures include feedforward (multilayer perceptron with backpropagation), associative memory, recurrent networks, Kohonen’s selforganizing feature maps, and Hopfield networks. The simplest architectures are feedforward networks, which involve information flowing unidirectionally from input layer to hidden layers to output layer. More complex architectures are recurrent; these have many connections in every direction between the layers and neurons, creating a complex connection structure. Kohonen networks SOM networks provide a way to represent multidimensional data in much lower dimensional spaces, usually one or two dimensions. One of the most interesting aspects of SOM is that they learn to classify data without supervision (i.e., no output vectors). Hopfield networks are interconnected networks of nonlinear neurons, and are effective in solving complex constraint optimization problems.

6 .

With Hopfield networks, each neuron is connected to every other neuron within the network.

Section 6.3 Review Questions 1.

List the nine steps in conducting a neural network project.

These are shown in the flowchart of Figure 6.9. After testing (step 8), it is possible to return to a previous step: Step 1: Collect data Step 2: Separate (data) into training and test sets Step 3: Define a network structure Step 4: Select a learning algorithm Step 5: Set parameters and values Step 6: Initialize weights and start training Step 7: Stop training, freeze the network weights Step 8: Test the trained network Step 9: Implementation: use the network with new cases

2.

What are some of the design parameters for developing a neural network?

Some design parameters to consider include the type of network to employ, the number of nodes (input, hidden, and output) and layers, the types of transformation functions within each neuron, the original weight settings, and the acceptable delta (error) level.

3.

How does backpropagation learning work?

Backpropagation (short for back-error propagation) is the most widely used supervised learning algorithm in neural computing. Backpropagation involves a feedforward network with one or more hidden layers. Backpropagation involves supervised learning because predetermined correct answers for each training pattern are given and compared to the results given by the backpropagation algorithm. The difference between the correct answer and the output (known as the delta value) is calculated, and weights of the output nodes are adjusted accordingly. Weight adjustment also proceeds backward to the hidden layers; hence the term “backpropagation.” This process repeats, continuously testing against training patterns, until the delta value is sufficiently low.

4.

Describe different types of neural network software available today.

Many commercial ANN software products function like software shells. They provide a set of standard architectures, learning algorithms, and parameters, along with the ability to manipulate the data. Some development tools can support up to several dozen network paradigms and learning algorithms. Most of the leading 7 .

data mining tools (e.g., SAS Enterprise Miner, IBM SPSS Modeler, Statistica Data Miner) include neural network learning algorithms. Some specialized neural network products include California Scientific (BrainMaker), NeuralWare, NeuroDimension Inc., Ward Systems Group (Neuroshell), and Megaputer. Others are implemented as spreadsheet add-ins. In addition, there are class libraries and APIs for languages such as Java and C++. Mathematical applications such as MATLAB also include neural network algorithms.

5.

How are neural networks implemented in practice when the training/testing is complete?

After testing and training, the network is deployed for use on unknown new cases.

It might be used as a stand-alone system or as part of another software system where new input data will be presented to it and its output will be a recommended decision. At this point the recommendations provided by the neural network are considered to be valid, because it has been extensively trained on training data and tested with test data.

Section 6.4 Review Questions 1.

What is the so-called “black-box” syndrome?

ANN are typically thought of as black boxes, capable of solving complex problems but lacking the explanation of its capabilities. This phenomenon is commonly referred to as the “black-box” syndrome.

2.

Why is it important to be able to explain an ANN’s model structure?

It is important to be able to explain a model’s “inner being”; such an explanation offers assurance that the network has been properly trained and will behave as desired once deployed in a business intelligence environment. Such a need to “look under the hood” might be attributable to a relatively small training set (as a result of the high cost of data acquisition) or a very high liability in case of a system error.

3.

How does sensitivity analysis work?

Sensitivity analysis is a method for extracting the cause-and-effect relationships among the inputs and the outputs of a trained neural network model. In the process of performing sensitivity analysis, the trained neural network’s learning capability is disabled so that the network weights are not affected.

4.

Search the Internet to find other ANN explanation methods.

Open-ended answer, depending on what a student finds.

8 .

Section 6.5 Review Questions 1.

How do SVM work?

Support vector machines (SVM) are supervised learning methods that produce input-output functions from a set of labeled training data. Both classification functions and regression functions are possible in SVMs, and these can be either linear or nonlinear functions. For example, given a classification-type prediction problem, linear classifiers (hyperplanes) can separate the data into multiple subsections, each representing one of the classes. If there are n classes to group data into, then the hyperplane will have n-1 dimensions.

2.

What are the advantages and disadvantages of SVM? SVMs are popular because of their superior predictive power and their theoretical foundation. SVMs have demonstrated highly competitive performance in numerous real-world prediction problems. A significant advantage of SVMs is that while ANNs may suffer from multiple local minima, the solutions to SVMs are global and unique. Two more advantages of SVMs are that they have a simple geometric interpretation and give a sparse solution. The reason that SVMs often outperform ANNs in practice is that they successfully deal with the “over fitting” problem, which is a big issue with ANNs. One disadvantage with SVM is the selection of the kernel type and kernel function parameters. A second and perhaps more important limitation of SVMs are the speed and size, both in the training and testing cycles. Model building in SVMs involves complex and timedemanding calculations. From the practical point of view, perhaps the most serious problem with SVMs is the high algorithmic complexity and extensive memory requirements of the required quadratic programming in large-scale tasks.

3.

What is the meaning of “maximum margin hyperplanes”? Why are they important in SVM? Although many linear classifiers (hyperplanes) can separate the data into multiple subsections, only one hyperplane achieves the maximum separation between the classes. This is the hyperplane whose distance from the nearest data points is maximized. The trick is to find the parallel hyperplanes that separate the classes and whose margin of distance is at a maximum. The assumption is that the larger the margin or distance between these parallel hyperplanes, the better the generalization power of the classifier.

4.

What is “kernel trick”? How is it used in SVM? The kernel trick is a method for converting a linear classifier algorithm into a nonlinear one by using a nonlinear function to map the original observations into 9 .

a higher-dimensional space; this makes a linear classification in the new space equivalent to nonlinear classification in the original space. This is what enables the general hyperplane approach to SVMs (which are inherently linear) to solve nonlinear classification problems. Common kernel types are polynomial, radial basis function (RBF), Gaussian RBF, and sigmoid.

Section 6.6 Review Questions 1.

What are the main steps and decision points in developing a SVM model?

First, numericize the data. Each data instance must be represented as a vector of numeric values, including the categorical variable (the classification). Second, normalize (scale) the data. This prevents larger-magnitude attributes from dominating the others during the learning process. Next, select the kernel type and the kernel parameters. Finally, deploy the model.

2.

How do you determine the optimal kernel type and kernel parameters?

You can do this experimentally, trying different ones out and comparing results.

Often the RBF is a good start. Deciding on the best parameters for a kernel involves a parameter search method, such as cross-validation or grid search.

3.

Compared to ANN, what are the advantages of SVM? A significant advantage of SVMs is that while ANNs may suffer from multiple local minima, the solutions to SVMs are global and unique. Two more advantages of SVMs are that they have a simple geometric interpretation and give a sparse solution. The reason that SVMs often outperform ANNs in practice is that they successfully deal with the “over fitting” problem, which is a big issue with ANNs.

4.

What are the common application areas for SVM? Conduct a search on the Internet to identify popular application areas and specific SVM software tools used in those applications.

Common applications of SVM include text and hypertext categorization, image classification, protein classification for medical science, and hand-written character recognition. Many of the same software products that include neural network algorithms also include SVM algorithms (e.g., MATLAB, SPSS).

(Student answers will vary.)

Section 6.7 Review Questions 1.

What is special about the kNN algorithm?

10 .

The k-nearest neighbor algorithm is among the simplest of all machine-learning algorithms. It is easy to understand (and explain to others) what it does and how it does it.

2.

What are the advantages and disadvantages of kNN as compared to ANN and SVM? Compared to both ANN and SVM, the k-nearest neighbor algorithm is very simple to learn and implement. But it is a lazy learner, often reaching local rather than global minima/maxima. In addition, the accuracy of the kNN algorithm can be significantly different with different values of k. Furthermore, the predictive power of the kNN algorithm degrades with the presence of noisy, inaccurate, or irrelevant features.

3.

What are the critical success factors for a kNN implementation?

One critical factor is selection of the best similarity metric for determining what is a “nearest” neighbor. A second is the selection of the correct parameter (i.e., the k value). This can be done using cross-validation.

4.

What is a similarity (or distance measure)? How can it be applied to both numerical and nominal valued variables?

A similarity measure is a mathematically calculable distance metric. Given a new case, kNN makes predictions based on the outcome of the k neighbors closest in distance to that point. Therefore, to make predictions with kNN, we need to define a metric for measuring the distance between the new case and the cases from the examples. Two popular approaches are Euclidean and rectilinear. Both of these assume numerical data points. There are ways to measure distance for nonnumerical (nominal) data as well. In the simplest case, for a multi-value nominal variable, if the value of that variable for the new case and that for the example case are the same, the distance would be zero, otherwise one. In cases such as text classification, more sophisticated metrics exist, such as the overlap metric (or Hamming distance).

5.

What are the common applications of kNN? Image recognition, DNA sequencing, pattern recognition, statistical classification, Internet marketing, and plagiarism detection are some of the possible applications.

(Student answers will vary.)

ANSWERS TO APPLICATION CASE QUESTIONS FOR DISCUSSION

 

Application Case 6.1: Neural Networks Are Helping to Save Lives in the Mining Industry 11 .

1.

How did neural networks help save lives in the mining industry?

The Council for Scientific and Industrial Research (CSIR) in South Africa developed a device with an embedded neural network that assists any miner in making an objective decision when determining the integrity of the hanging wall.

This helps prevent death and injury from rock falls, a common danger to miners.

2.

What were the challenges, the proposed solution, and the obtained results?

In the mining industry, most of the underground injuries and fatalities are due to rock falls (i.e., fall of hanging wall/roof). The method that has been used for many years in the mines when determining the integrity of the hanging wall is to tap the hanging wall with a sounding bar and listen to the sound emitted. An experienced miner can differentiate an intact/solid hanging wall from a detached/loose hanging wall by the sound that is emitted. But this method is subjective. The proposed solution is to provide miners with a device that uses a trained neural network to record and classify sounds to identify a hanging wall as either intact or detached.

The multilayer perceptron-type ANN architecture that was built achieved better than 70 percent prediction accuracy on sample data. At this point, the system is in its prototype-testing phase.

3.

What was their implementation strategy? Why is it important to produce results as early as possible in data mining studies?

The implementation was done using NeuroSolutions, a popular artificial neural network modeling software developed by NeuroDimensions, Inc., to develop the classification type prediction models.

Application Case 6.2: Predictive Modeling Is Powering the Power Generators 1.

What are the key environmental concerns in the electric power industry?

Even though some energy-generation methods are favored over others, all forms of electricity generation have positive and negative aspects. Some are environmentally favored but are economically unjustifiable; others are economically superior but environmentally prohibitive. In a market economy, the options with fewer overall costs are generally chosen above all other sources. It is not clear yet which form can best meet the necessary demand for electricity without permanently damaging the environment. Current trends indicate that increasing the shares of renewable energy and distributed generation from mixed sources has the promise of reducing/balancing environmental and economic risks.

2.

What are the main application areas for predictive modeling in the electric power industry?

12 .

Predictive modeling can be used to optimize operational parameters to produce cleaner combustion and more stable flame temperatures. Another application is to predict problems, such as failures or maintenance issues, before they happen.

Modeling can also be used to reduce NOx emissions.

3.

How was predictive modeling used to address a variety of problems in the electric power industry?

See answer to question #2.

Application Case 6.3: Sensitivity Analysis Reveals Injury Severity Factors in Traffic Accidents 1.

How does sensitivity analysis shed light on the black box (i.e., neural networks)?

Sensitivity analysis techniques provide a clear interpretation of how a neural network does what it does; that is, specifically how (and to what extent) the individual inputs factor into the generation of specific network output. This process extracts the cause-and-effect relationships among the inputs and the outputs of a trained neural network model.

2.

Why would someone choose to use a black-box tool like neural networks over theoretically sound, mostly transparent statistical tools like logistic regression?

ANN are known to be superior in capturing highly nonlinear complex relationships between predictor and target variables, assuming a linear relationship is often an oversimplification of the problem.

3.

In this case, how did neural networks and sensitivity analysis help identify injuryseverity factors in traffic accidents?

ANN and sensitivity analysis helped estimate the significance of the crash factors on the level of injury severity sustained by the driver. This study was a two-step process. In the first step, the testers developed a series of prediction models (one for each injury severity level) to capture the in-depth relationships between the crash-related factors and a specific level of injury severity. In the second step, they conducted sensitivity analysis on the trained neural network models to identify the prioritized importance of crash-related factors as they relate to different injury severity levels.

The study revealed that the variable seatbelt use was the most important determinant for predicting higher levels of injury severity but it was one of the least significant predictors for lower levels of injury severity. Other interesting

13 .

findings involved gender (good predictor for low injury severity, but not for high) and age (vice versa).

Application Case 6.4: Managing Student Retention with Predictive Modeling 1.

Why is attrition one of the most important issues in higher education?

Student attrition has become one of the most challenging problems for decision makers in academic institutions. In spite of all of the programs and services to help retain students, according to the U.S. Department of Education, Center for Educational Statistics (nces.ed.gov), only about half of those who enter higher education actually graduate with a bachelor’s degree. High rates of student attrition usually result in loss of financial resources, lower graduation rates, and inferior perception of the school in the eyes of all stakeholders.

2.

How can predictive analytics (ANN, SVM, and so forth) be used to better manage student retention?

An alternative (or a complementary) approach to the traditional survey-based retention research is an analytic approach where the data commonly found in institutional databases is used. Educational institutions routinely collect a broad range of information about their students, including demographics, educational background, social involvement, socioeconomic status, and academic progress.

Using institutional data, prediction models can be developed to accurately identify the students at risk of dropout, so that limited resources (people, money, time, etc., at an institution’s student success center) can be optimally used to retain most of them.

3.

What are the main challenges and potential solutions to the use of analytics in retention management?

In order to meet the challenges cited in the answer to #2, the main goals of analytic studies in this area are to (1) develop models to correctly identify the freshman students who are most likely to drop out after their freshman year, and (2) identify the most important variables by applying sensitivity analyses on developed models. The model building approach will involve the same steps you usually perform in predictive analytics tasks, including data collection/consolidation/preprocessing, cross-validation for parameter selection, use of various algorithms (ANN, SVM, etc.), and sensitivity analysis.

Application Case 6.5: Efficient Image Recognition and Categorization with kNN 1.

Why is image recognition/classification a worthy but difficult problem?

14 .

Application areas of image recognition and categorization range from agriculture to homeland security, personalized marketing to environmental protection. Image recognition is an integral part of an artificial intelligence field called computer vision. While the field of visual recognition and category recognition has been progressing rapidly, much remains to be done to reach human-level performance.

Current approaches are capable of dealing with only a limited number of categories (100 or so categories) and are computationally expensive.

2.

How can kNN be effectively used for image recognition/classification applications? kNN classifiers are natural in this setting, and have computational advantages over SVMs. But they suffered from the problem of high variance (in bias-variance decomposition) in the case of limited sampling. By combining kNN with SVM, you can improve performance while maintaining computational advantage.

Another possible hybrid is combining kNN with Naïve Bayes algorithm.

ANSWERS TO END OF CHAPTER QUESTIONS FOR DISCUSSION 1.

  

Compare artificial and biological neural networks. What aspects of biological networks are not mimicked by artificial ones? What aspects are similar?

The physical aspects of biological neural networks are not mimicked by ANNs. ANNs mimic the functional aspects of biological networks at an elementary level.

2.

The performance of ANN relies heavily on the summation and transformation functions. Explain the combined effects of the summation and transformation functions and how they differ from statistical regression analysis.

The summation and transformation functions use a neuron’s inputs to create its output. First, the summation function aggregates the inputs into a single value based on weighted summation. Second, the transformation function converts this input value to an output value, generally between 0 and 1.

This approach differs from statistical regression analysis in its ability to find more complex relationships than statistical comparisons can find.

3.

ANN can be used for both supervised and unsupervised learning. Explain how they learn in a supervised mode and in an unsupervised mode.

Learning in supervised mode is simpler. Since we know the desired output and the value of inputs as well as all the algorithms that express relationships in the system, it is possible to compute values of output for given weights. By comparing actual and desired weights one can see the difference. This difference can be reduced by adjusting the values of the weights. Ideally the differences 15 .

should be driven to zero. Finding the ideal value of the weights is called the learning or training.

In unsupervised situations, the desired outputs are not specified. Therefore, the process is only semi-automatic (i.e., a human must examine the result to determine when the training needs to stop). Weights and other parameters can be adjusted once the outputs are examined.

4.

Explain the difference between a training set and a testing set. Why do we need to differentiate them? Can the same set be used for both purposes? Why or why not?

The same set cannot be used for both purposes because that would create a “selffulfilling prophecy:” having learned to separate two parts of the input set from each other, correct behavior would then be tested by simply confirming that it has done so. This is not meaningful. It is necessary to test whether or not it has learned to separate the two parts of the overall universe under investigation from each other.

For example, suppose an aerial surveillance ANN is designed to tell tanks from rocks and is thus trained with photographs of those two items. Suppose, hypothetically, that the photographs of tanks were (because of differences in the time they were taken, equipment used, etc.) slightly darker than those of rocks.

The ANN might successfully learn to separate the photographs in the training set into the two appropriate parts. A person simply confirming that it did so, using those same photographs, would have no way of knowing that it might have just learned to tell darker photographs from lighter ones. It is necessary to test the ANN with a totally different set of photographs to confirm that it can actually identify tanks.

5.

Say that a neural network has been constructed to predict the creditworthiness of applicants. There are two output nodes: one for yes (1 = yes, 0 = no) and one for no (1= no, 0 = yes). An applicant receives a score of 0.83 for the “yes” output node and a 0.44 for the “no” output node. Discuss what may have happened and whether the applicant is a good credit risk.

The score for “yes” is stronger than the score for “no.” The factors that led to the scores are unknown, but the two scores can be expected to be independent of each other and may be based on different combinations of inputs. Were a simple threshold value of 0.5 used to determine the validity of an output value this network would have affirmed creditworthiness, denied non-creditworthiness.

The ANN output suggests that the applicant is probably a good (though perhaps not outstanding) credit risk. The relatively high score for non-creditworthiness suggests a possible problem in the applicant’s background that should be looked into further before credit is granted.

16 .

6.

Everyone would like to make a great deal of money on the stock market. Only a few are very successful. Why is using an ANN a promising approach? What can it do that other decision support technologies cannot do? How could it fail?

The factors that lead to changes in stock prices are imperfectly known. Attempts to understand them have not been successful. However, it is reasonable to suppose that those factors, while we may not understand them, do exist. An ANN might be able to find them in a mass of data because it has no preconceived notions about what they should be.

Other decision support technologies are driven by human guidance in some way and rely ultimately on human decision makers to identify the relevant factors.

An ANN could fail in this application if it cannot identify the relevant factors, or if it identifies a set of factors that would have predicted stock movements during one time period (when the market was driven by one set of factors) but are not useful in predicting them in another period (when it is driven by different factors). Attempting to use factors derived from analysis of an earlier period to guide investments in a later one could be a recipe for bankruptcy.

ANSWERS TO END OF CHAPTER APPLICATION CASE QUESTIONS 1.

 

Why is beer flavor important to the Coors profitability?

Beer flavor is a key factor in customer preferences and buying behavior.

2.

What is the objective of the neural network used at Coors?

To determine the link between the chemical composition of a beer, which can be measured, and its flavor, which cannot be measured but which consumers can detect and care about.

3.

Why were the results of the Coors neural network initially poor, and what was done to improve the results?

The results were initially poor for two reasons. First, the network was only trained with one type of beer, so the variation in its inputs was low. Second, it was only trained for one flavor factor, so its performance was impacted by the inputs that had no impact on that factor but whose variations within the training sample created distracting “noise.”

To improve the results, Coors trained the neural network using a wider variety of products and more combinations of inputs.

4.

What benefits might Coors derive if this project is successful?

If this project is successful, Coors might ultimately be able to control the flavors of its beers via their chemical composition, which could be monitored 17 .

automatically and presumably controlled during the brewing process. Being able to do this depends, of course, on more than knowing the relationships between the chemicals in beer and its flavor.

5.

What modifications would you provide to improve the results of beer flavor prediction?

Based on the next-to-last paragraph of the case, I would refine the sensitivity of the instrumentation, measure a larger number of flavor-active compounds, and measure the factors that contribute to mouth-feel and the beer’s physical characteristics. Whether this is practical, if practical whether it is cost-effective, and whether it would result in a more effective beer production process than Coors can achieve with experienced, professional brew masters, is (as far as we can tell from this case) still an open question.

18 .

Chapter 7: Text Analytics, Text Mining, and Sentiment Analysis Learning Objectives for Chapter 7 1.

2.

3.

4.

5.

6.

7.

8.

9.

Describe text mining and understand the need for text mining Differentiate among text analytics, text mining, and data mining Understand the different application areas for text mining Know the process of carrying out a text mining project Appreciate the different methods to introduce structure to text-based data Describe sentiment analysis Develop familiarity with popular applications of sentiment analysis Learn the common methods for sentiment analysis Become familiar with speech analytics as it relates to sentiment analysis

CHAPTER OVERVIEW This chapter provides a rather comprehensive overview of text mining and one of its most popular applications, sentiment analysis, as they both relate to business analytics and decision support systems. Generally speaking, sentiment analysis is a derivative of text mining, and text mining is essentially a derivative of data mining. Because textual data is increasing in volume more than the data in structured databases, it is important to know some of the techniques used to extract actionable information from the large quantity of unstructured data.

CHAPTER OUTLINE 7.1

OPENING VIGNETTE: MACHINE VERSUS MEN ON JEOPARDY!: THE STORY OF WATSON  Questions for the Opening Vignette A. WHAT WE CAN LEARN FROM THIS VIGNETTE

7.2

TEXT ANALYTICS AND TEXT MINING CONCEPTS AND DEFINITIONS ⧫ Technology Insights 7.1: Text Mining Lingo ⧫ Application Case 7.1: Text Mining for Patent Analysis  Section 7.2 Review Questions

7.3

NATURAL LANGUAGE PROCESSING 1 .

⧫ 

Application Case 7.2: Text Mining Improves Hong Kong Government’s Ability to Anticipate and Address Public Complaints Section 7.3 Review Questions

7.4

TEXT MINING APPLICATIONS A. MARKETING APPLICATIONS B. SECURITY APPLICATIONS ⧫ Application Case 7.3: Mining for Lies C. BIOMEDICAL APPLICATIONS D. ACADEMIC APPLICATIONS ⧫ Application Case 7.4: Text Mining and Sentiment Analysis Helps Improve Customer Service Performance  Section 7.4 Review Questions

7.5

TEXT MINING PROCESS A. TASK 1: ESTABLISH THE CORPUS B. TASK 2: CREATE THE TERM–DOCUMENT MATRIX 1.

Representing the Indices 2.

Reducing the Dimensionality of the Matrix C. TASK 3: EXTRACT THE KNOWLEDGE 1.

Classification 2.

Clustering 3.

Association 4.

Trend Analysis ⧫ Application Case 7.5: Research Literature Survey with Text Mining  Section 7.5 Review Questions

7.6

TEXT MINING TOOLS A. COMMERCIAL SOFTWARE TOOLS B. FREE SOFTWARE TOOLS ⧫ Application Case 7.6: A Potpourri of Text Mining Cases  Section 7.6 Review Questions

7.7

SENTIMENT ANALYSIS OVERVIEW ⧫ Application Case 7.7: Whirlpool Achieves Customer Loyalty and Product Success with Text Analytics  Section 7.7 Review Questions

7.8

SENTIMENT ANALYSIS APPLICATIONS 1.

Voice of the Customer (VOC) 2.

Voice of the Market (VOM) 3.

Voice of the Employee (VOE) 4.

Brand Management 5.

Financial Markets 2 .

6.

7.

8.

7.9

Politics Government Intelligence Other Interesting Areas  Section 7.8 Review Questions

SENTIMENT ANALYSIS PROCESS 1.

Step 1: Sentiment Detection 2.

Step 2: N-P Polarity Classification 3.

Step 3: Target Identification 4.

Step 4: Collection and Aggregation A. METHODS FOR POLARITY IDENTIFICATION B. USING A LEXICON C. USING A COLLECTION OF TRAINING DOCUMENTS ⧫ Technology Insights 7.2: Large Textual Data Sets for Predictive Text Mining and Sentiment Analysis D. IDENTIFYING SEMANTIC ORIENTATION OF SENTENCES AND PHRASES E. IDENTIFYING SEMANTIC ORIENTATION OF DOCUMENT  Section 7.9 Review Questions

7.10

SENTIMENT ANALYSIS AND SPEECH ANALYTICS A. HOW IS IT DONE? 1.

The Acoustic Approach 2.

The Linguistic Approach ⧫ Application Case 7.8: Cutting Through the Confusion: Blue Cross Blue Shield of North Carolina Uses Nexidia’s Speech Analytics to Ease Member Experience in Healthcare  Section 7.10 Review Questions Chapter Highlights Key Terms Questions for Discussion Exercises Teradata University Network (TUN) and Other Hands-On Exercises Team Assignments and Role-Playing Projects Internet Exercises ⧫ End of Chapter Application Case: BBVA Seamlessly Monitors and Improves Its Online Reputation  Questions for the Case References

TEACHING TIPS/ADDITIONAL INFORMATION

       

Text analytics, text mining, and sentiment analysis are topics that students find especially interesting and (relatively) easier than data mining. They are able to 3 .

relate to text mining applications. Students are familiar with social media outlets such as Facebook and Twitter, and often contribute sentiments of their own. The opening vignette discusses IBM’s Watson, which utilized text analytics (along with many other AI-related technologies) to win at Jeopardy!. Students can come up with examples of value-added text mining and sentiment analysis applications.

It will be useful to distinguish between text mining and the broader topic of text analytics. This will be a good chapter to engage students in class discussions, and especially to relate the concepts of the technologies to the various application cases in the chapter. Students should also be able to discuss other possible applications, perhaps even within the university or college that they are attending.

For example, how can text analytics be used to improve the student experience?

ANSWERS TO END OF SECTION REVIEW QUESTIONS

     

Section 7.1 Review Questions 1.

What is Watson? What is special about it?

Watson is a question answering (QA) computer system developed by an IBM Research team and named after IBM’s first president as part of a project called DeepQA. What makes it special is that it is able to compete at the human champion level in real time on the TV quiz show, Jeopardy!; in fact, in 2011, it was able to defeat Ken Jennings, who held the record for the longest winning streak in the game. Like Deep Blue has done with chess, Watson is showing that computer systems are getting quite good at demonstrating human-like intelligence.

2.

What technologies were used in building Watson (both hardware and software)?

Watson is built on the DeepQA framework. The hardware for this system involves a massively parallel processing architecture. In terms of software, Watson uses a variety of AI-related QA technologies, including text mining, natural language processing, question classification and decomposition, automatic source acquisition and evaluation, entity and relation detection, logical form generation, and knowledge representation and reasoning.

3.

What are the innovative characteristics of DeepQA architecture that made Watson superior?

The DeepQA architecture involves massive parallelism, many experts, pervasive confidence estimation, and integration of the-latest-and-greatest in text analytics, involving both shallow and deep semantic knowledge. As implemented in Watson, DeepQA brings more than 100 different techniques for analyzing natural language, identifying sources, finding and generating hypotheses, finding and scoring evidence, and merging and ranking hypotheses. More important than any particular technique is the combination of overlapping approaches that can bring 4 .

their strengths to bear and contribute to improvements in accuracy, confidence, and speed.

4.

Why did IBM spend all that time and money to build Watson? Where is the ROI? IBM’s goal was to advance computer science by exploring new ways for computer technology to affect science, business, and society. The techniques IBM developed with DeepQA and Watson are relevant in a wide variety of domains central to IBM’s mission. For example, IBM is currently working on a version of Watson to take on surmountable problems in healthcare and medicine. If successful, this could give IBM a distinct competitive advantage in this important technological application area.

5.

Conduct an Internet search to identify other previously developed “smart machines” (by IBM or others) that compete against the best of man. What technologies did they use?

The most obvious competitive smart machine, also developed by IBM, is Deep Blue, which in 1997 defeated the world chess champion, Garry Kasparov. The technology involved mostly brute force computing power, involving massive parallelism. This enabled Deep Blue to evaluate 200 million positions per second, with search heuristics enabling Deep Blue to search from six to twenty moves ahead, far more than any previous chess playing program. Deep Blue was inspired by and evolved from another chess playing system called Deep Thought, which was developed at Carnegie Mellon and IBM. In general, artificial intelligence techniques have often been applied toward competitive game playing systems. For example, in 1979, a backgammon-playing programmer developed by Hans Berliner defeated the world champion backgammon player at the time. Some techniques for this system included the use of nonlinear functions with application coefficients for evaluating board positions.

Berliner’s work was inspired by machine learning algorithms developed by A. L. Simon in the 1950s and 1960s for playing checkers.

Section 7.2 Review Questions 1.

What is text analytics? How does it differ from text mining?

Text analytics is a concept that includes information retrieval (e.g., searching and identifying relevant documents for a given set of key terms) as well as information extraction, data mining, and Web mining. By contrast, text mining is primarily focused on discovering new and useful knowledge from textual data sources. The overarching goal for both text analytics and text mining is to turn unstructured textual data into actionable information through the application of natural language processing (NLP) and analytics. However, text analytics is a

5 .

broader term because of its inclusion of information retrieval. You can think of text analytics as a combination of information retrieval plus text mining.

2.

What is text mining? How does it differ from data mining?

Text mining is the application of data mining to unstructured, or less structured, text files. As the names indicate, text mining analyzes words; and data mining analyzes numeric data.

3.

Why is the popularity of text mining as an analytics tool increasing?

Text mining as a BI is increasing because of the rapid growth in text data and availability of sophisticated BI tools. The benefits of text mining are obvious in the areas where very large amounts of textual data are being generated, such as law (court orders), academic research (research articles), finance (quarterly reports), medicine (discharge summaries), biology (molecular interactions), technology (patent files), and marketing (customer comments).

4.

What are some popular application areas of text mining?

• Information extraction. Identification of key phrases and relationships within

text by looking for predefined sequences in text via pattern matching.

• Topic tracking. Based on a user profile and documents that a user views, text

mining can predict other documents of interest to the user.

• Summarization. Summarizing a document to save time on the part of the reader.

• Categorization. Identifying the main themes of a document and then placing the

document into a predefined set of categories based on those themes.

• Clustering. Grouping similar documents without having a predefined set of

categories.

• Concept linking. Connects related documents by identifying their shared

concepts and, by doing so, helps users find information that they perhaps would not have found using traditional search methods.

• Question answering. Finding the best answer to a given question through knowledge-driven pattern matching.

Section 7.3 Review Questions 1.

What is natural language processing?

Natural language processing (NLP) is an important component of text mining and is a subfield of artificial intelligence and computational linguistics. It studies the problem of “understanding” the natural human language, with the view of converting depictions of human language (such as textual documents) into more formal representations (in the form of numeric and symbolic data) that are easier for computer programs to manipulate.

6 .

2.

How does NLP relate to text mining?

Text mining uses natural language processing to induce structure into the text collection and then uses data mining algorithms such as classification, clustering, association, and sequence discovery to extract knowledge from it.

3.

What are some of the benefits and challenges of NLP? NLP moves beyond syntax-driven text manipulation (which is often called “word counting”) to a true understanding and processing of natural language that considers grammatical and semantic constraints as well as the context. The challenges include: •

• • •

• •

4.

Part-of-speech tagging. It is difficult to mark up terms in a text as corresponding to a particular part of speech because the part of speech depends not only on the definition of the term but also on the context within which it is used.

Text segmentation. Some written languages, such as Chinese, Japanese, and Thai, do not have single-word boundaries.

Word sense disambiguation. Many words have more than one meaning.

Selecting the meaning that makes the most sense can only be accomplished by taking into account the context within which the word is used.

Syntactic ambiguity. The grammar for natural languages is ambiguous; that is, multiple possible sentence structures often need to be considered. Choosing the most appropriate structure usually requires a fusion of semantic and contextual information.

Imperfect or irregular input. Foreign or regional accents and vocal impediments in speech and typographical or grammatical errors in texts make the processing of the language an even more difficult task.

Speech acts. A sentence can often be considered an action by the speaker. The sentence structure alone may not contain enough information to define this action.

What are the most common tasks addressed by NLP? Following are among the most popular tasks: • Question answering.

• Automatic summarization.

• Natural language generation.

• Natural language understanding.

• Machine translation.

• Foreign language reading.

• Foreign language writing.

• Speech recognition.

• Text-to-speech.

7 .

• Text proofing.

• Optical character recognition.

Section 7.4 Review Questions 1.

List and briefly discuss some of the text mining applications in marketing.

Text mining can be used to increase cross-selling and up-selling by analyzing the unstructured data generated by call centers.

Text mining has become invaluable for customer relationship management.

Companies can use text mining to analyze rich sets of unstructured text data, combined with the relevant structured data extracted from organizational databases, to predict customer perceptions and subsequent purchasing behavior.

2.

How can text mining be used in security and counterterrorism?

Students may use the introductory case in this answer.

In 2007, EUROPOL developed an integrated system capable of accessing, storing, and analyzing vast amounts of structured and unstructured data sources in order to track transnational organized crime.

Another security-related application of text mining is in the area of deception detection.

3.

What are some promising text mining applications in biomedicine?

As in any other experimental approach, it is necessary to analyze this vast amount of data in the context of previously known information about the biological entities under study. The literature is a particularly valuable source of information for experiment validation and interpretation. Therefore, the development of automated text mining tools to assist in such interpretation is one of the main challenges in current bioinformatics research.

Section 7.5 Review Questions 1.

What are the main steps in the text mining process?

• • •

See Figure 7.6 (p. 309). Text mining entails three tasks: Establish the Corpus: Collect and organize the domain-specific unstructured data Create the Term–Document Matrix: Introduce structure to the corpus Extract Knowledge: Discover novel patterns from the T-D matrix 8 .

2.

What is the reason for normalizing word frequencies? What are the common methods for normalizing word frequencies?

The raw indices need to be normalized in order to have a more consistent TDM for further analysis. Common methods are log frequencies, binary frequencies, and inverse document frequencies.

3.

What is singular value decomposition? How is it used in text mining?

Singular value decomposition (SVD), which is closely related to principal components analysis, reduces the overall dimensionality of the input matrix (number of input documents by number of extracted terms) to a lower dimensional space, where each consecutive dimension represents the largest degree of variability (between words and documents) possible

4.

What are the main knowledge extraction methods from corpus?

The main categories of knowledge extraction methods are classification, clustering, association, and trend analysis.

Section 7.6 Review Questions 1.

What are some of the most popular text mining software tools?

1. ClearForest offers text analysis and visualization tools.

2. IBM offers SPSS Modeler and data and text analytics toolkits.

3. Megaputer Text Analyst offers semantic analysis of free-form text, summarization, clustering, navigation, and natural language retrieval with search dynamic refocusing.

4. SAS Text Miner provides a rich suite of text processing and analysis tools.

5. KXEN Text Coder (KTC) offers a text analytics solution for automatically preparing and transforming unstructured text attributes into a structured representation for use in KXEN Analytic Framework.

6. The Statistica Text Mining engine provides easy-to-use text mining functionally with exceptional visualization capabilities.

7. VantagePoint provides a variety of interactive graphical views and analysis tools with powerful capabilities to discover knowledge from text databases.

8. The WordStat analysis module from Provalis Research analyzes textual information such as responses to open-ended questions, interviews, etc.

9. Clarabridge text mining software provides end-to-end solutions for customer experience professionals wishing to transform customer feedback for marketing, service, and product improvements.

9 .

2.

Why do you think most of the text mining tools are offered by statistics companies?

Students should mention that many of the capabilities of data mining apply to text mining. Since statistics companies offer data mining tools, offering text mining is a natural business extension.

3.

What do you think are the pros and cons of choosing a free text mining tool over a commercial tool?

Free tools have fewer features, more difficult user-interfaces, lack support, and have slower or reduced processing capabilities. The advantage of free tools is obviously the cost.

Section 7.7 Review Questions 1.

What is sentiment analysis? How does it relate to text mining?

Sentiment analysis tries to answer the question, “What do people feel about a certain topic?” by digging into opinions of many using a variety of automated tools. It is also known as opinion mining, subjectivity analysis, and appraisal extraction Sentiment analysis shares many characteristics and techniques with text mining.

However, unlike text mining, which categorizes text by conceptual taxonomies of topics, sentiment classification generally deals with two classes (positive versus negative), a range of polarity (e.g., star ratings for movies), or a range in strength of opinion.

2.

What are the sources of data for sentiment analysis?

Common data sources for sentiment analysis include opinion-rich Internet resources such as social media outlets (e.g., Twitter, Facebook, etc.), online review sites, personal blogs, online communities, chat rooms, news groups, and search engine logs. For many companies, customer call center transcripts, emails, product reviews, and price comparison portals provide another rich source of sentiment data.

3.

What are the common challenges that sentiment analysis has to deal with?

Sentiment that appears in text comes in two flavors: explicit, where the subjective sentence directly expresses an opinion (“It’s a wonderful day”), and implicit, where the text implies an opinion (“The handle breaks too easily”). Implicit sentiment analysis is harder to analyze because it may not include words that are obviously evaluations or judgments. Another challenge involves the timeliness of 10 .

collection/analysis of textual data coming from a wide variety of data sources. A third challenge is the difficulty of identifying whether a piece of text involves sentiment or not, especially with implicit sentiment analysis. The same sorts of issues involving text mining in natural language settings also apply to sentiment analysis.

Section 7.8 Review Questions 1.

What are the most popular application areas for sentiment analysis? Why? Customer relationship management (CRM) and customer experience management are popular “voice of the customer (VOC)” applications. Other application areas include “voice of the market (VOM)” and “voice of the employee (VOE).”

2.

How can sentiment analysis be used for brand management?

Various areas related to brand management can benefit from sentiment analysis.

This includes many public and private sectors including financial markets, politics, and government intelligence. E-commerce sites, e-mail filtration/prioritization, and citation analysis are just some of the application areas that can benefit from the information derived from sentiment analysis.

3.

What would be the expected benefits and beneficiaries of sentiment analysis in politics?

Opinions matter a great deal in politics. Because political discussions are dominated by quotes, sarcasm, and complex references to persons, organizations, and ideas, politics is one of the most difficult, and potentially fruitful, areas for sentiment analysis. By analyzing the sentiment on election forums, one may predict who is more likely to win or lose. Sentiment analysis can help understand what voters are thinking and can clarify a candidate’s position on issues.

Sentiment analysis can help political organizations, campaigns, and news analysts to better understand which issues and positions matter the most to voters. The technology was successfully applied by both parties to the 2008 and 2012 American presidential election campaigns.

4.

How can sentiment analysis be used in predicting financial markets?

Many financial analysts believe that the stock market is mostly sentiment driven, so use of sentiment analysis has much relevance for financial markets. Automated analysis of market sentiments using social media, news, blogs, and discussion groups can help with predicting market movements. If done correctly, sentiment analysis can identify short-term stock movements based on the buzz in the market, potentially impacting liquidity and trading.

11 .

Section 7.9 Review Questions 1.

What are the main steps in carrying out sentiment analysis projects?

The first step when performing sentiment analysis of a text document is called sentiment detection, during which text data is differentiated between fact and opinion (objective vs. subjective). This is followed by negative-positive (N-P) polarity classification, where a subjective text item is classified on a bipolar range. Following this comes target identification (identifying the person, product, event, etc. that the sentiment is about). Finally come collection and aggregation, in which the overall sentiment for the document is calculated based on the calculations of sentiments of individual phrases and words from the first three steps.

2.

What are the two common methods for polarity identification? What is the main difference between the two?

Polarity identification can be done via a lexicon (as a reference library) or by using a collection of training documents and inductive machine learning algorithms. The lexicon approach uses a catalog of words, their synonyms, and their meanings, combined with numerical ratings indicating the position on the NP polarity associated with these words. In this way, affective, emotional, and attitudinal phrases can be classified according to their degree of positivity or negativity. By contrast, the training-document approach uses statistical analysis and machine learning algorithms, such as neural networks, clustering approaches, and decision trees to ascertain the sentiment for a new text document based on patterns from previous “training” documents with assigned sentiment scores.

3.

Describe how special lexicons are used in identification of sentiment polarity.

Special sentiment-related lexicons build upon general-purpose lexicons by including polarity (positive-negative) and objectivity (subjective-objective) labels for each term (and its set of synonyms, or synset) in the lexicon. An additional approach is to include other affective labels including emotion, cognitive state, attitude, feeling, etc.

Section 7.10 Review Questions 1.

What is speech analytics? How does it relate to sentiment analysis?

Speech analytics is a growing field of science that allows users to analyze and extract information from both live and recorded conversations. This technology can deliver meaningful and quantitative business intelligence through the analysis of the millions of recorded calls that occur in customer contact centers around the 12 .

world. With respect to sentiment analysis, speech analytics can help to assess the emotional states expressed in a conversation and on measuring the presence and strength of positive and negative feelings that are exhibited by the participants.

This can be a valuable tool for customer relationship management and agent training.

2.

Describe the acoustic approach to speech analytics.

The acoustic approach to sentiment analysis relies on extracting and measuring a specific set of features of the audio. Features include intensity, pitch, jitter, shimmer, glottal pulse, harmonics-to-noise ratio (HNR), and speaking rate. These features can give insight into the emotional and attitudinal state of the speaker.

3.

Describe the linguistic approach to speech analytics.

The linguistic approach focuses on the explicit indications of sentiment and context of the spoken content within the audio. In some ways, it is similar to the lexicon approach to polarity identification, in that it is using lexical analysis. In addition, it analyzes features of disfluencies (pauses, hesitations, laughter, etc.) as well as higher semantic features such as taxonomy/ontology, dialogue history, and pragmatics. Various methods can be used in the linguistic approach, including catching keywords with domain-specific sentiment significance, searching out linguistic elements that are predictors of particular sentiments, and phonetic indexing and search.

ANSWERS TO APPLICATION CASE QUESTIONS FOR DISCUSSION

 

Application Case 7.1: Text Mining for Patent Analysis 1.

Why is it important for companies to keep up with patent filings?

Patents provide exclusive rights to an inventor, granted by a country, for a limited period of time in exchange for a disclosure of an invention. The disclosure of these inventions is critical to future advancements in science and technology. If carefully analyzed, patent documents can help identify emerging technologies, inspire novel solutions, foster symbiotic partnerships, and enhance overall awareness of business’ capabilities and limitations. Patent filings are therefore important for companies that hope to remain competitive in a highly changing technological climate.

2.

How did Kodak use text analytics to better analyze patents?

Using dedicated analysts and state-of-the-art software tools (including specialized text mining tools from ClearForest Corp.), Kodak continuously digs deep into 13 .

various data sources (patent databases, new release archives, and product announcements) in order to develop a holistic view of the competitive landscape.

3.

What were the challenges, the proposed solution, and the obtained results?

Kodak’s challenges are to apply more than a century’s worth of knowledge about imaging science and technology to new uses and to secure those new uses with patents. The problem is that it is nearly impossible to efficiently process such enormous amounts of semistructured data (patent documents usually contain partially structured and partially textual data). But through the use of text mining tools, Kodak is able to obtain a wide range of benefits. These include enabling competitive intelligence, making critical business decisions, identifying and recruiting new talent, identifying unauthorized use of Kodak’s patents, identifying complementary inventions for forming symbiotic partnerships, and preventing competitors from creating similar products.

Application Case 7.2: Text Mining Improves Hong Kong Government’s Ability to Anticipate and Address Public Complaints 1.

How did the Hong Kong government use text mining to better serve its constituents?

The government built a Complaints Intelligence System in order to analyze social messages hidden in the complaints data, which provides important feedback on public service and highlights opportunities for service improvement. Utilizing SAS Text Miner, the government is able to better understand underlying issues and quickly respond even as situations evolve. Text mining also enables the government to quickly discover the correlations among some key words in the complaints.

2.

What were the challenges, the proposed solution, and the obtained results?

The major challenge facing the Hong Kong government was analyzing and responding to public complaints to their call center. This involves addressing answers to about 2.65 million calls and 98,000 e-mails per year. Originally, they attempted to compile reports on complaint statistics for reference by government departments manually. But through ‘eyeball’ observations, it was impossible to effectively reveal new or more complex potential public issues and identify their root causes, as most of the complaints were recorded in unstructured textual format. So, the government decided to utilize text processing and mining approaches to uncover trends, patterns, and relationships in the text data. This allowed the government to better understand the voice of the people, improve service delivery, make informed decisions, and develop smart strategies. The result was a boost in public satisfaction.

Application Case 7.3: Mining for Lies 14 .

1.

Why is it difficult to detect deception?

Humans tend to perform poorly at deception-detection tasks. This phenomenon is exacerbated in text-based communications. Although some people believe that they can readily identify those who are not being truthful, a summary of deception research showed that, on average, people are only 54 percent accurate in making veracity determinations.

2.

How can text/data mining be used to detect deception in text?

Through a process known as message feature mining, statements are transcribed for processing, then cues are extracted and selected. Text processing software identifies cues in statements and generates quantified cues. Classification models are trained and tested on quantified cues, and based on this, statements are labeled as truthful or deceptive (e.g., by law enforcement personnel). The featureselection methods along with 10-fold cross-validation allow researchers to compare the prediction accuracy of different data mining methods (for example, neural networks).

3.

What do you think are the main challenges for such an automated system?

One challenge is that the training the system depends on humans to ascertain the truthfulness of statements in the training data itself. You can’t know for sure whether these statements are true or false, so you may be using incorrect training samples when “teaching” the machine learning system to predict lies in new text data. (This answer will vary by student.)

Application Case 7.4: Text Mining and Sentiment Analysis Help Improve Customer Service Performance 1.

How did the financial services firm use text mining and text analytics to improve its customer service performance?

The company used PolyAnalyst’s text analysis tools for extracting complex word patterns, grammatical and semantic relationships, and expressions of sentiment.

The results were classified into context-specific themes to identify actionable issues. The relationships between structured fields and text analysis results were established to identify patterns and interactions. Results were presented via graphical, interactive, web-based reports. Actionable issues were assigned to relevant individuals responsible for their resolution, and criteria were established to capture and detect compliance with the company’s Quality Standards.

2.

What were the challenges, the proposed solution, and the obtained results?

15 .

Continually monitoring service levels is essential for service quality control.

Customer surveys and associate-customer interactions are the best way to obtain data for such monitoring. But manually evaluating this data is subjective, errorprone, and time/labor-intensive. Therefore, the company needed a system for (1) automatically evaluating associate-customer interactions for compliance with quality standards and (2) analyzing survey responses to extract positive and negative feedback, while allowing for the diversity of natural language expression. The solution was to use PolyAnalyst. (See answer to Question #1 for the remainder of the answer.)

Application Case 7.5: Research Literature Survey with Text Mining 1.

How can text mining be used to ease the task of literature review?

Text mining enables a semiautomated analysis of large volumes of published literature. Clustering was used in this study to identify the natural groupings of the articles, and list the most descriptive terms that characterized those clusters. This led to discovery and exploration of interesting patterns using tabular and graphical representation of their findings. Use of text and data mining can thus speed up and simplify the literature review process for academic researchers.

2.

What are the common outcomes of a text mining project on a specific collection of journal articles? Can you think of other potential outcomes not mentioned in this case?

Common outcomes include identifying natural clusters of similar articles, helping to identify the optimal number of cluster classifications. Using text mining, you can answer questions such as “Are there clusters that represent different research themes specific to a single journal?” and “Is there a time-varying characterization of those clusters?” Text mining also has other possible applications in literature reviews. For example, sentiment analysis can help to identify positive and negative judgments. Text mining can be used to build taxonomies of concepts and terms within and between research articles. You can find common themes by author as well as by journal.

Application Case 7.6: A Potpourri of Text Mining Case Synopses 1.

What do you think are the common characteristics of the kind of challenges these five companies were facing?

In all cases, the companies face the challenge of analyzing large amounts of unstructured data in text form. Time is also a critical element; this analysis must be done quickly and accurately. In most cases, the business challenge had to do with customer satisfaction; in one it had to do with fraud detection.

16 .

2.

What are the types of solution methods and tools proposed in these case synopses?

SAS Text Miner was the primary tool used in all cases. Additional tools (all from SAS) included Enterprise Miner and BI Server. So, the primary method used for solving these companies’ problems involved text and data mining and analytics.

3.

What do you think are the key benefits of using text mining and advanced analytics (compared to the traditional way to do the same)?

The key benefits include increased accuracy of analysis results, speed of response to potential problems, cost savings, enhanced customer relationships, improved product/service quality, and greater competitiveness.

Application Case 7.7: Whirlpool Achieves Customer Loyalty and Product Success with Text Analytics 1.

How did Whirlpool use capabilities of text analytics to better understand their customers and improve product offerings?

Whirlpool uses Attensity products for deep text analytics of their multi-channel customer data, which includes e-mails, CRM notes, repair notes, warranty data, and social media. The company uses text analytics solutions every day to get to the root cause of product issues and receive alerts on emerging issues. Users of Attensity’s analytics products at Whirlpool include product/ service managers, corporate/product safety staff, consumer advocates, service quality staff, innovation managers, the Category Insights team, and all of Whirlpool’s manufacturing divisions (across five countries).

2.

What were the challenges, the proposed solution, and the obtained results?

Customer satisfaction and feedback are at the center of how Whirlpool drives its overarching business strategy, so gaining insight into customer satisfaction and product feedback is paramount. Whirlpool needed to more effectively understand and react to customer and product feedback data, originating from blogs, e-mails, reviews, forums, repair notes, and other data sources. Managers needed to report on longitudinal data, and be able to compare issues by brand over time. The solution was to use Attensity’s Text Analytics application. This enabled the company to more proactively identify and mitigate quality issues before issues escalated and claims were filed. Since then, Whirlpool has also been able to avoid recalls, which increased customer loyalty and reduced costs (realizing 80% savings on their costs of recalls due to early detection).

Application Case 7.8: Cutting Through the Confusion: Blue Cross Blue Shield of North Carolina Uses Nexidia’s Speech Analytics to Ease Member Experience in Healthcare 17 .

1.

For a large company like BCBSNC with a lot of customers, what does “listening to customers” mean?

Listening to customers often means understanding the confusion of its members regarding benefits. Members reach out via the BCBSNC’s contact center, and it is important to make this a timely and cost-efficient process. Part of this involves use of customer interaction and speech analytics.

2.

What were the challenges, the proposed solution, and the obtained results for BCBSNC? Contact center calls are costly and time-consuming, especially when dealing with members’ “confusion calls.” This causes reductions in customer satisfaction.

Asking customer service professionals to more thoroughly document the nature of the calls within the contact center desktop application is not a viable option because this is largely a manual effort (using a desktop application), which adds significantly to labor costs. BCBSNC’s solution was to leverage its partnership with Nexidia, a leading provider of customer interaction analytics, and use speech analytics to better understand the cause and depth of member confusion. The company used sentiment analysis (utilizing the linguistic approach) to better understand how members perceived the value they received from BCBSNC and their overall opinion of the company. Based on what BCBSNC learned, the company implemented strategies to improve member communication and customer experience. This included development of more reader-friendly literature and website redesigns to support easier navigation and education. As a result, BCBSNC projects a 10 to 25 percent drop in “confusion calls,” resulting in a better customer service experience and a lower cost to serve.

ANSWERS TO END OF CHAPTER QUESTIONS FOR DISCUSSION

  

1. Explain the relationship among data mining, text mining, and sentiment analysis.

Technically speaking, data mining is a process that uses statistical, mathematical, and artificial intelligence techniques to extract and identify useful information and subsequent knowledge (or patterns) from large sets of data. Data mining is the general concept. Text mining is a specific application of data mining: applying it to unstructured text files. Sentiment analysis is a specialized form of text and data mining that identifies and classifies terms in text sources according to sentiment (e.g. judgment, opinion, and emotional content).

2. What should an organization consider before making a decision to purchase text mining software?

18 .

Before making a decision to purchase any mining software organizations should consider the standard criteria to use when investing in any major software: cost/benefit analysis, people with the expertise to use the software and perform the analyses, availability of data/information, and a business need for the software and capabilities.

3.

Discuss the differences and commonalities between text mining and sentiment analysis.

Text mining is a specific application of data mining: applying it to unstructured text files. Sentiment analysis is a specialized form of text and data mining that identifies and classifies terms in text sources according to sentiment (e.g. judgment, opinion, and emotional content).

4.

In your own words, define text mining and discuss its most popular applications.

Students’ answers will vary.

5.

Discuss the similarities and differences between the data mining process (e.g., CRISP-DM) and the three-step, high-level text mining process explained in this chapter.

CRISP-DM provides a systematic and orderly way to conduct data mining projects. This process has six steps. First, an understanding of the data and an understanding of the business issues to be addressed are developed concurrently.

Next, data are prepared for modeling; are modeled; the model results are evaluated; and the models can be employed for regular use.

• • •

6.

Text mining entails three tasks: See Figure 7.6 (p. 309).

Establish the Corpus: Collect and organize the domain-specific unstructured data Create the Term–Document Matrix: Introduce structure to the corpus Extract Knowledge: Discover novel patterns from the T-D matrix

What does it mean to introduce structure into the text-based data? Discuss the alternative ways of introducing structure into text-based data.

Text mining, like other data mining approaches, are inductive approaches for finding patterns and trends in data. One difference between text mining and other data mining approaches is the use of natural language processing.

Text-based data is inherently unstructured and must be converted to a structured format for predictive modeling or other type of analysis. The third step of the 3step text mining process utilizes inductive algorithms to classify and structure the corpus of text sources so that knowledge can be extracted.

19 .

Four possible approaches for inducing structure in text in order to extract knowledge are (a) classification (grouping terms into predefined categories), (b) clustering (coming up with “natural” groupings), (c) association rule learning (finding frequent combinations of terms), and (d) trend analysis (recognizing concept distributions based on specific collections of documents).

7.

What is the role of natural language processing in text mining? Discuss the capabilities and limitations of NLP in the context of text mining.

NLP is an important component of text mining. It studies the problem of “understanding” the natural human language, with the view of converting depictions of human language (such as textual documents) into more formal representations (in the form of numeric and symbolic data) that are easier for computer programs to manipulate. The goal of NLP is to move beyond syntaxdriven text manipulation (which is often called “word counting”) to a true understanding and processing of natural language that considers grammatical and semantic constraints as well as the context.

8.

List and discuss three prominent application areas for text mining. What is the common theme among the three application areas you chose?

National security and counterterrorism: The FBI’s system is expected to create a gigantic data warehouse along with a variety of data and text mining modules to meet the knowledge discovery needs of federal, state, and local law enforcement agencies.

Biomedical: A system extracts disease–gene relationships from literature accessed via MEDLINE. Marketing: Text mining can be used to increase cross-selling and up-selling by analyzing the unstructured data generated by call centers.

The value of all of these applications is significantly increased by extracting knowledge from huge volumes of text-data and documents through the use of text mining tools.

9.

What is sentiment analysis? How does it relate to text mining?

Sentiment analysis tries to answer the question, “What do people feel about a certain topic?” by digging into opinions of many using a variety of automated tools. It is also known as opinion mining, subjectivity analysis, and appraisal extraction.

Sentiment analysis shares many characteristics and techniques with text mining.

However, unlike text mining, which categorizes text by conceptual taxonomies of topics, sentiment classification generally deals with two classes (positive versus 20 .

negative), a range of polarity (e.g., star ratings for movies), or a range in strength of opinion.

10.

What are the sources of data for sentiment analysis?

Common data sources for sentiment analysis include opinion-rich Internet resources such as social media outlets (e.g., Twitter, Facebook, etc.), online review sites, personal blogs, online communities, chat rooms, news groups, and search engine logs. For many companies, customer call center transcripts, emails, product reviews, and price comparison portals provide another rich source of sentiment data.

11.

What are the common challenges that sentiment analysis has to deal with?

Sentiment that appears in text comes in two flavors: explicit, where the subjective sentence directly expresses an opinion (“It’s a wonderful day”), and implicit, where the text implies an opinion (“The handle breaks too easily”). Implicit sentiment analysis is harder to analyze because it may not include words that are obviously evaluations or judgments. Another challenge involves the timeliness of collection/analysis of textual data coming from a wide variety of data sources. A third challenge is the difficulty of identifying whether a piece of text involves sentiment or not, especially with implicit sentiment analysis. The same sorts of issues involving text mining in natural language settings also apply to sentiment analysis.

12.

What are the most popular application areas for sentiment analysis? Why? Customer relationship management (CRM) and customer experience management are popular “voice of the customer (VOC)” applications. Other application areas include “voice of the market (VOM)” and “voice of the employee (VOE).”

13.

How can sentiment analysis be used for brand management?

Various areas related to brand management can benefit from sentiment analysis.

This includes many public and private sectors including financial markets, politics, and government intelligence. E-commerce sites, e-mail filtration/prioritization, and citation analysis are just some of the application areas that can benefit from the information derived from sentiment analysis.

14.

What would be the expected benefits and beneficiaries of sentiment analysis in politics?

Opinions matter a great deal in politics. Because political discussions are dominated by quotes, sarcasm, and complex references to persons, organizations, and ideas, politics is one of the most difficult, and potentially fruitful, areas for sentiment analysis. By analyzing the sentiment on election forums, one may 21 .

predict who is more likely to win or lose. Sentiment analysis can help understand what voters are thinking and can clarify a candidate’s position on issues.

Sentiment analysis can help political organizations, campaigns, and news analysts to better understand which issues and positions matter the most to voters. The technology was successfully applied by both parties to the 2008 and 2012 American presidential election campaigns.

15.

How can sentiment analysis be used in predicting financial markets?

Many financial analysts believe that the stock market is mostly sentiment driven, so use of sentiment analysis has much relevance for financial markets. Automated analysis of market sentiments using social media, news, blogs, and discussion groups can help with predicting market movements. If done correctly, sentiment analysis can identify short-term stock movements based on the buzz in the market, potentially impacting liquidity and trading.

16.

What are the main steps in carrying out sentiment analysis projects?

The first step when performing sentiment analysis of a text document is called sentiment detection, during which text data is differentiated between fact and opinion (objective vs. subjective). This is followed by negative-positive (N-P) polarity classification, where a subjective text item is classified on a bipolar range. Following this comes target identification (identifying the person, product, event, etc. that the sentiment is about). Finally come collection and aggregation, in which the overall sentiment for the document is calculated based on the calculations of sentiments of individual phrases and words from the first three steps.

17.

What are the two common methods for polarity identification? What is the main difference between the two?

Polarity identification can be done via a lexicon (as a reference library) or by using a collection of training documents and inductive machine learning algorithms. The lexicon approach uses a catalog of words, their synonyms, and their meanings, combined with numerical ratings indicating the position on the NP polarity associated with these words. In this way, affective, emotional, and attitudinal phrases can be classified according to their degree of positivity or negativity. By contrast, the training-document approach uses statistical analysis and machine learning algorithms, such as neural networks, clustering approaches, and decision trees to ascertain the sentiment for a new text document based on patterns from previous “training” documents with assigned sentiment scores.

18.

Describe how special lexicons are used in identification of sentiment polarity.

Special sentiment-related lexicons build upon general-purpose lexicons by including polarity (positive-negative) and objectivity (subjective-objective) labels 22 .

for each term (and its set of synonyms, or synset) in the lexicon. An additional approach is to include other affective labels including emotion, cognitive state, attitude, feeling, etc.

19.

What is speech analytics? How does it relate to sentiment analysis?

Speech analytics is a growing field of science that allows users to analyze and extract information from both live and recorded conversations. This technology can deliver meaningful and quantitative business intelligence through the analysis of the millions of recorded calls that occur in customer contact centers around the world. With respect to sentiment analysis, speech analytics can help to assess the emotional states expressed in a conversation and on measuring the presence and strength of positive and negative feelings that are exhibited by the participants.

This can be a valuable tool for customer relationship management and agent training.

20.

Describe the acoustic approach to speech analytics.

The acoustic approach to sentiment analysis relies on extracting and measuring a specific set of features of the audio. Features include intensity, pitch, jitter, shimmer, glottal pulse, harmonics-to-noise ratio (HNR), and speaking rate. These features can give insight into the emotional and attitudinal state of the speaker.

21.

Describe the linguistic approach to speech analytics.

The linguistic approach focuses on the explicit indications of sentiment and context of the spoken content within the audio. In some ways, it is similar to the lexicon approach to polarity identification, in that it is using lexical analysis. In addition, it analyzes features of disfluencies (pauses, hesitations, laughter, etc.) as well as higher semantic features such as taxonomy/ontology, dialogue history, and pragmatics. Various methods can be used in the linguistic approach, including catching keywords with domain-specific sentiment significance, searching out linguistic elements that are predictors of particular sentiments, and phonetic indexing and search.

ANSWERS TO END OF CHAPTER APPLICATION CASE QUESTIONS 1.

 

How did BBVA use text mining?

BBVA used text mining and sentiment analysis to help the company understand what existing and potential clients say about it through social media. This enabled the company to monitor its reputation and make improvements when needed.

BBVA used an IBM social media research asset called Corporate Brand

23 .

Reputation Analysis (COBRA). The company followed up by implementing IBM Cognos Consumer Insight to unify all its branches worldwide.

2.

What were BBVA’s challenges? How did BBVA overcome them with text mining and social media analysis?

BBVA’s challenge was to better manage its reputation in the face of myriads of comments (both positive and negative) about BBVA that were being posted on social media sites, blogs, and other commentaries worldwide. This challenge was overcome by partnering with IBM to implement a global corporate-wide system for reputation analysis. The results were a one percent increase in positive comments and a 1.5 percent reduction in negative comments. In addition, BBVA was better able to monitor reputations, respond quickly to “reputation risk” indicators, unify the online measuring of its business strategies, and enable more detailed, structured, and controlled online data analysis.

3.

In what other areas, in your opinion, can BBVA use text mining?

(This answer will vary by student. Here are two options.)

•

•

Another possible way to use text mining is for analyzing customer queries regarding BBVA’s products and services. By analyzing e-mails and client phone conversations, BBVA can identify common customer issues and thereby modify their customer service, as well as improve on quality of their products and services.

Another possible way to use text mining is for analyzing news and financial literature in order to better predict financial trends.

24 .

Chapter 8: Web Analytics, Web Mining, and Social Analytics Learning Objectives for Chapter 8 1.

2.

3.

4.

5.

6.

7.

8.

9.

Define Web mining and understand its taxonomy and its application areas Differentiate between Web content mining and Web structure mining Understand the internals of Web search engines Learn the details about search engine optimization Define Web usage mining and learn its business application Describe the Web analytics maturity model and its use cases Understand social networks and social analytics and their practical applications Define social network analysis and become familiar with its application areas Understand social media analytics and its use for better customer engagement

CHAPTER OVERVIEW This chapter is all about Web mining and its application areas. As you will see, Web mining is one of the fastest growing technologies in business intelligence and business analytics. Under the umbrella of Web mining, in this chapter, we will cover Web analytics, search engines, social analytics and their enabling methods, algorithms, and technologies.

CHAPTER OUTLINE 8.1

OPENING VIGNETTE: SECURITY FIRST INSURANCE DEEPENS CONNECTION WITH POLICYHOLDERS  Questions for the Opening Vignette A. WHAT WE CAN LEARN FROM THIS VIGNETTE

8.2

WEB MINING OVERVIEW  Section 8.2 Review Questions

8.3

WEB CONTENT AND WEB STRUCTURE MINING ⧫ Application Case 8.1: Identifying Extremist Groups with Web Link and Content Analysis  Section 8.3 Review Questions

1 .

8.4

SEARCH ENGINES A. ANATOMY OF A SEARCH ENGINE B. 1. DEVELOPMENT CYCLE C. WEB CRAWLER D. DOCUMENT INDEXER 1.

Step 1: Preprocessing the Documents 2.

Step 2: Parsing the Documents 3.

Step 3: Creating the Term-by-Document Matrix E. 2. RESPONSE CYCLE F. QUERY ANALYZER G. DOCUMENT MATCHER/RANKER ⧫ Technology Insights 8.1: PageRank Algorithm H. HOW DOES GOOGLE DO IT? ⧫ Application Case 8.2: IGN Increases Search Traffic by 1500 Percent  Section 8.4 Review Questions

8.5

SEARCH ENGINE OPTIMIZATION ⧫ Technology Insights 8.2: Top 15 Most Popular Search Engines (March 2013) A. METHODS FOR SEARCH ENGINE OPTIMIZATION ⧫ Application Case 8.3: Understanding Why Customers Abandon Shopping Carts Results in $10 Million Sales Increase  Section 8.5 Review Questions

8.6

WEB USAGE MINING (WEB ANALYTICS) A. WEB ANALYTICS TECHNOLOGIES ⧫ Application Case 8.4: Allegro Boosts Online Click-Thru Rates by 500 Percent with Web Analysis B. WEB ANALYTICS METRICS C. WEBSITE USABILITY D. TRAFFIC SOURCES E. VISITOR PROFILES F. CONVERSION STATISTICS  Section 8.6 Review Questions

8.7

WEB ANALYTICS MATURITY MODEL AND WEB ANALYTICS TOOLS A. WEB ANALYTICS TOOLS 1.

Google Web Analytics (Google.com/Analytics) 2.

Yahoo! Web Analytics (Web.Analytics.Yahoo.com) 3.

Open Web Analytics (Openwebanalytics.com) 4.

Piwik (Piwik.erg) 5.

FireStat (FireStats.cc) 6.

Site Meter (Sitemeter.com) 7.

Woopra (Woopra.com) 2 .

B. C.

8.

Awstats (Awstats.org) 9.

Snoop (Reinvigorate.net) 10.

Mochibot (Mochibot.com) PUTTING IT ALL TOGETHER—A WEBSITE OPTIMIZATION ECOSYSTEM A FRAMEWORK FOR VOICE OF THE CUSTOMER STRATEGY  Section 8.7 Review Questions

8.8

SOCIAL ANALYTICS AND SOCIAL NETWORK ANALYSIS A. SOCIAL NETWORK ANALYSIS 1.

Communication Networks 2.

Community Networks 3.

Criminal Networks 4.

Innovation Networks B. SOCIAL NETWORK ANALYSIS METRICS ⧫ Application Case 8.5: Social Network Analysis Helps Telecommunication Firms C. CONNECTIONS D. DISTRIBUTIONS E. SEGMENTATION  Section 8.8 Review Questions

8.9

SOCIAL MEDIA DEFINITIONS AND CONCEPTS A. HOW DO PEOPLE USE SOCIAL MEDIA? ⧫ Application Case 8.6: Measuring the Impact of Social Media at Lollapalooza  Section 8.9 Review Questions

8.10

SOCIAL MEDIA ANALYTICS A. MEASURING THE SOCIAL MEDIA IMPACT B. BEST PRACTICES IN SOCIAL MEDIA ANALYTICS 1.

Think of Measurement as a Guidance System, Not a Rating System 2.

Track the Elusive Sentiment 3.

Continuously Improve the Accuracy of Text Analysis 4.

Look at the Ripple Effect 5.

Look Beyond the Brand 6.

Identify Your Most Powerful Influencers 7.

Look Closely at the Accuracy of Your Analytic Tool 8.

Incorporate Social Media Intelligence into Planning ⧫ Application Case 8.7: eHarmony Uses Social Media to Help Take the Mystery out of Online Dating C. SOCIAL MEDIA ANALYTICS TOOLS AND VENDORS 1.

Attensity360 2.

Radian 6/Salesforce Cloud 3.

Sysomos 3 .

4.

5.

6.

7.

8.

9.

10.

Collective Intellect Webtrends Crimson Hexagon Converseon Spiral16 BuzzLogic SproutSocial  Section 8.10 Review Questions

Chapter Highlights Key Terms Questions for Discussion Exercises Teradata University Network (TUN) and Other Hands-on Exercises Team Assignments and Role-Playing Projects Internet Exercises ⧫ End of Chapter Application Case: Keeping Students on Track with Web and Predictive Analytics  Questions for the Case References

TEACHING TIPS/ADDITIONAL INFORMATION

       

Web analytics and mining, along with social network analysis and social media analytics, are topics that students find especially interesting. They can relate to the impact of social media in their lives, and are very tuned in to the Internet as a social community. The opening vignette discusses Security First, an insurance company that uses Web mining and social media analytics to assist with processing huge volumes of claims and communications on social media during a hurricane.

The chapter gives a comprehensive overview of how search engines work and the methods used for ranking pages in response to user queries as well as the methods for search engine optimization. Students will enjoy learning about social analytics and social network analysis, and can relate it to other classes they may have taken in social sciences.

ANSWERS TO END OF SECTION REVIEW QUESTIONS Section 8.1 Review Questions 1.

What does Security First do?

4 .

     

Security First Insurance is one of the largest homeowners’ insurance companies in Florida. Headquartered in Ormond Beach, it employs more than 80 insurance professionals to serve its nearly 190,000 customers. Security First is one of a few Florida homeowners’ insurance companies that has the financial strength to withstand multiple natural disasters.

2.

What were the main challenges Security First was facing?

During a typical month, Security First processes 700 claims. However, in the aftermath of a hurricane, that number can swell to tens of thousands within days.

It was a challenge for the company to quickly scale up to handle the influx of customers trying to file post-storm insurance claims, via phone, email, Facebook, or Twitter.

3.

What was the solution approach? What types of analytics were integrated in the solution?

Security First contacted IBM Business Partner Integritie for help with harnessing social media to improve the customer experience. Integritie configured a solution called Social Media Capture built on key IBM Enterprise Content Management and Content Analytics tools. The Integritie solution offers four critical capabilities for managing social media platforms: capture, control, compliance, and communication.

4.

Based on what you learn from the vignette, what do you think are the relationships between Web analytics, text mining, and sentiment analysis?

As we learned in Chapter 7, sentiment analysis can be thought of as a specialized form of text mining, geared specifically at feeling, opinion, and judgment. Web analytics is a broad area including Web content, structure, and usage mining, including both text mining and social network analysis. Much of Web analytics involves social media analysis.

5.

What were the results Security First obtained? Were any surprising benefits realized?

The text analysis capabilities provided in the IBM software help Security First filter inappropriate incoming communications and audit outbound communications, avoiding potential issues with message content. This ensured compliance with industry regulations and Security First policies. Security First had the confidence to expand its use of social media. Because compliance is covered, the company can focus on additional opportunities for direct dialog with customers. Before this solution, Security First filtered customer communications through agents. Now it can reach out to customers directly and proactively as a company.

5 .

Section 8.2 Review Questions 1.

What are some of the main challenges the Web poses for knowledge discovery?

• The Web is too big for effective data mining.

• The Web is too complex.

• The Web is too dynamic.

• The Web is not specific to a domain.

• The Web has everything.

2.

What is Web mining? How does it differ from regular data mining or text mining?

Web mining is the discovery and analysis of interesting and useful information from the Web and about the Web, usually through Web-based tools. Text mining is less structured because it’s based on words instead of numeric data.

3.

What are the three main areas of Web mining?

The three main areas of Web mining are Web content mining, Web structure mining, and Web usage (or activity) mining.

4.

Identify three application areas for Web mining (at the bottom of Figure 8.1).

Based on your own experiences, comment on their use cases in business settings.

(Since there are several application areas, this answer will vary for different students. Following is one possible answer.)

Three possible application areas for Web mining include sentiment analysis, clickstream analysis, and customer analytics. Clickstream analysis helps to better understand user behavior on a website. Sentiment analysis helps us understand the opinions and affective state of users on a system. Customer analytics helps to provide solutions for sales, service, marketing, and product teams, and optimize the customer life cycles. The use cases for these applications center on user experience, and primarily affect customer service and customer relationship management functions of an organization.

Section 8.3 Review Questions 1.

What is Web content mining? How can it be used for competitive advantage?

Web content mining refers to the extraction of useful information from Web pages. The documents may be extracted in some machine-readable format so that automated techniques can generate some information about the Web pages.

Collecting and mining Web content can be used for competitive intelligence 6 .

(collecting intelligence about competitors’ products, services, and customers), which can give your organization a competitive advantage.

2.

What is an “authoritative page”? What is a hub? What is the difference between the two?

A search on the Web to obtain information on a specific topic usually returns a few relevant, high-quality Web pages and a larger number of unusable Web pages. Use of an index based on authoritative pages (or some measure of it) will improve the search results and ranking of relevant pages.

The structure of Web hyperlinks has led to another important category of Web pages called a hub. A hub is one or more Web pages that provide a collection of links to authoritative pages.

Whereas an authoritative page is specific to a particular topic, a hub is a central location for accessing several authoritative pages. Authoritative pages may have many links to them, whereas hub pages may not be prominent and only a few links may point to them.

3.

What is Web structure mining? How does it differ from Web content mining?

Web structure mining is the process of extracting useful information from the links embedded in Web documents. By contrast, Web content mining involves analysis of the specific textual content of web pages. So, Web structure mining is more related to navigation through a website, whereas Web content mining is more related to text mining and the document hierarchy of a particular web page.

Section 8.4 Review Questions 1.

What is a search engine? Why are they important for today’s businesses?

A search engine is a software program that searches for documents (Internet sites or files) based on the keywords (individual words, multi-word terms, or a complete sentence) that users have provided that have to do with the subject of their inquiry. This is the most prominent type of information retrieval system for finding relevant content on the Web. Search engines have become the centerpiece of most Internet-based transactions and other activities. Because people use them extensively to learn about products and services, it is very important for companies to have prominent visibility on the Web; hence the major effort of companies to enhance their search engine optimization (SEO).

2.

What is the relationship between search engines and text mining?

7 .

Many of the concepts and techniques related to text analytics and text mining also apply to the tasks performed by search engines. The goal of a search engine is to return a rank-ordered list of documents/pages that best match the user’s query.

Therefore, parsing and understanding the query is important, as well as parsing the pages that are being examined during the search. Therefore, text mining techniques are important for maximizing the effectiveness and accuracy of search results.

3.

What are the two main cycles in search engines? Describe the steps in each cycle.

The two main cycles performed by search engines are: (a) the development cycle, and (b) the response cycle. The main components of the development cycle are the Web crawler and the document indexer. For document indexing, there are three main steps. The first involves preprocessing the document, to convert them into a standard format. Next comes parsing the document, through the use of text mining techniques. Finally, a term-by-document matrix is created through which weighted relationships of terms and documents are established. For the response cycle, the main components are query analysis (converting the user’s query into a standardized data structure) and document matching/ranking (matching the structured query against the document database to find and rank the most relevant documents/pages).

4.

What is a web crawler? What is it used for? How does it work?

A Web crawler (also called a spider or a Web spider) is a piece of software that systematically browses (crawls through) the World Wide Web for the purpose of finding and fetching Web pages. It starts with a list of “seed” URLs, goes to the pages of those URLs, and then follows each page’s hyperlinks, adding them to the search engine’s database. Thus, the Web crawler navigates through the Web in order to construct the database of websites.

5.

How does a query analyzer work? What is PageRank algorithm and how does it work?

A query analyzer parses the search string into individual words/terms using a series of tasks that include tokenization, removal of stop words, stemming, and word/term disambiguation (identification of spelling errors, synonyms, and homonyms). PageRank is a link analysis algorithm, named after Google cofounder Larry Page, used by the Google search engine. It rank-orders documents/pages based on their relevance and value/importance. The rank for a given page involves a formula based on hyperlinks leading into it and is similar to the types of rankings involved with citation analysis of academic journal articles.

Section 8.5 Review Questions

8 .

1.

What is “search engine optimization”? Who benefits from it?

Search engine optimization (SEO) is the intentional activity of affecting the visibility of an e-commerce site or a website in a search engine’s natural (unpaid or organic) search results. It involves editing a page’s content, HTML, metadata, and associated coding to both increase its relevance to specific keywords and to remove barriers to the indexing activities of search engines. In addition, SEO efforts include promoting a site to increase its number of inbound links. SEO primarily benefits companies with e-commerce sites by making their pages appear toward the top of search engine lists when users query.

2.

Describe the old and new ways of indexing performed by search engines.

In the early days, in order to be indexed, Webmasters would submit the URL of a page to a search engine, which would then send a Web spider to crawl the page, extract links to other pages from it, and return information found on the page to the server for indexing. The search engine’s spider would download a page and store it on its own server. Then the search engine's indexer would extract various information about the page, such as the words it contains and where these are located, as well as any weight for specific words, and all links the page contains.

These would then be placed into a scheduler for crawling at a later date. Current search engines no longer rely on Webmasters submitting URLs. Instead, they proactively and continuously crawl the Web for finding, fetching, and indexing everything about it.

3.

What are the things that help Web pages rank higher in the search engine results?

Cross-linking between pages of the same website to provide more links to the most important pages may improve its visibility. Writing content that includes frequently searched keyword phrases, so as to be relevant to a wide variety of search queries, will tend to increase traffic. Updating content so as to keep search engines crawling back frequently can give additional weight to a site. Adding relevant keywords to a Web page’s metadata, including the title tag and metadescription, will tend to improve the relevancy of a site’s search listings, thus increasing traffic. URL normalization of Web pages so that they are accessible via multiple URLs. Using canonical link elements and redirects can help make sure links to different versions of the URL all count toward the page’s link popularity score.

4.

What are the most commonly used methods for search engine optimization?

The book classifies two types of SEO method, “white hat” and “black hat.” White hat techniques are those that search engines recommend as part of good site design. White-hat advice is generally summed up as creating content for users, not for search engines, and then making that content easily accessible to the spiders,

9 .

rather than attempting to trick the algorithm from its intended purpose. See the answer to question #3 for more details on this approach.

Black hat techniques are disapproved by search engines, and involve deceptive practices, sometimes called “spamdexing.” One black-hat technique uses text that is hidden, either as text colored similar to the background or positioned offscreen. Another method gives a different page depending on whether the page is being requested by a human visitor or a search engine, a technique known as cloaking.

Section 8.6 Review Questions 1.

What are the three types of data generated through Web page visits?

• • •

2.

Automatically generated data stored in server access logs, referrer logs, agent logs, and client-side cookies User profiles Metadata, such as page attributes, content attributes, and usage data.

What is clickstream analysis? What is it used for?

Analysis of the information collected by Web servers can help us better understand user behavior. Analysis of this data is often called clickstream analysis. By using the data and text mining techniques, a company might be able to discern interesting patterns from the clickstreams.

3.

What are the main applications of Web mining?

• • • • • •

4.

Determine the lifetime value of clients.

Design cross-marketing strategies across products.

Evaluate promotional campaigns.

Target electronic ads and coupons at user groups based on user access patterns.

Predict user behavior based on previously learned rules and users’ profiles.

Present dynamic information to users based on their interests and profiles.

What are commonly used Web analytics metrics? What is the importance of metrics?

There are four main categories of Web analytic metrics: • •

Website usability: How were they using my website? These involve page views, time on site, downloads, click map, and click paths.

Traffic sources: Where did they come from? These include referral websites, search engines, direct, offline campaigns, and online campaigns.

10 .

• •

Visitor profiles: What do my visitors look like? These include keywords, content groupings, geography, time of day, and landing page profiles.

Conversion statistics: What does all this mean for the business? Metrics include new visitors, returning visitors, leads, sales/conversions, and abandonments.

These metrics are important because they provide access to a lot of valuable marketing data, which can be leveraged for better insights to grow your business and better document your ROI. The insight and intelligence gained from Web analytics can be used to effectively manage the marketing efforts of an organization and its various products or services.

Section 8.7 Review Questions 1.

What is a maturity model?

A maturity model is a formal depiction of critical dimensions and their competency levels of a business practice. Collectively, these dimensions and levels define the maturity level of an organization in that area of practice. It often describes an evolutionary improvement path from ad hoc, immature practices to disciplined, mature processes with improved quality and efficiency.

2.

List and comment on the six stages of TDWI’s BI maturity framework.

The TDWI BI Maturity Model involves the following six stages: Management Reporting, Spreadmarts, Data Marts, Data Warehouse, Enterprise Data Warehouse, and BI Services. This maturity model pertains specifically to data warehousing, and its main purpose is to gauge where organization data warehousing initiatives are at a point in time and where it should go next.

3.

What are the six dimensions used in Hamel’s Web analytics maturity model?

Hamel’s six dimensions include: (1) Management, Governance and Adoption, (2) Objectives Definition, (3) Scoping, (4) The Analytics Team and Expertise, (5) The Continuous Improvement Process and Analysis Methodology, (6) Tools, Technology and Data Integration. Each of these is rated by proficiency levels (impaired, initiated, operational, integrated, competitor, addicted).

4.

Describe Attensity’s framework for VOC strategy. List and describe the four stages.

Attensity’s Voice of Customer (VOC) strategy is called LARA, for Listen, Analyze, Relate, and Act. The first stage is to listen (both to the open Web and to enterprise information), and involves scraping, searching, and data integration.

The next stage, analyzing the unstructured data, involves use of text analytics 11 .

(keyword, statistical, and natural language processing) for tagging terms and their relationships. Then comes relating, during which you relate the analysis of the unstructured Web data to your enterprise’s structured data (customers, products, parts, locations, etc.). Based on the accomplishments from listening, analyzing, and relating, you can then act.

Section 8.8 Review Questions 1.

What is meant by social analytics? Why is it an important business topic?

From a philosophical perspective, social analytics focuses on a theoretical object called a “socius,” a kind of “commonness” that is neither a universal account nor a communality shared by every member of a body. Thus, social analytics in this sense attempts to articulate the differences between philosophy and sociology.

From a BI perspective, social analytics involves “monitoring, analyzing, measuring and interpreting digital interactions and relationships of people, topics, ideas and content.” In this perspective, social analytics involves mining the textual content created in social media (e.g., sentiment analysis, natural language processing) and analyzing socially established networks (e.g., influencer identification, profiling, prediction). This is an important business topic because it helps companies gain insight about existing and potential customers’ current and future behaviors, and about the likes and dislikes toward a firm’s products and services.

2.

What is a social network? What is social network analysis?

A social network is a social structure composed of individuals/people (or groups of individuals or organizations) linked to one another with some type of connections/relationships. Social network analysis (SNA) is the systematic examination of social networks. Dating back to the 1950s, social network analysis is an interdisciplinary field that emerged from social psychology, sociology, statistics, and graph (network) theory.

3.

List and briefly describe the most common social network types.

There are several social network types. Communication networks involve the transfer of information from one source to another. Community networks were traditionally geographically based, but with social media online communities have now evolved. Criminal networks are watched/pursued by security agencies using state-of-the-art Internet tools and tactics. Innovation networks affect the spread of an innovation and innovative behavior in a society.

4.

List and briefly describe the social network analysis metrics.

12 .

SNA metrics are often grouped into three categories: connections, distributions, and segmentation. Connection metrics include homophily (groupings based on similarity), multiplexity (multiple connection types, such as friends who also work together), mutuality/reciprocity, network closure (transitivity), and propinquity (geographic closeness). Distribution metrics include bridge (weak ties linked by single individuals), centrality (importance or influence of an individual of the group), density (proportion of direct ties between individuals in the group), distance, structural holes (absence of ties between two members of a group), and tie strength. Segmentation metrics include cliques (every individual in the group connected to every other), clustering coefficient (likelihood of direct connection between two members of the group), and cohesion (minimum number of members whose departure would discontinue the group).

Section 8.9 Review Questions 1.

What is social media? How does it relate to Web 2.0?

Social media refers to the enabling technologies of social interactions among people in which they create, share, and exchange information, ideas, and opinions in virtual communities and networks. It is a group of Internet-based software applications that build on the ideological and technological foundations of Web 2.0, and that allow the creation and exchange of user-generated content.

2.

What are the differences and commonalities between Web-based social media and traditional/industrial media?

Social media is typically less expensive and more accessible than industrial media. The quality of social media varies widely, compared to stricter quality standards in industrial media. Both have global reach, but industrial media is more centralized and hierarchical and social media more distributed. Content update takes place more frequently in social media. Industrial media tends to require more training and expertise to use.

3.

How do people use social media? What are the evolutionary levels of engagement?

People continue to make increasing use of social media. The evolutionary levels of engagement are: inactive, spectator, collector, joiner, critic, and creator.

Section 8.10 Review Questions 1.

What is social media analytics? What type of data is analyzed with it?

13 .

Social media analytics refers to the systematic and scientific ways to consume the vast amount of content created by Web-based social media outlets, tools, and techniques for the betterment of an organization’s competitiveness. Data includes anything posted in a social media site.

2.

What are the reasons/motivations behind the exponential growth of social media analytics?

The exponential growth of social media analytics stems largely from the similarly exponential growth of social media together with another exponential growth in the capacities of text and Web analytics technologies.

3.

How can you measure the impact of social media analytics?

First, determine what your social media goals are. From here, you can use analysis tools such as descriptive analytics, social network analysis, and advanced (predictive, text examining content in online conversations), and ultimately prescriptive analytics tools.

4.

List and briefly describe the best practices in social media analytics.

Measurement should be for guidance, not for rating or judging. Try to be accurate with sentiment, especially with mixed-polarity statements and terms.

Continuously refine and improve the text analytics techniques. Consider the ripple effect; how many references about the site are retweeted and reblogged. Don’t just focus on sentiments about your brand; also consider the larger social context.

Identify your most powerful influencers. Look closely at the accuracy of your analytics tools. Incorporate your social media analysis into your organizational planning.

5.

Why do you think social media analytics tools are usually offered as a service and not a tool?

Many complex and widely-needed tasks are more often provided as a service on the Web. SMA is no exception to this rule. Plus, the fact that SMA is so tightly linked to Web-based social media (Facebook, Twitter, blogs, etc.) makes it natural to house in an Internet-based architecture, as opposed to a desktop application or local area network.

ANSWERS TO APPLICATION CASE QUESTIONS FOR DISCUSSION

 

Application Case 8.1: Identifying Extremist Groups with Web Link and Content Analysis 1.

How can Web link/ content analysis be used to identify extremist groups?

14 .

Start by gathering a superior-quality collection of relevant extremist and terrorist websites. Then perform hyperlink analysis to find networks of relationships.

Follow this with content analysis to codify the nature of the communications, on keywords related to fundraising, ideology sharing, etc.

2.

What do you think are the challenges and the potential solution to such intelligence gathering activities?

It’s hard to know which links really translate into terrorist relationships. So, finding clusters of links and analyzing the content of the communications helps to more accurately pinpoint terrorist activity. Another challenge will be that terrorists will counteract against the intelligence gathering activity, perhaps through coded conversations. Thus text analytics must be continuously refined to account for coded conversations.

Application Case 8.2: IGN Increases Search Traffic by 1500 Percent 1.

How did IGN dramatically increase search traffic to its Web portals?

The company used search engine optimization to increase its search engine ratings, with the goal to “dominate the search rankings for keywords related to any video games and gaming systems reviewed on the site.”

2.

What were the challenges, the proposed solution, and the obtained results?

IGN’s proprietary in-house content management system generated pages that were not very well optimized when new game reviews and information were added to the site. In addition, there were serious architectural issues with the site, which prevented search engine spiders from thoroughly and consistently crawling the site. The solution was to contract with SEO, Inc. to improve the company’s CMS with customized search engine optimization. The result was dramatic; search engine rankings skyrocketed.

Application Case 8.3: Understanding Why Customers Abandon Shopping Carts Results in $10 Million Sales Increase 1.

How did Lotte.com use analytics to improve sales?

Lotte.com introduced its integrated Web traffic analysis system using the SAS for Customer Experience Analytics solution. This enabled Lotte.com to accurately measure and analyze website visitor numbers (UV), page view (PV) status of site visitors and purchasers, the popularity of each product category and product, clicking preferences for each page, the effectiveness of campaigns, and much more. This information enables Lotte.com to better understand customers and

15 .

their behavior online, and conduct sophisticated, cost-effective targeted marketing.

2.

What were the challenges, the proposed solution, and the obtained results?

With almost 1 million website visitors each day, Lotte.com needed to know how many visitors were making purchases and which channels were bringing the most valuable traffic. SAS for Customer Experience Analytics solution fully transformed the Lotte.com website. As a result, Lotte.com has been able to improve the online experience for its customers as well as generate better returns from its marketing campaigns. This led to a jump in customer loyalty, improved market analysis, a jump in customer satisfaction, and higher sales.

3.

Do you think e-commerce companies are in better position to leverage benefits of analytics? Why? How? To the degree that e-commerce companies integrate analytics into their systems, they can take advantage of these technologies to make improvements in the user experience on their e-commerce systems and thereby increase customer satisfaction.

Application Case 8.4: Allegro Boosts Online Click-Thru Rates by 500 Percent with Web Analysis 1.

How did Allegro significantly improve click-through rates with Web analytics?

Allegro created a new, highly personalized solution integrating predictive analytics and campaign management into a real-time recommendation system.

The company applied Social Network Analysis (SNA) as the analytic methodology underlying its product recommendation system. The SNA process grouped similar products into communities based on their commonalities; then, communities were weighted based on visitor click paths, items placed in shopping carts, and purchases to create predictive attributes. From these, rules were established for optimizing product recommendation placement.

2.

What were the challenges, the proposed solution, and the obtained results?

Allegro’s challenge was how to match the right offer to the right customer while still being able to support the extraordinary amount of data it held. The company had over 75 proprietary websites in 11 European countries, over 15 million products, and generated over 500 million page views per day. In particular Allegro aimed to increase its income and gross merchandise volume from its current network, as measured by click-through rates and conversion rates. The solution was to implement the SNA-based product recommendation system described above. The results were dramatic. User traffic increased by 30%, click-

16 .

through rates increased by 500%, and conversion rates are 40 times what they were previously.

Application Case 8.5: Social Network Analysis Helps Telecommunication Firms 1.

How can social network analysis be used in the telecommunications industry?

It can help telecommunications companies listen to and understand the needs and wants of customers. This enables them to offer communication plans, prices, and features that are tailored to the individual customer. SNA can be applied to call records that already exist in a telecommunication company’s database. This data is analyzed using SNA metrics such as influencers, degree, density, betweenness and centrality.

2.

What do you think are the key challenges, potential solution, and probable results in applying SNA in telecommunications firms?

Because of the widespread use of free Internet tools and techniques (VoIP, video conferencing tools such as Skype, free phone calls within the United States by Google Voice, etc.), the telecommunication industry is going through a tough time. In order to stay viable and competitive, companies need to make the right decisions and utilize their limited resources optimally. Market pressures force telecommunication companies to be more innovative. Using SNA, companies can manage churn, improve cross-sell and technology transfer, manage viral campaigns, better identify customers, and gain competitor insight.

Application Case 8.6: Measuring the Impact of Social Media at Lollapalooza 1.

How did C3 Presents use social media analytics to improve its business?

The company contracted with Cardinal Path to architect and implement a solution based on an existing Google Analytics implementation. A combination of customized event tracking, campaign tagging, custom variables, and a complex implementation and configuration was deployed to include the tracking of each social media outlet on the site.

2.

What were the challenges, the proposed solution, and the obtained results?

C3 Presents was making extensive use of social media to publicize the Lollapalooza event, and needed to know if it was working. The company wanted to answer several questions. Which fans are using social media and sharing content? What social media is being used the most, and how? Are visitors that interact with social media more likely to buy a ticket? Is social media driving more traffic to the site? Is that traffic buying tickets? As a result of implementing the systems with Cardinal Path, the company was able to gain insights that helped C3 quantify the return on their social media investment.

17 .

Application Case 8.7: eHarmony Uses Social Media to Help Take the Mystery Out of Online Dating 1.

How did eHarmony use social media to enhance online dating? eHarmony started using Salesforce Marketing Cloud to listen to conversations around the brand and around keywords like “bad date” or “first date.” The company also took to Facebook and Twitter to connect with members, share success stories—including engagement and wedding videos—and answer questions from those looking for dating advice.

2.

What were the challenges, the proposed solution, and the obtained results?

The team at eHarmony saw social media as a chance to dispel any myths and preconceived notions about online dating and, more importantly, have some fun with their social media presence. By using social media to help educate and create awareness around the benefits of online dating, eHarmony has built a strong and loyal community. The social media team now has eight staff members working to respond to social interactions and posts, helping them reach out to clients and respond to hundreds of posts a week.

ANSWERS TO END OF CHAPTER QUESTIONS FOR DISCUSSION 1.

  

Explain the relationship among data mining, text mining, and Web mining.

Technically speaking, data mining is a process that uses statistical, mathematical, and artificial intelligence techniques to extract and identify useful information and subsequent knowledge (or patterns) from large sets of data. Data mining is the general concept. Text mining is a specific application of data mining: applying it to unstructured text files. Web mining is a specific application of data mining: applying it to information on and about the Web (content, structure, and usage).

2.

What should an organization consider before making a decision to purchase Web mining software?

Before making a decision to purchase any mining software organizations should consider the standard criteria to use when investing in any major software: cost/benefit analysis, people with the expertise to use the software and perform the analyses, availability of data/information, and a business need for the software and capabilities.

3.

Discuss the differences and commonalities between text mining and Web mining.

18 .

Text mining is a specific application of data mining: applying it to unstructured text files. Web mining is a specific application of data mining: applying it to information on and about the Web (content, structure, and usage).

4.

In your own words, define Web mining and discuss its importance.

Students’ answers will vary.

5.

What are the three main areas of Web mining? Discuss the differences and commonalities among these three areas.

Web mining consists of three areas: Web content mining, Web structure mining, and Web usage mining.

Web content mining refers to the automatic extraction of useful information from Web pages. It may be used to enhance search results produced by search engines.

Web structure mining refers to generating interesting information from the links included in Web pages. Web structure mining can also be used to identify the members of a specific community and perhaps even the roles of the members in the community.

Web usage mining refers to developing useful information through analysis of Web server logs, user profiles, and transaction information.

6.

What is Web content mining? How does it differ from text mining? Discuss and justify your answers with concrete examples.

Web content mining refers to the automatic extraction of useful information from Web pages. It may be used to enhance search results produced by search engines.

Students’ examples will differ.

7.

What is Web structure mining? What are authoritative pages and hubs? How do they relate to Web structure mining?

Web structure mining refers to generating interesting information from the links included in Web pages. Web structure mining can also be used to identify the members of a specific community and perhaps even the roles of the members in the community.

A search on the Web to obtain information on a specific topic usually returns a few relevant, high-quality Web pages and a larger number of unusable Web pages. Use of an index based on authoritative pages (or some measure of it) will improve the search results and ranking of relevant pages.

19 .

The structure of Web hyperlinks has led to another important category of Web pages called a hub. A hub is one or more Web pages that provide a collection of links to authoritative pages.

HITS is a link analysis algorithm that rates Web pages using the hyperlink information contained within them. In the context of Web search, the HITS algorithm collects a base document set for a specific query.

8.

Discuss the expected benefits of Web structure mining. Provide examples from real-world applications that you are familiar with.

Web structure mining is expected to create value from the immense volume of data collected by Web servers. Students’ examples will vary.

9.

What is Web usage mining? Draw a picture of the Web usage mining process and explain/discuss the major steps in the process.

Web usage mining refers to developing useful information through analysis of Web server logs, user profiles, and transaction information.

See Figure 7.9 on p. 321.

10.

Provide two exemplary business applications of Web usage mining; discuss their usage and business value.

Students’ answers will vary. Examples are: 1. Determine the lifetime value of clients.

2. Design cross-marketing strategies across products.

3. Evaluate promotional campaigns.

4. Target electronic ads and coupons at user groups based on user access patterns.

5. Predict user behavior based on previously learned rules and users’ profiles.

6. Present dynamic information to users based on their interests and profiles.

11.

What is a search engine? Why are they important for businesses?

A search engine is a software program that searches for documents (Internet sites or files) based on the keywords (individual words, multi-word terms, or a complete sentence) that users have provided that have to do with the subject of their inquiry. This is the most prominent type of information retrieval system for finding relevant content on the Web. Search engines have become the centerpiece of most Internet-based transactions and other activities. Because people use them extensively to learn about products and services, it is very important for companies to have prominent visibility on the Web; hence the major effort of companies to enhance their search engine optimization (SEO).

20 .

12.

What is search engine optimization? Who benefits from it? How? Search engine optimization (SEO) is the intentional activity of affecting the visibility of an e-commerce site or a website in a search engine’s natural (unpaid or organic) search results. It involves editing a page’s content, HTML, metadata, and associated coding to both increase its relevance to specific keywords and to remove barriers to the indexing activities of search engines. In addition, SEO efforts include promoting a site to increase its number of inbound links. SEO primarily benefits companies with e-commerce sites by making their pages appear toward the top of search engine lists when users query.

13.

What is Web analytics? What are the metrics used in Web analytics?

Web analytics, also called Web usage mining, can be considered a part of Web mining, and aims to describe what has happened on the website (employing a predefined, metrics-driven descriptive analytics methodology). There are four main categories of Web analytic metrics: • • • •

Website usability: How were they using my website? These involve page views, time on site, downloads, click map, and click paths.

Traffic sources: Where did they come from? These include referral websites, search engines, direct, offline campaigns, and online campaigns.

Visitor profiles: What do my visitors look like? These include keywords, content groupings, geography, time of day, and landing page profiles.

Conversion statistics: What does all this mean for the business? Metrics include new visitors, returning visitors, leads, sales/conversions, and abandonments.

These metrics are important because they provide access to a lot of valuable marketing data, which can be leveraged for better insights to grow your business and better document your ROI. The insight and intelligence gained from Web analytics can be used to effectively manage the marketing efforts of an organization and its various products or services.

14.

Define social analytics, social network, and social network analysis. What are the relationships among them?

Social analytics involves monitoring, analyzing, measuring, and interpreting digital interactions and relationships of people, topics, ideas, and content. It involves mining the textual content created in social media (e.g., sentiment analysis, natural language processing) and analyzing socially established networks (e.g., influencer identification, profiling, prediction). A social network is a social structure composed of individuals/people (or groups of individuals or organizations) linked to one another with some type of connections/relationships.

Social network analysis (SNA) is the systematic examination of social networks, and is an interdisciplinary field that emerged from social psychology, sociology, 21 .

statistics, and graph (network) theory. Social analytics therefore combines text analysis for content and sentiment in online communications with social network analysis to identify and analyze relationships between individuals in a community.

15.

What is social media analytics? How is it done? Who does it? What comes out of it?

Social media analytics refers to the systematic and scientific ways to consume the vast amount of content created by Web-based social media outlets, tools, and techniques for the betterment of an organization’s competitiveness. It is done using many analytic methods, including text mining, sentiment analysis, and social network analysis. Companies use it to get a better understanding of their customer base, and can gain financial and competitive advantages from doing so.

Governments use it to track potential terrorist threats, which can lead to enhanced national security. Social scientists use it to get a better understanding of how communities and societies work, which can provide guidance on how to best manage these societies.

ANSWERS TO END OF CHAPTER APPLICATION CASE QUESTIONS 1.

 

Describe challenges that APUS was facing. Discuss the ramifications of such challenges.

One big challenge is keeping and improving retention in an online university. But educators lacked the hard data to prove conclusively what really drives retention.

Colleges and universities have struggled to understand how to lower dropout rates and keep students on track all the way to graduation. APUS faced this challenge in a new online environment, which further complicates the retention dilemma.

2.

What types of data did APUS tap into? What do you think are the main obstacles one would have to overcome when using data that comes from different domains and sources?

APUS tapped into demographic data, registration data, and course level data.

They used online activity as well as end-of-course survey data. (The remainder of this answer may vary according to student.) One obstacle with bringing in data from different domains is the complexity of finding meaningful patterns in the data. This could require data extraction, cleaning, and transforming the source data.

3.

What solutions did they pursue? What tools and technologies did they use?

22 .

They used predictive analytics to zero in on those factors that most influence a student’s decision to stay in school or drop out. In addition, they built dashboards to put the predictive analytics into the hands of deans and other administrators.

4.

What were the results? Do you think these results are also applicable to other educational institutions? Why? Why not?

Some of the findings generated by its predictive models actually came as a surprise to APUS. For example, it had long been assumed that gender and ethnicity were good predictors of attrition, but the models proved otherwise.

Educators also assumed that a preparatory course called “College 100: Foundations of Online Learning” was a major driver of retention, but an in-depth analysis using Modeler came to a different conclusion. Also, the results showed that a sense of community was an important retention factor. So, APUS is putting more effort into promoting interaction between students via social media and online collaboration tools.

Predictive analytics are clearly applicable at other universities. But some metrics may not apply. For example, click-stream analysis won’t apply for universities with traditional face-to-face classroom delivery.

5.

What additional analysis are they planning on conducting? Can you think of other data analyses that they could apply and benefit from?

APUS also plans to delve deeper into the surveys by mining the open-ended text responses that are part of each questionnaire (IBM SPSS Text Analytics for Surveys will help with that initiative). All of these data-driven initiatives aim to increase student learning, enhance the student’s experience, and build an environment that encourages retention at APUS.

23 .

Chapter 9: Model-Based Decision Making: Optimization and Multi-Criteria Systems Learning Objectives for Chapter 9 1. Understand the basic concepts of analytical decision modeling 2. Describe how prescriptive models interact with data and the user 3. Understand some different, well-known model classes 4. Understand how to structure decision making with a few alternatives 5. Describe how spreadsheets can be used for analytical modeling and solution 6. Explain the basic concepts of optimization and when to use them 7. Describe how to structure a linear programming model 8. Describe how to handle multiple goals 9. Explain what is meant by sensitivity analysis, what-if analysis, and goal seeking 10. Describe the key issues of multi-criteria decision making

CHAPTER OVERVIEW In this chapter, we describe selected techniques employed in prescriptive analytics. We present this material with a note of caution: Modeling can be a very difficult topic and is as much an art as a science. The purpose of this chapter is not necessarily for you to master the topics of modeling and analysis. Rather, the material is geared toward gaining familiarity with the important concepts as they relate to DSS and their use in decision making. It is important to recognize that the modeling we discuss here is only cursorily related to the concepts of data modeling. You should not confuse the two. We walk through some basic concepts and definitions of modeling before introducing the influence diagrams, which can aid a decision maker in sketching a model of a situation and even solving it. We next introduce the idea of modeling directly in spreadsheets. We then discuss the structure and application of some successful time-proven models and methodologies: optimization, decision analysis, decision trees, and analytic hierarchy process. This chapter includes the following sections: CHAPTER OUTLINE 9.1

OPENING VIGNETTE: MIDWEST ISO SAVES BILLIONS BY BETTER PLANNING OF POWER PLANT OPERATIONS AND CAPACITY PLANNING  Questions for the Opening Vignette 1 .

A.

WHAT WE CAN LEARN FROM THIS VIGNETTE

9.2

DECISION SUPPORT SYSTEMS MODELING ⧫ Application Case 9.1: Optimal Transport for ExxonMobil Downstream Through a DSS A. CURRENT MODELING ISSUES 1.

Identification of the Problem and Environmental Analysis 2.

Variable Identification 3.

Forecasting (Predictive Analytics) ⧫ Application Case 9.2: Forecasting/Predictive Analytics Proves to Be a Good Gamble for Harrah’s Cherokee Casino and Hotel 4.

Model Categories 5.

Model Management 6.

Knowledge-Based Modeling 7.

Current Trends in Modeling  Section 9.2 Review Questions

9.3

STRUCTURE OF MATHEMATICAL MODELS FOR DECISION SUPPORT A. THE COMPONENTS OF DECISION SUPPORT MATHEMATICAL MODELS 1.

Result (Outcome) Variables 2.

Decision Variables 3.

Uncontrollable Variables, or Parameters 4.

Intermediate Result Variables B. THE STRUCTURE OF MATHEMATICAL MODELS  Section 9.3 Review Questions

9.4

CERTAINTY, UNCERTAINTY, AND RISK A. DECISION MAKING UNDER CERTAINTY B. DECISION MAKING UNDER UNCERTAINTY C. DECISION MAKING UNDER RISK (RISK ANALYSIS) ⧫ Application Case 9.3: American Airlines Uses Should-Cost Modeling to Assess the Uncertainty of Bids for Shipment Routes  Section 9.4 Review Questions

9.5

DECISION MODELING WITH SPREADSHEETS ⧫ Application Case 9.4: Showcase Scheduling at Fred Astaire East Side Dance Studio  Section 9.5 Review Questions

9.6

MATHEMATICAL PROGRAMMING OPTIMIZATION ⧫ Application Case 9.5: Spreadsheet Model Helps Assign Medical Residents 2 .

A. B. C. D.

MATHEMATICAL PROGRAMMING LINEAR PROGRAMMING MODELING IN LP: AN EXAMPLE ⧫ Technology Insights 9.1: Linear Programming IMPLEMENTATION  Section 9.6 Review Questions

9.7

MULTIPLE GOALS, SENSITIVITY ANALYSIS, WHAT-IF ANALYSIS, AND GOAL SEEKING A. MULTIPLE GOALS B. SENSITIVITY ANALYSIS 1.

Automatic Sensitivity Analysis 2.

Trial-and-Error Sensitivity Analysis C. WHAT-IF ANALYSIS D. GOAL SEEKING 1.

Computing a Break-Even Point by Using Goal Seeking  Section 9.7 Review Questions

9.8

DECISION ANALYSIS WITH DECISION TABLES AND DECISION TREES A. DECISION TABLES 1.

Treating Uncertainty 2.

Treating Risk B. DECISION TREES  Section 9.8 Review Questions

9.9

MULTICRITERIA DECISION MAKING WITH PAIRWISE COMPARISIONS A. THE ANALYTIC HIERARCHY PROCESS ⧫ Application Case 9.6: U.S. HUD Saves the House by Using AHP for Selecting IT Projects B. TUTORIAL ON APPLYING ANALYTIC HIERARCHY PROCESS USING WEB-HIPRE  Section 9.9 Review Questions

Chapter Highlights Key Terms Questions for Discussion Exercises Teradata UNIVERSITY NETWORK (TUN) and Other Hands-on Exercises ⧫ End of Chapter Application Case: Pre-Positioning of Emergency Items for CARE International  Questions for the Case References

3 .

TEACHING TIPS/ADDITIONAL INFORMATION

       

The purpose of this chapter is to teach the concepts of prescriptive mathematical modeling so students will begin to understand how models can be used to evaluate decision alternatives and find optimal solutions. It does so by going through a wide variety of model types, from the familiar (most DSS students will have encountered at least one financial model on a spreadsheet) through some that may have been seen in other courses (linear programming is often taught in operations management courses) to many that students will not have seen before. This can be a daunting amount of information to cover in the time you can allocate to a single chapter.

For this reason, you may want to select parts of this chapter to go into more deeply than others. Your choice of topics to stress will depend on your own perspective as to what is important, what isn’t. It’s still a good idea to at least touch on all the sections, though. Each one discusses a method that has been used in some DSS and that students may therefore encounter on the job. Your professional experience may tell you that some are less important than others, but that doesn’t mean any are totally unimportant.

Modeling is an applied discipline, best learned by reinforcement, through exercises and problems. Encourage the students to understand it through trial-and-error.

The chapter mentions several software tools. You may wish to download, install, and use some of them. You can also work through a simple what-if analysis by hand, or do a goal seek problem using Excel’s Solver and what-if analysis features in class. Students should do practical work using some of the available tools for simple but realistic and relevant business problems. Many of these can be found by doing a Google search. Because so much of the material in this chapter can be done with Excel, you should take advantage of this to give plenty of exercises to your students.

Another good in-class exercise would be to give a description of a problem and have groups of students try come up with a conceptual decision model by identifying the decision, uncontrollable results, and intermediate results variable of the system. Students can go further by postulating potential mathematical functions of their model. Even if this is not ultimately implemented in Excel or some other software, it will give students experience in coming up with a conceptual model design.

ANSWERS TO END OF SECTION REVIEW QUESTIONS

     

Section 9.1 Review Questions 1.

In what ways were the individual companies in Midwest ISO better off being part of MISO as opposed to operating independently?

Individually, the companies had to make extra investments to manage risk. Their mode of operation resulted in inefficient use of transmission lines. MISO also ensures that all members of the organization have equal access to high-voltage power lines. Individually, the companies had to make extra investments to 4 .

manage risk. As part of MISO, risk is managed collectively for the members by keeping a contingency reserve.

2.

The dispatch problem was solved with a linear programming method. Explain the need of such method in light of the problem discussed in the case.

In terms of MISO’s DSS, the dispatch problem was to decide how much output should be produced by each power plant, and determine the price of energy based on the location of the power plant. The linear programming method fits well with this type of problem. Constraints include limits on frequency deviation (60 hertz) and power-loss time (10 minutes). The objective function involves expending the least possible total power to meet the electrical needs, and the decision variables are the power plant’s output.

3.

What were the two main optimization algorithms used? Briefly explain the use of each algorithm.

For the commitment problem, which determined when power plants should be turned on or off, a Lagrangian relaxation method was used. For the dispatch problem, which determined the amount of output from a plant, a linear programming approach was used. In addition, a mixed integer programming model was used for the ancillary service market commitment problem.

Section 9.2 Review Questions 1.

List three lessons learned from modeling.

This question refers to lessons found in the examples in this section, not to lessons learned from modeling in general. Many examples could be cited, among them these three: PillowTex learned how a simulation model can lead to lean manufacturing and save millions of dollars. IBM was able to use a combination of weather and sensor data to build a river system simulation application that could simulate thousands of river branches at a time for flood prediction and irrigation management. TurboRouter, a DSS for ship routing and scheduling, claims that over the course of just a three-week period, a company used this model to better utilize its fleet, generating additional profit of $1–2 million.

2.

List and describe the major issues in modeling.

Major modeling issues include: • problem identification and environmental analysis: scanning the environment to figure out what problems exist and can be solved via a model • variable identification: identifying the critical factors in a model and their relationships 5 .

• • • •

3.

forecasting: predicting the future model categories: selecting the right type of model for the problem or subproblem model management: coordinating a firm’s models and their use knowledge-based modeling: how to take advantage of human knowledge in modeling

What are the major types of models used in DSS? The major types from Table 9.1 follow.

• Optimization with few alternatives • Optimization via an algorithm • Optimization via an analytical formula • Simulation • Heuristics (“rules of thumb”) • Predictive models • Other models

4.

Why are models not used in industry as frequently as they should or could be?

There are several reasons. The most important is that useful models are often complex, requiring major investments of time and money as well as scarce, expensive expertise.

5.

What are the current trends in modeling?

One recent trend is the development of model libraries and solution technique libraries. Another is an increasing emphasis toward developing and using Web tools and software to access and even run software to perform modeling, optimization, and simulation. There is a continuing trend toward making analytics models completely transparent to the decision maker. There is also a trend to build a model of a model to help in its analysis. Influence diagram help in this respect.

Section 9.3 Review Questions 1.

What is a decision variable?

A decision variable is a data element controlled by the decision maker, whose possible values describe alternative courses of action.

6 .

2.

List and briefly discuss the three major components of linear programming.

Of the four components of any decision support mathematical model, linear programming uses result (outcome) variables, decision variables, and uncontrollable variables (parameters). Linear programming models do not use the fourth component, intermediate result variables.

There are other possible answers that do not tie in to the concepts of this section.

For example, one could say that the three major components of a linear programming problem are its objective function, its decision variables, and its constraints. You might wish to connect this question to Section 4.8.

3.

Explain the role of intermediate result variables.

An intermediate result variable reflects an intermediate result of the mathematical model. It is a result variable, and therefore is influenced by decision and uncontrollable variables in the mathematical model. An intermediate variable also influences other result variables in the mathematical model. For example, in a human resources system, employee salaries (decision variable) affect employee satisfaction (intermediate result variable), which further affects productivity (result variable).

Section 9.4 Review Questions 1.

Define what it means to perform decision making under assumed certainty, risk, and uncertainty.

• • •

2.

Decision making under assumed certainty: the values of all variables affecting the decision, including future values, are known or can be assumed to be known.

Decision making under risk: the exact value of decision variables is not known, but their statistical probability distributions are known (or can be assumed to be).

Decision making under uncertainty: even these distributions are not known.

How can decision-making problems under assumed certainty be handled?

Decision-making problems under assumed certainty can be handled by mathematical models that will yield an optimum solution.

3.

How can decision-making problems under assumed uncertainty be handled?

Decision-making problems under uncertainty can be handled, first, by trying to get more information. If this is not possible or if the uncertainty remains after this has been done, it is necessary to evaluate the outcomes of many possible values of 7 .

the decision variables. The choice then depends on the decision maker’s attitude toward risk. For example, one solution may have a higher expected value than another, but also a higher likelihood of a negative outcome. A manager with an inclination to gamble might choose that solution, whereas a more conservative and risk-averse manager would choose a solution that offers a lower probable return but also minimizes the likelihood of loss.

4.

How can decision-making problems under assumed risk be handled?

Decision-making problems under assumed risk can be handled by taking the probability distributions of the decision variables and working through the model with those distributions. The result will be a distribution of outcomes.

Note that the mean of the distribution of outcomes is often not the outcome that would be obtained under assumed certainty if each variable had its mean value.

This is because, in a complex model, decision variables interact in complex ways and effect is not always immediately obvious.

Another approach to problems under assumed risk is to select a random value from each decision variable’s probability distribution and to run the model, under assumed certainty, with those values. This is repeated until a statistical distribution of results is obtained. This approach is used when a model is too complex to work through with statistical distributions directly.

Section 9.5 Review Questions 1.

What is a spreadsheet?

This section does not define the concept, relying on student’s experience to tell them what a spreadsheet is. Given the audience of this book, that should normally be appropriate. Instructors who assign this question with no further explanation should expect some students to search for definitions (finding one or more like the above two) whereas others come up with their own and still others use descriptive phrases from the text. If a specific approach is wanted, that should be stated in advance.

2.

What is a spreadsheet add-in? How can add-ins help in DSS creation and use?

Spreadsheet add-ins are small programs designed to extend the capabilities of a spreadsheet package. They help in DSS creation and use because many add-ins are designed specifically for that purpose, often by structuring and solving specific types of models.

8 .

3.

Explain why a spreadsheet is so conducive to the development of DSS. A spreadsheet is conducive to DSS development because it provides an easily understood metaphor for the computation and typically incorporates many powerful modeling functions.

Section 9.6 Review Questions 1.

List and explain the assumptions involved in LP. The LP approach involves both economic and technical assumptions.

Economic assumptions are: • Returns from different allocations can be measured by a common unit (e.g., dollars, utility).

• The return from any allocation is independent of other allocations.

• The total return is the sum of the returns yielded by the different activities.

• All data are known with certainty.

• Resources are to be used in the most economical manner.

Technical assumptions are: • • •

The objective function (that is to be maximized) is a linear combination of the outputs.

Each output (decision variable) uses a linear combination of the inputs.

Constraints on inputs are linear (or constant, which is a special case of linear) inequalities.

The requested explanations should show that the student understands each of the concepts, and is not simply parroting items from a list.

2.

List and explain the characteristics of LP. Linear programming is an optimization method used to solve problems in which the objective function and the constraints are all linear. It rests on the assumptions listed in the previous answer. LP problems can be solved by a wide variety of software for all popular computers.

3.

Describe an allocation problem.

An allocation problem is a linear programming problem in which limited resources can be allocated among several possible uses, each of which yields a known return per unit and are subject to constraints.

4.

Define the product-mix problem.

The product-mix problem is a linear programming problem in which a variety of different products are made from common resources. Each product requires a

9 .

known resource mix and has a known profitability. Some resources are limited.

Total profitability is to be maximized.

A product-mix problem can be viewed as an allocation problem. The difference is that a person formulating the problem as a product-mix problem is typically interested in the quantities of each product to be produced, while a person formulating the same problem as an allocation problem is usually interested in the quantities of each resource to be used. The solutions are identical, and each approach can produce both answers.

5.

Define the blending problem.

The blending problem is a linear programming problem in which resources can be used in different ways to create a desired end product. The ways in which the resources combine to create the characteristics of the end product are known.

Total cost of the end product is to be minimized.

6.

List several common optimization models.

Several common optimization models are listed at the end of the section. These include the assignment problem, dynamic programming, goal programming, investment, linear and integer programming, network models for planning and scheduling, nonlinear programming, replacement (capital budgeting), simple inventory models (e.g., economic order quantity) and transportation (minimizing cost of shipments). A motivated student will be able to find others through a Web search.

Section 9.7 Review Questions 1.

List some difficulties that may arise when analyzing multiple goals.

• • • • • •

It is usually difficult to obtain an explicit statement of the organization’s goals.

The importance of specific goals may change over time or in different situations.

Goals and subgoals are viewed and weighted differently by different people and in different parts of the organization.

Goals change in response to changes in the organization and its environment.

The relationship between alternatives and their role in determining goals may be difficult to quantify.

Participants assess the importance (priorities) of the various goals differently.

10 .

2.

List the reasons for performing sensitivity analysis.

Sensitivity analysis attempts to assess the impact of a change in input data or parameters on the result variable(s). Reasons for using it listed in this section include: • Revising models to eliminate too-large sensitivities • Adding details about sensitive variables or scenarios • Obtaining better estimates of sensitive external variables • Altering a real-world system to reduce actual sensitivities • Accepting and using the sensitive (and hence vulnerable) real world, leading to the continuous and close monitoring of actual results Another reason for using sensitivity analysis is to understand the sensitivity of different decision choices to variations in external conditions. One alternative may promise excellent results under nominal conditions but performs poorly if conditions vary from the nominal. Another may not turn out quite as well if all goes according to plan but will perform better if conditions vary from it. For example, one possible supplier may offer low prices for deliveries according to a pre-planned schedule but impose large penalties if the schedule is changed; another might have a higher base price but lower penalties for changing the delivery schedule.

3.

Explain why a manager might perform what-if analysis.

A manager might perform what-if analysis to find out what will happen if a particular action is taken. A manager might also perform what-if analysis to try out several alternatives, choosing the one that works out best.

4.

Explain why a manager might use goal seeking.

A manager might use goal seeking to find values of decision variables that enable him or her to meet a predetermined criterion. For example, a manager may need to find an advertising budget that will reach X households at a cost of not over $Y.

Section 9.8 Review Questions 1.

What is a decision table?

A decision table is a way to organize information in a systematic way to prepare it for analysis. Typically, decisions are shown along one axis, states of nature on the other. The range of outcomes that can result from any decision, given the possible states of nature, can then be easily seen in one column (or row).

11 .

2.

What is a decision tree?

A decision tree is an alternative representation of a decision situation, in which choices and states of nature are shown as alternating nodes along the branches of a tree. The range of outcomes that can result from any decision, given the possible states of nature, can then be easily seen by following all branches from that decision to their ends.

3.

How can a decision tree be used in decision making?

By showing the decision maker the possible outcomes that could result from a given choice, the tree gives the decision maker information by which to compare choices.

4.

Describe what it means to have multiple goals.

Having multiple goals means that a decision maker hopes to obtain the best possible combination of several factors, all of which depend on the decision to be made. For example, a student may want to find an instructor who is entertaining, has a good reputation for teaching, grades easily, assigns little homework, and whose section meets at convenient times. It will be difficult, if not impossible, to achieve all these at the same time. Multiple goals require willingness to compromise.

(You may want to defer this question until students have read Section 9.9, or to refer them forward to it at this point.)

Section 9.9 Review Questions 1.

What is analytic hierarchy process?

The analytic hierarchy process (AHP) is an excellent modeling structure for representing multicriteria (multiple goals, multiple objectives) problems—with sets of criteria and alternatives (choices)—commonly found in business environments.

2.

What steps are needed in applying AHP? The decision maker uses AHP to decompose a decision-making problem into relevant criteria and alternatives. The AHP separates the analysis of the criteria from the alternatives, which helps the decision maker to focus on small, manageable portions of the problem.

12 .

3.

What software can be used for AHP? Expert Choice (expertchoice.com; a demo is available directly on its Web site) is an excellent commercial implementation of AHP. Another product is Web-HIPRE (hipre.aalto.fi), an adaptation of AHP and several other weighting schemes, which enables a decision maker to create a decision model, enter pairwise preferences, and analyze the optimal choice.

ANSWERS TO APPLICATION CASE QUESTIONS FOR DISCUSSION

 

Application Case 9.1: Optimal Transport for ExxonMobil Downstream Through a DSS 1.

List three ways in which manual scheduling of ships could result in more operational cost as compared to the tool developed.

Without the optimization model, there is likely to be greater shipping cost and more demurrage expenses. This is because the manual process is unable to maximize the effective utilization of ships and design the best routing schedules.

In addition, the manual process is cumbersome and time consuming, involving several parties.

2.

In what other ways can ExxonMobil leverage the decision support tool developed to expand and optimize their other business operations?

(Answers will vary.) Since this DSS is geared to shipping transportation concerns, ExxonMobil could potentially market it as a product for other shipping companies. Also, the methodology used and lessons learned from this project could be applied to other areas at ExxonMobil, such as oil drilling.

3.

What are some strategic decisions that could be made by decision makers using the tool developed?

Ship selection and route design are two immediate benefits. Also, the DSS could help ExxonMobil balance their own managed vessels and third-party vessels. This tool can also affect investment decisions in their shipping infrastructure as well as in future DSS projects.

Application Case 9.2: Forecasting/Predictive Analytics Proves to Be a Good Gamble for Harrah’s Cherokee Casino and Hotel (This application case has no discussion questions.)

13 .

Application Case 9.3: American Airlines Uses Should-Cost Modeling to Assess the Uncertainty of Bids for Shipment Routes 1.

Besides reducing the risk of overpaying or underpaying suppliers, what are some other benefits AA would derive from its “should be” model?

As a model for evaluating bids, the DSS could be used to streamline and standardize AA’s RFQ process. It also helps to give users a better understanding and quicker analysis of different bids, and potentially leads to better relations with suppliers.

2.

Can you think of other domains besides air transportation where such a model could be used?

Since this is a model for determining fair payment to suppliers, it can apply to most supplier relationships involving product inventory. Any industry involving a supply chain could benefit from models like AA’s should-cost model.

3.

Discuss other possible methods with which AA could have solved its bid overpayment and underpayment problem.

AA could use other BI-related approaches besides decision analysis via DPL. For example, this problem may have been formulated using a linear programming approach, multi-criteria decision model, or some other optimization approach.

Even text analytics could be useful; by searching past RFQ bids from suppliers, AA could get a better idea of the bid process.

Application Case 9.4: Showcase Scheduling at Fred Astaire East Side Dance Studio (This application case has no discussion questions.)

Application Case 9.5: Spreadsheet Model Helps Assign Medical Residents (This application case has no discussion questions.)

Application Case 9.6: U.S. HUD Saves the House by Using AHP for Selecting IT Projects (This application case has no discussion questions.)

14 .

ANSWERS TO END OF CHAPTER QUESTIONS FOR DISCUSSION 1.

  

What is the relationship between environmental analysis and problem identification?

Environmental analysis is the monitoring, scanning, and interpretation of collected information. Its purpose is to detect problem situations, after which they can be identified more precisely.

2.

Explain the differences between static and dynamic models. How can one evolve into the other?

A static model describes relationships among parts of a system at a point in time.

A dynamic model describes relationships among parts of a system as it moves through time, with its state at one instant influencing (together with its inputs) its state at the next.

The relationships that hold in a static model at a point in time must also hold in a dynamic model of the same system at every time. To modify a static model into a dynamic model, it is necessary to add relationships between time periods so that the static model in one time period depends, in an appropriate way, on the values of system variables in the prior time period rather than standing on its own. This may increase the model’s complexity dramatically and make it harder, if not impossible, to solve.

3.

What is the difference between an optimistic approach and a pessimistic approach to decision making under assumed uncertainty?

An optimist makes the choice that has the best best-case outcome, even if it gives worse results under other conditions. The pessimist makes the choice as the best worst-case outcome, even if it gives worse results under other conditions.

For example: Suppose one choice has equally probable results of +$80 and +$30. Another has equally probable results of +$120 and –$10. (The expected value of both is +$55, so a rational decision maker with a linear utility function would call it a tie.) An optimist will choose the second because, in the best case, it will yield +$120: better than the best-case result of +$80 for the other option. A pessimist will choose the first because, at worst, it will yield +$30: better than the worst-case result of –$10 for the other choice.

4.

Explain why solving problems under uncertainty sometimes involves assuming that the problem is to be solved under conditions of risk.

Solving problems under uncertainty means the decision maker has no idea of likelihood of one outcome or another. However, decision makers usually have some idea of the relative probability of certain influencing factors. By assuming approximate probabilities (changing the situation to one of risk even if the probabilities are not really known), the decision maker can predict the results to be expected from a course of action. Sensitivity analysis can then be used, if 15 .

desired, to determine how much the prediction depends on the accuracy of the estimates.

5.

Excel is probably the most popular spreadsheet software for PCs. Why? What can we do with this package that makes it so attractive for modeling efforts?

The reasons why Excel is the most popular spreadsheet program for PCs have more to do with industry dynamics and Lotus’s slowness in porting 1-2-3 to Windows than with its technical merits. Although undoubtedly a good program, it achieved dominance in the mid-1990s when it was not markedly superior to competitors. Its superiority over most competing products today is due to the limited development resources available for products with far smaller market shares and came about only after its dominance was already well established. A full discussion of how and why Excel became the most popular PC spreadsheet program is beyond the scope of a DSS book or course.

Once a program such as Excel achieves a dominant market share, factors such as size and inertia of its user community, the infrastructure that builds up around it (add-ins, books, etc.) and (in its case) bundling into Microsoft Office combine to help it maintain that position.

Excel’s attractiveness for modeling efforts is due to five factors: the intuitiveness of the spreadsheet metaphor, its ubiquity (which makes Excel models easy to share), user familiarity with it, its flexibility, and its power. You can do many things with Excel. Its ease of use facilitates creating ad-hoc DSS. Macros allow complex procedures to be automated, with VBA available when macros fall short.

Built-in goal-seeking and scenario analysis capabilities enable a model, once developed, to become part of a larger problem-solving process.

In short: (nearly) everyone has Excel, and Excel has what (nearly) everyone needs.

6.

Explain how decision trees work. How can a complex problem be solved by using a decision tree?

A decision tree is an alternative representation of a decision situation, in which choices and states of nature are shown as alternating nodes along the branches of a tree. The range of outcomes that can result from any decision, given the possible states of nature, can then be easily seen by following all branches from that decision to their ends. By showing the decision maker the possible outcomes that could result from a given choice, the tree gives the decision maker information by which to compare choices. (This question is essentially the same as Review Questions 2 and 3 of Section 9.8, combined.)

16 .

7.

Explain how LP can solve allocation problems.

LP solves allocation problems by using mathematical relationships (inputs required per unit of output) under known constraints (resource availability, etc.) to find the allocation of finite resources that produces the greatest value (highest value of the objective function).

8.

What are the advantages of using a spreadsheet package to create and solve LP models? What are the disadvantages?

Advantages: The spreadsheet framework is well understood by many modern managers, linear programming models fit well into the tabular format, solutions can be stored directly in the spreadsheet, and it is possible to explain linear programming using what-if analysis directly in a spreadsheet for small problems.

Spreadsheet models are not bound by the theoretical limitations of the linear programming approach, so they can (for example) work with nonlinear constraints.

Disadvantages: Large problems are difficult to create, debug, modify, and interpret. The fact that spreadsheets are a general-purpose tool, not a specialized one, makes them slower than LP software, and special structures, such as for network optimization problems, cannot be exploited for efficient solution.

9.

What are the advantages of using an LP package to create and solve LP models?

What are the disadvantages?

Advantages: The software was designed to do that one thing, it can be quite fast, the best packages can interface with large-scale databases, and some systems even feature modeling languages to enter problems.

Disadvantages: The output can be cryptic, it may require an analyst to operate and even develop the model, the software can be cumbersome to operate, and limitations of a particular package can make it difficult to solve large problems.

10.

What is the difference between decision analysis with a single goal and decision analysis with multiple goals (i.e., criteria)? Explain in detail the difficulties that may arise when analyzing multiple goals.

In a single goal we compare only one result (e.g., profit). In multiple goals we compare several results. Since in the latter case we rarely find an alternative that is superior along all the criteria, it is necessary to prioritize goals, weight them, or combine them in some other way in order to make a choice.

11.

Explain how multiple goals can arise in practice.

It is rare to have only one objective in making a business or personal decision. In choosing a flight, for example, people consider schedule, price, airline preference, 17 .

and more. When multiple stakeholders are involved, which is common in business decisions, there are even more goals to compromise among. (Consider, for example, three business travelers trying to pick a single flight. They might agree on cost, but have different schedule preferences and want frequent flyer credit on different airlines.)

12.

Compare and contrast what-if analysis and goal seeking.

What-if analysis begins with conditions and determines their result. Goal seeking begins with a desired result and determines the conditions that will produce it.

That often calls for using what-if analysis in an iterative process: assume some conditions, examine the result, on that basis choose other conditions, examine their result, and continue (using an intelligent search process that considers how prior condition changes affected the result) until the desired result is reached.

13.

Does Simon’s four-phase decision-making model fit into most of the modeling methodologies described? Explain.

Yes. They have a problem identification phase, followed by design and choice phases and then an implementation phase. This is only to be expected, since Simon’s model is a general description of how decisions are made. The modeling methodologies described, as specific approaches to specific decisions, of necessity fall within the general framework.

ANSWERS TO END OF CHAPTER APPLICATION CASE QUESTIONS 1.

 

What were the main challenges encountered by CARE International before they created their warehouse pre-positioning model?

CARE International faced challenges in offering the needed help to affected areas in the event of natural disasters. The previous mode of transportation, where thirdparty vendors fly into affected areas and CARE distributes by ground, was unreliable and slow. CARE wanted the ability to gather supplies and relief items from both local and international suppliers in an agile manner so they could better serve people affected by disasters. Once the supplies are mobilized, they wanted to be able to effectively distribute them in the most timely and cost-efficient manner to affected regions.

2.

How does the objective function relate to the organization’s need to improve relief services to affected areas?

The objective function of the model calculates the total response time in moving items to affected areas. The goal of the linear programming model is to minimize this value. Thus, the objective function is directly related to the main problem for CARE International, especially in response to natural disasters.

18 .

3.

Conduct online research and suggest at least three other applications or types of software that could handle the magnitude of variables and constraints CARE International used in their MIP model.

(Answers will vary by student.)

An Internet search finds many documented applications that involve the magnitude of variables and constraints of CARE International. Domain areas include large-scale farming econometrics (http://econpapers.repec.org/paper/wpawuwpem/0112002.htm), airline crew scheduling (http://www.cc.gatech.edu/projects/ihpcl/people/projects/lso.html), and supply chain (http://www.ima.umn.edu/optimization/abstracts/9-9abs.html).

Software products for large scale linear programming problems include AMPL, Gurobi, and SAS.

4.

Elaborate on some benefits CARE International stands to gain from implementing their pre-positioning model on a large scale in the future.

The main benefit is their ability to provide relief services more rapidly, especially in times of crisis. In addition, the pre-positioning model promises to save money, labor, and time by increasing the efficiency of their operations. Most importantly, CARE will save more lives and help people get back onto their feet more effectively over time.

19 .

Chapter 10: Modeling and Analysis: Heuristic Search Methods and Simulation Learning Objectives for Chapter 10 1.

2.

3.

4.

5.

6.

Explain the basic concepts of simulation and heuristics, and when to use them Understand how search methods are used to solve some decision support models Know the concepts behind and applications of genetic algorithms Explain the differences among algorithms, blind search, and heuristics Understand the concepts and applications of different types of simulation Explain what is meant by systems dynamics, agent-based modeling, Monte Carlo, and discrete event simulation 7. Describe the key issues of model management

CHAPTER OVERVIEW In this chapter, we continue to explore some additional concepts related to the model base, one of the major components of decision support systems (DSS). As pointed out in the last chapter, we present this material with a note of caution: The purpose of this chapter is not necessarily for you to master the topics of modeling and analysis. Rather, the material is geared toward gaining familiarity with the important concepts as they relate to DSS and their use in decision making. We discuss the structure and application of some successful time-proven models and methodologies: search methods, heuristic programming, and simulation. Genetic algorithms mimic the natural process of evolution to help find solutions to complex problems. The concepts and motivating applications of these advanced techniques are described in this chapter, which is organized into the following sections: CHAPTER OUTLINE 10.1

OPENING VIGNETTE: SYSTEM DYNAMICS ALLOWS FLUOR CORPORATION TO BETTER PLAN FOR PROJECT AND CHANGE MANAGEMENT  Questions for the Opening Vignette A. WHAT WE CAN LEARN FROM THIS VIGNETTE

10.2

PROBLEM-SOLVING SEARCH METHODS A. ANALYTICAL TECHNIQUES 1 .

B. C. D.

ALGORITHMS BLIND SEARCHING HEURISTIC SEARCHING ⧫ Application Case 10.1: Chilean Government Uses Heuristics to Make Decisions on School Lunch Providers  Section 10.2 Review Questions

10.3

GENETIC ALGORITHMS AND DEVELOPING GA APPLICATIONS A. EXAMPLE: THE VECTOR GAME B. TERMINOLOGY OF GENETIC ALGORITHMS C. HOW DO GENETIC ALGORITHMS WORK? D. LIMITATIONS OF GENETIC ALGORITHMS E. GENETIC ALGORITHM APPLICATIONS  Section 10.3 Review Questions

10.4

SIMULATION ⧫

A. B. C. D. E.

F. G.

Application Case 10.2: Improving Maintenance Decision Making in the Finnish Air Force Through Simulation ⧫ Application Case 10.3: Simulating Effects of Hepatitis B Interventions MAJOR CHARACTERISTICS OF SIMULATION ADVANTAGES OF SIMULATION DISADVANTAGES OF SIMULATION THE METHODOLOGY OF SIMULATION SIMULATION TYPES 1.

Probabilistic Simulation 2.

Time-Dependent Versus Time-Independent Simulation MONTE CARLO SIMULATION DISCRETE EVENT SIMULATION 1.

Visual Simulation  Section 10.4 Review Questions

10.5

VISUAL INTERACTIVE SIMULATION A. CONVENTIONAL SIMULATION INADEQUACIES B. VISUAL INTERACTIVE SIMULATION C. VISUAL INTERACTIVE MODELS AND DSS ⧫ Application Case 10.4: Improving Job-Shop Scheduling Decisions Through RIFD: A Simulation-Based Assessment D. SIMULATION SOFTWARE  Section 10.5 Review Questions

10.6

SYSTEM DYNAMICS MODELING  Section 10.6 Review Questions

10.7

AGENTS-BASED MODELING

2 .

⧫

Application Case 10.5: Agent-Based Simulation Helps Analyze Spread of a Pandemic Outbreak

Chapter Highlights Key Terms Questions for Discussion Exercises Teradata University Network (TUN) and Other Hands-On Exercises ⧫ End of Chapter Application Case: HP Applies Management Science Modeling to Optimize Its Supply Chain and Wins a Major Award  Questions for the Case References TEACHING TIPS/ADDITIONAL INFORMATION

       

The purpose of this chapter is to teach the basics of some key modeling techniques, particularly with respect to prescriptive analytics. The concept of problemsolving search is described, including optimization, blind search, and heuristics. Then genetic algorithms and simulation approaches (including visual, system dynamic, and agent-based) are described. It is important to stress to students that genetic algorithms are geared toward finding optimal solutions to problems, whereas simulation techniques are more experimental in nature. So, whereas genetic algorithms attempt to find the “best” solution to a problem, simulation techniques essentially allow users to play “what-if” games. So, it is important to stress the difference between the prescriptive nature of genetic algorithms vs. the descriptive nature of simulation.

When covering genetic algorithms in class, it is useful to play the Vector game as an exercise (the Vector game is described in section 10.3). After covering the concepts underlying evolutionary theory (reproduction, crossover, mutation, and elitism or survival of the fittest) and explaining the rules of the game, you can break the students into pairs.

One student of the pair thinks of a six-digit string of 1s and 0s. The second presents four guesses, which the first evaluates based on the number of correct digits (the fitness function). The second then selects the best two and applies reproduction, crossover, and mutation for the next generation of guesses. Have your teams continue this process until correct solutions are found. You can then compare the number of generations required vs. what would have been required by chance (an average of 32 given 6 digits). You should find that the majority of teams find solutions considerably faster than the random-guess approach.

When covering simulation, you can work through a simple simulation by hand— for example, modeling waiting times and customers turned away at a barber shop with two barbers, two more chairs in the waiting area, and any distribution of arrival and service times you choose. Such a simulation can easily be done using a standard six-sided die to generate random numbers or by any of several other “low-tech” methods. If you have access to and are familiar with a simulation package, showing students how this 3 .

simulation is programmed for that package will bring home the lessons of what simulation is really about.

Most of the application cases in this chapter involve applications of simulation to real-world problems. These are useful for giving further illustrations of how what-if analysis can be used. Have students read and discuss these cases during class. You can also assign additional readings of the articles cited in the chapter to deepen their understanding of the material.

ANSWERS TO END OF SECTION REVIEW QUESTIONS

     

Section 10.1 Review Questions 1.

Explain the use of system dynamics as a simulation tool for solving complex problems.

System dynamics is a methodology and simulation-modeling technique for analyzing complex systems using principles of cause and effect, feedback loops, and time-delayed and nonlinear effects. System dynamics fosters the understanding of business processes by creating “what-if” analyses of the impact of both expected and unexpected changes in project plans. As a simulationmodeling technique, it falls under the same general category with other simulation modeling techniques (Monte Carlo, discrete/continuous event, probabilistic, agent-based). The objective of system dynamics is to study the overall behavior of a system, focusing on the interplay between the system's components over time. A key feature of system dynamics involves identifying and diagramming causal loops in a system.

2.

In what ways was it applied in Fluor Corporation to solve complex problems?

At Fluor, system dynamics was used to analyze changes in project plans over time, and the company was able to save $10 million by predicting future impact of changes to a mining project. Fluor’s use of system dynamics helped the company understand the dynamics of the different situations that could warrant changes to project plans. Using the original project plan, industry standards, and company references as input, their system dynamics model simulated the correct amounts and timing of other factors like staffing, project progress, and effects on productivity. The resultant system provided reports on project impacts as well as helping to perform cause–effect diagnostics.

3.

How does a what-if analysis help a decision maker to save on cost?

Through the use of what-if analysis stemming from their system dynamics modeling approaches, Fluor’s customers were able to perform “what-if” analysis even before a project was started so the project performance could be gauged. The 4 .

model explained why certain effects were realized based on impact to the project plan. Fluor’s model was used to analyze and save $10 million in the future impact of changes to a mining project, as well as saving $10 million when the project team used the model to redesign the process of reviewing changes so that the speed of the company’s definition and approval procedures was increased.

4.

In your own words, explain the factors that might have triggered the use of system dynamics to solve change management problems in Fluor Corporation.

Fluor had experienced situations of reduced efficiencies, friction with their clients, and legal ramifications in previous projects. This is because project changes were collated and reported at a later period and the burden of cost allocated to the stakeholder responsible, leading to late surprises regarding project costs and schedules.

5.

Pick a geographic region and business domain and list some corresponding relevant factors that would be used as inputs in building such a system.

Fluor’s business domain is engineering and construction, and it is a worldwide company. System dynamics have also been used for modeling impact in other domains, such as electronic health records in hospital settings, In principal, any complex system with multiple interacting components acting over time can be modeled using the system dynamic approach.

Section 10.2 Review Questions 1.

What is a search approach?

A search approach is a general method, or a category of closely related methods, used to find the best solution to a problem based on some criterion. The criterion usually involves maximizing or minimizing a numeric value.

2.

List the different problem-solving search methods.

Analytical techniques, algorithms, blind searching, and heuristic searching

3.

What are the practical limits to blind searching?

Practical limits are the time and computer resources available for the search.

4.

How are algorithms and heuristic search methods similar? How are they different?

The similarity is that they are both search approaches by the above definition (see Question 1 above).

5 .

The difference is that algorithms always find a solution, if one exists within the constraints of the method, and that solution can always be proven to be mathematically optimal. Heuristic search methods may or may not find one. If one is found, it may or may not be “best” by the stated criteria. Their advantages over algorithms are that they can be used for some problems that lack applicable algorithms, and they are often faster.

(This answer uses the term “algorithm” within the context of solving business problems. The term has a more general meaning: “an explicit numerical step-bystep procedure that produces a solution to a known problem.” Some algorithms do not involve optimization, as an algorithm to calculate square roots. Others apply to areas where no mathematical optimum exists, for example, Google’s search algorithm mentioned in the text. There is no way to know if the pages it finds are best for a particular purpose, since people will disagree on what “best” means in this context. A person searching for “diamonds,” wanting information on gemstones, might find its answers useful, but the same results would be less useful to someone in need of information on baseball parks, geometric shapes, or the suit in a deck of cards.)

Section 10.3 Review Questions 1.

Define genetic algorithm.

A genetic algorithm is a machine learning heuristic search technique for finding approximate solutions to optimization-type problems that are too complex to be solved with traditional optimization methods. It applies the steps of the biological process of evolution and the principle of “survival of the fittest” in order to determine the best solution to a problem.

2.

Describe the evolution process in genetic algorithms. How is it similar to biological evolution?

The evolution process in genetic algorithms involves many elements similar to biological evolution. First is the notion of “mating” in which portions of two individuals (in GA these are potential solutions) are mixed together to produce an offspring, similar to how genes of individual organisms are mixed. Second is the notion of mutation, which can result either in “death” or in a more optimal solution. Third is the notion of survival-of-the-fittest in which only the most adaptive individuals survive to mate and produce offspring in the next generation.

3.

Describe the major genetic algorithm operators.

The major genetic algorithm operators include: reproduction (creating new generations of improved solutions), crossover (random exchange of segments of strings from previous solution candidates to produce new ones), mutation 6 .

(random and arbitrary changes to solutions string elements), and elitism (the survival-of-fittest principle). Elitism is implemented through application of a fitness function, which determines the “best” solutions of a given generation.

4.

List major areas of genetic algorithm application.

Genetic algorithms have been applied to dynamic process control, rule optimization induction, discovery of new connectivity topologies, simulation of biological models of behavior and evolution, complex design of engineering structures, pattern recognition, scheduling, transportation and routing, layout and circuit design, telecommunication, and graph-based problems.

5.

6.

Describe in detail three genetic algorithm applications.

•

Vehicle routing

•

Bankruptcy prediction

•

Web searching

Describe the capability of Evolver as an optimization tool.

Evolver (from Palisade Corp., palisade.com) is an optimization add-in for Excel.

It uses a genetic algorithm to solve complex optimization problems in finance, scheduling, manufacturing, and other problem domains. The Palisade web site states capabilities for model definition, optimization, and reporting features. It also includes a developer kit for integrating with software applications.

Section 10.4 Review Questions 1.

List the characteristics of simulation.

• • • • •

2.

An imitation of reality rather than a representation of it A technique for conducting experiments Fewer simplifications than with most other methods Descriptive rather than normative; users can use simulation results to search if desired Usually used for problems too complex for numerical optimization methods

List the advantages and disadvantages of simulation.

Advantages: • The theory is fairly straightforward.

7 .

• • • • • • • • • • • • • •

Time can be compressed a great deal, quickly giving a manager some feel as to the long-term effects of many policies.

Simulation is descriptive rather than normative. This allows a manager to pose what-if questions. Managers can use trial-and-error quickly, at little expense, accurately, and with low risk.

A manager can experiment to determine which decision variables and which parts of the environment are really important, and with different alternatives.

An accurate simulation model requires intimate knowledge of the problem, thus forcing its builder to interact with the manager. This leads to better understanding of the problem and the potential decisions available.

The model is built from the manager’s perspective.

No generalized understanding is required of the manager; as every component in the model corresponds to part of the real system.

Simulation can handle a wide variety of problem types, such as inventory and staffing, as well as higher-level managerial functions, such as longrange planning.

Simulation can include most real complexities of problems; simplifications are not needed. For example, it can use real probability distributions rather than approximate theoretical ones.

Simulation automatically produces many important performance measures.

Simulation is often the only DSS modeling method that can readily handle relatively unstructured problems.

Simulation generally can include the real complexities of problems; simplifications are not necessary.

Simulation automatically produces many important performance measures.

Simulation is often the only DSS modeling method that can readily handle relatively unstructured problems.

There are some relatively easy-to-use simulation packages. These include add-in spreadsheet packages, influence diagram software, Java-based (and other Web development) packages, and visual interactive simulation systems.

Disadvantages: • An optimal solution cannot be guaranteed, though relatively good ones are generally found.

• Simulation model construction can be a slow and costly process, although newer modeling systems are easier to use than ever.

• Solutions and inferences from a simulation study are usually not transferable to other problems because the model incorporates unique problem factors. It doesn’t generalize.

8 .

• •

3.

Simulation is sometimes so easy to explain to managers that analytic methods are often overlooked. (This is not a disadvantage of the method, but a practical caution about using it.)

Simulation software sometimes requires special skills because of its complexity. Some simulation packages require the model developer to be, in effect, a programmer.

List and describe the steps in the methodology of simulation.

The steps are shown in Figure 10.4. They are: • Define the problem • Construct the simulation model • Test and validate the model • Design the experiments • Conduct the experiments • Evaluate the results • Implement the results

4.

List and describe the types of simulation.

Important types of simulation include probabilistic simulation, time-dependent and time-independent simulation, and visual simulation.

Probabilistic simulation is simulation where one or more of the independent variables is described by a statistical distribution.

Time-dependent and time-independent simulations are distinguished by whether or not it is important, in order to model the system correctly, to know exactly when events occur.

Visual simulation is the graphical display of computerized results and is one of the most successful developments in computer-human interaction and problem solving.

Section 10.5 Review Questions 1.

Define visual simulation and compare it to conventional simulation.

Visual simulation uses graphical representation to show the situation to the end user. It does everything a conventional simulation does, which is any technique for conducting experiments (such as what-if analyses) with a digital computer on a model of a management system, but does it using visually pleasing and informative representation.

9 .

Visual interactive simulation is a type of visual simulation in which input is provided directly to the visual representation of the system, with results visible immediately or nearly so.

2.

Describe the features of VIS (i.e., VIM) that make it attractive for decision makers.

The main attraction of VIS (VIM) is the graphical display. This type of interaction can help managers learn about the decision-making situation, though some people respond to graphical displays better than others. Non-technical managers, who may not understand a situation from data in tables or graphs, can grasp what goes on in a visual presentation easily.

3.

How can VIS be used in operations management?

According to the text, VIS (VIM) “has been used in several operations management decisions.” Examples given there include plant operations and waiting lines. The two are related, as waiting line buildup indicates an unbalanced plant. With VIS, a manager can see work in progress piling up at one stage of a process—which is far less expensive than watching it pile up in the real factory!

4.

How is an animated film like a VIS application?

An animated film is (with modern animation techniques; this question is not about hand-drawn animations of a few decades ago) a computer-generated representation of a physical system, as is a VIS application. While some animated films use computers simply to generate final artwork and to fill in the gaps between key frames created manually, others use complex algorithms to simulate realistic motion of animated people or animals, calculate trajectories of physical objects, and determine what happens when two objects interact (e.g., a vehicle hitting a wall). These are true VIS applications, even if their purpose is not to support management decision making.

Section 10.6 Review Questions 1.

What is the key difference between system dynamics simulation and other simulation types?

Whereas discrete-event and agent-based simulation models are most appropriate for middle or low levels of abstraction, considering individual elements of a system, systems dynamics is more appropriate for aggregate analysis at a higher level of abstraction.

2.

What is the purpose of a causal loop diagram?

10 .

Causal loop diagrams show the relationships between variables in a system. This is a qualitative step in which the processes, variables, and relationships within the conceptual model are identified. Causal loop diagrams can be transformed into mathematical equations that represent the relations among variables.

3.

How are relationships between two variables represented in a causal loop diagram?

A link between two elements shows that changes in one element lead to changes in the other one. The direction of the link shows the direction of influence between two elements. The sign of each arrow shows the direction of change between each pair of elements. Feedback processes in the causal loops are the key components by which a variable re-affects itself over time through a chain of causal relationships.

Section 10.7 Review Questions (This section has no review questions.)

ANSWERS TO APPLICATION CASE QUESTIONS FOR DISCUSSION

 

Application Case 10.1: Chilean Government Uses Heuristics to Make Decisions on School Lunch Providers 1.

What were the main challenges faced by JUNAEB? Reduction of suppliers due to Chilean economic conditions put strain on the school meal policy and system. Disparities between territorial units (TUs) led to a need to configure new regional configurations.

2.

What operation research methodologies were employed in achieving homogeneity across territorial units?

JUNAEB utilized the analytic hierarchy process (AHP), local search heuristics, cluster enumeration, and integer linear programming (ILP) approaches. AHP was used for determining initial weights of characteristics for each TU in each region.

Search heuristics were used to find homogenously attractive TUs for a region.

Cluster enumeration and ILP were used to minimize differences between cluster attractiveness.

3.

What other approaches could you use in this case study?

There are many approaches to optimization, including blind search (exhaustive or partial), heuristic search, or genetic algorithms. Simulation approaches could also be employed.

11 .

Application Case 10.2: Improving Maintenance Decision Making in the Finnish Air Force through Simulation (This application case has no discussion questions.)

Application Case 10.3: Simulating Effects of Hepatitis B Interventions 1.

Explain the advantage of operation research methods such as simulation over clinical trial methods in determining the best control measure for Hepatitis B. Clinical trials are more costly, and there is a time delay in Hepatitis B onset that complicates analysis via clinical trials.

2.

In what ways do the decision and Markov models provide cost-effective ways of combating the disease?

The Markov model helps measure the economic and health benefits of various possibilities of screening, treatment, and vaccination, helping decision makers to select the most cost-effective strategy.

3.

Discuss how multidisciplinary background is an asset in finding a solution for the problem described in the case.

By bringing in experts from medical, management science, and engineering backgrounds, the right blend of knowledge and skills were combined to foster creative and innovative solutions to a complicated problem. Decision support models are effective for solving problems in a wide variety of domains, including medicine and epidemiology.

4.

Besides health care, in what other domain could such a modeling approach help reduce cost?

This modeling approach is also used widely in business, for example in solving operations problems such as supply chain, marketing, production, and scheduling.

There are applications in many other domains, including economics, biology, and even music composition and baseball analysis.

Application Case 10.4: Improving Job-Shop Scheduling Decisions Through RFID: A Simulation-Based Assessment (This application case has no discussion questions.)

Application Case 10.5: Agent-Based Simulation Helps Analyze Spread of a Pandemic Outbreak 12 .

1.

What are the characteristics of an agent-based simulation model?

ABM is a bottom-up approach to modeling complex systems, utilizing an “adaptive learning” property rather than “optimizing.” It is characterized by a set of autonomous decision-making units called agents that individually evaluate their situation and make decisions on the basis of a set of predefined behavior and interaction rules. ABM is suitable for understanding evolving and dynamic systems.

2.

List the various factors that were fed into the agent-based simulation model described in the case.

Factors included an individual’s age, residence, level of general interaction with other members of population, number of individuals in each household, distribution of households, and behavioral aspects involving daily commutes, attendance at schools, and asymptotic time period of disease.

3.

Elaborate on the benefits of using agent-based simulation models.

ABMs are better at dealing with heterogeneous behavior, and they allow each member of the population to be simulated individually. These models enhance the support for policy decision making.

4.

Besides disease prevention, in which other situations could agent-based simulations be employed?

Agent-based simulations can be used for many applications. ABMs have been used to simulate flocks of birds, social dynamics of science, traffic jams and crowd behavior, ant colonies, financial contagion, movements of ancient societies, housing segregation and other urban issues, and operations management problems. ABMs are viable for business problems in which many interrelated factors, irregular data, and high uncertainty and emergent behaviors exist, interactions between agents are complex, discrete, or nonlinear, and the population is heterogeneous.

ANSWERS TO END OF CHAPTER QUESTIONS FOR DISCUSSION 1.

  

Compare the effectiveness of genetic algorithms against standard methods for problem solving, as described in the literature. How effective are genetic algorithms?

Genetic algorithms have their limitations. They require expertise in programming and statistics in order to be useful techniques. They have the danger of converging on local maxima, and therefore missing the globally optimal solution. Coming up 13 .

with initial data conditions can be challenging. Nevertheless, for heuristic applications, genetic algorithms can be preferable to blind search, which is computationally demanding, or other optimization methods which may not have the flexibility and adaptiveness of Gas. 2.

Describe the general process of simulation.

The steps are shown in Figure 10.4. They are: • Define the problem • Construct the simulation model • Test and validate the model • Design the experiments • Conduct the experiments • Evaluate the results • Implement the results

3.

List some of the major advantages of simulation over optimization and vice versa.

Advantages: Simulation can deal with realistic, complex situations (it models risk), simulation theory is straightforward, time is compressed, there is no need for restrictive assumptions, and it fits how managers think (models are built from the manager’s perspective).

Disadvantages: An optimal solution is not guaranteed; construction can be slow; and simulation software is often not user-friendly, requiring programming.

4.

What are the advantages of using a spreadsheet package to perform simulation studies? What are the disadvantages?

Advantages: Ease of use because most managers understand the structure of spreadsheets.

Disadvantages: (For complex simulations) slow running time and the fact that the spreadsheet metaphor does not incorporate facilities for many types of simulations, so the developer must work around the limitations of the tool to construct these. In those cases, using a simulation package would be more straightforward.

5.

Compare the methodology of simulation to Simon’s four-phase model of decision making. Does the methodology of simulation map directly into Simon’s model?

Explain.

Both start off with a problem identification phase, called Intelligence in Simon’s model. Simon’s design phase corresponds to Steps 2–4 of the simulation model: constructing it, testing and validating it, and designing the experiment.

14 .

Conducting simulation experiments and evaluating the results would be Simon’s choice phase. Finally, both have an implementation phase.

6.

Many computer games can be considered visual simulation. Explain why.

Many computer games simulate a fictional (realistic or fantasy) environment and the action that takes place in it. They use visual presentation to show the progress of the simulation. This enhances the understanding of the environment and how the game is played, often providing immediate feedback as to the success or failure of a player action.

7.

Explain why VIS is particularly helpful in implementing recommendations derived by computers.

Implementation is a process of human change. People must be motivated to change. They are more likely to accept change if they see its benefits. VIS makes it possible to see those benefits.

ANSWERS TO END OF CHAPTER APPLICATION CASE QUESTIONS 1.

 

Describe the problem that a large company such as HP might face in offering many product lines and options.

The something-for-everyone approach drives sales, but at what cost? At what point does the price of designing, manufacturing, and introducing yet another new product, feature, or option exceed the additional revenue it is likely to generate?

Just as important, what are the costs associated with too much or too little inventory for such a product, not to mention additional supply chain complexity, and how does all of that impact customer satisfaction?

2.

Why is there a possible conflict between marketing and operations?

Marketing and sales always wanted more—more SKUs, more features, more configurations— and for good reason. Providing every possible product choice was considered an obvious way to satisfy more customers and generate more sales.

Supply chain managers, however, always wanted less. They wanted less to forecast, less inventory, and less complexity to manage. The drivers (on the supply chain side) were cost control.

3.

Summarize your understanding of the models and the algorithms.

1 produced an analytically driven process for evaluating new products for introduction, 15 .

2 created a tool for prioritizing existing products in a portfolio, and 3 developed an algorithm that solves the problem many times faster than previous technologies, thereby advancing the theory and practice of network optimization.

4.

Perform an online search to find more details of the algorithms.

Open-ended answer and will be dependent upon the students’ online search results.

5.

Why would there be a need for such a system in an organization?

A very large organization requires tenacity and skill to bring about major changes in the processes of a company.

6.

What benefits did HP derive from implementation of the models?

The RCO software algorithm was developed as part of HP Labs’ “analytics” theme, which applies mathematics and scientific methodologies to help decision making and create better-run businesses. Analytics is one of eight major research themes of HP Labs, which last year refocused its efforts to address the most complex challenges facing technology customers in the next decade. The RCO algorithm is an example of an innovation that helps drive efficiency with businesses and our customers.

16 .

Chapter 11: Automated Decision Systems and Expert Systems Learning Objectives for Chapter 11 1. Understand the concept and applications of automated rule-based decision systems 2. Understand the importance of knowledge in decision support 3. Describe the concept and evolution of rule-based expert systems (ES) 4. Understand the architecture of rule-based ES 5. Learn the knowledge engineering process used to build ES 6. Explain the benefits and limitations of rule-based systems for decision support 7. Identify proper applications of ES 8. Learn about tools and technologies for developing rule-based DSS

CHAPTER OVERVIEW This chapter addresses two issues. First, how do some of the analytics technologies, including predictive and optimization models, get used in practice? In many cases, results of predictive models or even optimization models get simplified as rules that are then implemented in other applications. We call these automated decision systems. Second, in addition to the use of data and mathematical models, some managerial decisions require qualitative information and the judgmental knowledge that resides in the minds of human experts. Therefore, it is necessary to find effective ways to incorporate such information and knowledge into decision support systems (DSS). A system that integrates knowledge from experts is commonly called a knowledge-based decision support system (KBDSS) or an intelligent decision support system (IDSS). A KBDSS can enhance the capabilities of decision support not only by supplying a tool that directly supports a decision maker, but also by enhancing various computerized DSS environments. The foundation for building such systems is the techniques and tools that have been developed in the area of artificial intelligence—rule-based expert systems being the primary one. This chapter introduces the essentials of automated decision systems and provides a detailed description of expert systems.

1 .

CHAPTER OUTLINE 11.1

OPENING VIGNETTE: INTERCONTINENTAL HOTEL GROUP USES DECISION RULES FOR OPTIMAL HOTEL ROOM RATES  Questions for the Opening Vignette A. WHAT WE CAN LEARN FROM THIS VIGNETTE

11.2

AUTOMATED DECISION SYSTEMS ⧫ Application Case 11:1: Giant Food Stores Prices the Entire Store  Section 11.2 Review Questions

11.3

THE ARTIFICIAL INTELLIGENCE FIELD  Section 11.3 Review Questions

11.4

BASIC CONCEPTS OF EXPERT SYSTEMS A. EXPERTS B. EXPERTISE C. FEATURES OF ES ⧫ Application Case 11.2: Expert System Helps in Identifying Sport Talents  Section 11.4 Review Questions

11.5

APPLICATIONS OF EXPERT SYSTEMS ⧫ Application Case 11.3: Expert System Aids in Identification of Chemical, Biological, and Radiological Agents A. CLASSICAL APPLICATIONS OF ES 1.

DENDRAL 2.

MYCIN 3.

XCON B. NEWER APPLICATIONS OF ES 1.

Credit Analysis Systems 2.

Pension Fund Advisors 3.

Automated Help Desks C. AREAS FOR ES APPLICATIONS  Section 11.5 Review Questions

11.6

STRUCTURE OF EXPERT SYSTEMS A. KNOWLEDGE ACQUISITION SUBSYSTEM B. KNOWLEDGE BASE C. INFERENCE ENGINE D. USER INTERFACE E. BLACKBOARD (WORKPLACE) F. EXPLANATION SYSTEM (JUSTIFIER) G. KNOWLEDGE-REFINING SYSTEM 2 .

⧫ 

Application Case 11.4: Diagnosing Heart Diseases by Signal Processing Section 11.6 Review Questions

11.7

KNOWLEDGE ENGINEERING A. KNOWLEDGE ACQUISITION ⧫ Technology Insights 11.1: Difficulties in Knowledge Acquisition B. KNOWLEDGE VERIFICATION AND VALIDATION C. KNOWLEDGE REPRESENTATION 1.

Production Rules 2.

Knowledge and Inference Rules D. INFERENCING 1.

Forward and Backward Chaining Example 2.

Inferencing with Uncertainty E. EXPLANATION AND JUSTIFICATION 1.

Why Explanations 2 How Explanations  Section 11.7 Review Questions

11.8

PROBLEM AREAS SUITABLE FOR EXPERT SYSTEMS  Section 11.8 Review Questions

11.9

DEVELOPMENT OF EXPERT SYSTEMS A. DEFINING THE NATURE AND SCOPE OF THE PROBLEM B. IDENTIFYING PROPER EXPERTS C. ACQUIRING KNOWLEDGE D. SELECTING THE BUILDING TOOLS 1.

General-Purpose Development Environment 2.

ES Shells 3.

Tailored Turn-Key Solutions 4.

Choosing an ES Development Tool E. CODING THE SYSTEM F. EVALUATING THE SYSTEM ⧫ Application Case 11.5: Clinical Decision Support System for Tendon Injuries  Section 11.9 Review Questions

11.10 CONCLUDING REMARKS Chapter Highlights Key Terms Questions for Discussion Exercises Teradata UNIVERSITY NETWORK (TUN) and Other Hands-on Exercises

3 .

⧫ 

End of Chapter Application Case: Tax Collections Optimization for New York State Questions for the Case

References TEACHING TIPS/ADDITIONAL INFORMATION

       

This chapter focuses on two rule-based BI technologies. The first is called Automated Decision Systems (ADS) or Decision Automation Systems (DAS). These types of systems use rule-based reasoning to automatically make decisions in situations where high volumes of these decisions must be made quickly. These types of systems often replace human beings. The second rule-based system category is Expert Systems (ES). Expert systems are generated using a knowledge acquisition methodology and provide advice to novices in a domain of expertise.

Section 11.1 presents a case of how ADS is used in the hotel industry. This is followed in section 11.2 with an overview of automated decision systems with an example of their use in the airline industry. Section 11.3 discusses how previous research in artificial intelligence has led to the types of algorithms currently used in ADS systems.

The AI discipline is influenced by a wide variety of other domains, including psychology, philosophy, neuroscience, linguistics, computer science, mathematics, and management science. Research in AI has led to a wide domain of intelligent systems that can perform complex reasoning actions; these include natural language understanding, voice recognition, game-playing, computer vision, and intelligent tutoring.

Section 11.4 follows with an overview of expert systems (ES), giving detailed discussion on the idea of expertise and knowledge, and comparing ES to conventional information systems. Section 11.5 discusses some applications of ES, including some of the early ES applications in medicine and science. This is followed in section 11.6 with a breakdown of the various components of ES systems, including development and consultation environments, knowledge acquisition, knowledge bases, inference engines, user interfaces, blackboards, and knowledge refinement.

Section 11.7 discusses the role of the knowledge engineer, and describes the knowledge engineering process. Also discussed are the various rule-based reasoning techniques, including forward and backward chaining and inference under uncertainty.

Section 11.8 describes problem areas that ES are typically applied to, including (but not limited to) interpretation, prediction, diagnosis, design, and planning. Section 11.9 describes the Expert Systems development process, including the use of expert system shells for knowledge engineering. Section 11.10 provides concluding remarks.

The vignettes and application cases provide good examples of ADS and ES in different domains. These include the hotel industry, the airline industry, and the food industry. Other cases describe applications for identifying sports talent, responding to terrorist chemical attacks, diagnosing heart diseases, and treating tendon injuries. It is useful to compare and contrast these cases in terms of the technologies used and resulting benefits.

4 .

ANSWERS TO END OF SECTION REVIEW QUESTIONS

     

Section 11.1 Review Questions 1.

Describe the challenges faced by IHG during development of their retail price optimization system.

There were many challenges faced by IHG. One was a drop in revenue in the hospitality industry overall. They also suffered from a flaw in their pricing decisions, involving an assumption of independence of demand from room charged. Another challenge was managing their large number of hotels, resulting in 273 million pricing decisions per day.

2.

Besides the hotel business in the hospitality industry, explain at least three other areas where an optimization model could be used.

In addition to the hospitality industry, the type of price optimization described in this case is also used in the airline industry (for ticket pricing), decisions related to purchasing from vendors, and pricing in other areas such as grocery services.

(This is an open-ended question and can include many other examples.)

3.

What other methods could be used to solve IHG’s price optimization problem?

Rule-based expert systems, linear and nonlinear regression models, and genetic algorithms are all possible approaches for solving IHG’s price optimization problems.

Section 11.2 Review Questions 1.

Define decision automation systems.

A decision automation system (DAS) or automated decision system (ADS) is a rule-based system that provides a solution, usually in one functional area (e.g., finance, manufacturing), to a specific repetitive managerial problem, usually in one industry (e.g., to approve or not to approve a request for a loan, to determine the price of an item in a store).

2.

What are the key components of a decision automation system?

5 .

The key components of a DAS are business rules and predictive and analytic models. These are processed by rule-based engines supplemented by forecasting and optimization algorithms.

3.

Which industries are big users of decision automation systems?

Decision automation systems are used extensively in the airline industry, credit industry, retail industry, and engineering industry, to name a few. Almost any industry faces complex tasks that require repetitive decisions in high volume and at high speed. DAS systems help with these types of tasks.

4.

How could decision automation systems assist consumers?

One example in which consumers are aided indirectly is when DAS helps companies better segment the customer base, enabling more customized offerings.

But DAS can also be used to assist customers directly by offering advice for products and/or services based on rule-based reasoning.

Section 11.3 Review Questions 1.

What is the definition of artificial intelligence?

Artificial intelligence (AI) is a collection of concepts and ideas related to the development of intelligent systems, and based on computer science, philosophy, electrical engineering, management science, psychology, and linguistics. AI is considered to be an area of computer science, involving the study of human thought processes and the representation and duplication of those thought processes in computers and robots. In essence, AI systems aim to produce behavior by a machine that, if performed by a human being, would be called intelligent.

2.

What are some of the major application areas of artificial intelligence?

Major AI application areas include intelligent tutoring, autonomous robots, intelligent agents, speech understanding, natural language processing, voice recognition, automated programming, machine learning, neural networks, genetic algorithms, computer vision, game playing, expert systems, and fuzzy logic.

3.

Identify some key characteristics of AI. The major characteristics of AI are symbolic processing, the use of heuristics instead of algorithms, and the application of inference techniques.

6 .

Section 11.4 Review Questions 1.

What is an ES? Expert systems are computer-based information systems that use expert knowledge to attain high-level decision performance in a narrowly defined problem domain.

2.

Explain why we need ES. Companies need expert systems for several reasons: to preserve professional expertise when an expert leaves; to document knowledge for retention, examination, and improvement; for training new employees and disseminating knowledge in an organization; and to transfer knowledge less expensively than transferring a human expert.

3.

What are the major features of ES? Expertise, symbolic reasoning, deep knowledge, and self-knowledge

4.

What is expertise? Provide an example.

Expertise is the extensive, task-specific knowledge that experts possess.

An example of expertise is the ability of an experienced automobile mechanic to go directly to the source of “that funny sound, sort of like ‘kabump, kabump,’ when I turn right.”

5.

Define deep knowledge and give an example of it.

Deep knowledge is complex knowledge not easily found among non-experts.

Deep knowledge is usually considered to be about the central ideas of a topic. It involves establishing complex connections to these concepts.

An example of deep knowledge is a physician’s understanding of a disease in terms of the underlying mechanism through which it affects the body and can be cured, versus a lay person’s superficial knowledge of the symptoms (e.g., headache). These can then only be alleviated (e.g., by aspirin that will not deal with the underlying cause of the condition).

7 .

Section 11.5 Review Questions 1.

What is MYCIN’s problem domain?

MYCIN was designed to diagnose bacterial blood infections.

2.

Name two applications of ES in finance and describe their benefits.

Open-ended answer, which depends on the applications each student or group finds.

3.

Name two applications of ES in marketing and describe their benefits.

Open-ended answer, which depends on the applications each student or group finds.

4.

Name two applications of ES in homeland security and describe their benefits.

Open-ended answer, which depends on the applications each student or group finds.

Section 11.6 Review Questions 1.

Describe the ES development environment.

The development environment is used by (or in conjunction with) the expert to introduce knowledge into an expert system’s knowledge base.

2.

List and define the major components of an ES. The major components of ES are: knowledge base, inference engine, and user interface.

3.

What are the major activities performed in the ES blackboard (workplace)?

The blackboard stores information about the current problem, including input data, intermediate hypotheses, and decisions. These decisions can include a plan of attack on a problem, a list of actions awaiting execution, and alternative solutions.

8 .

4.

What are the major roles of the explanation subsystem?

The roles are tracing conclusions to their sources, in terms of rules and data.

5.

Describe the difference between a knowledge base of an ES and an organizational knowledge base.

An organizational knowledge base contains unstructured knowledge about problem areas. It must be interpreted by people, often using a variety of software packages to deal with a variety of documents, before it is useful. The knowledge base of an ES contains knowledge that has been structured into the form (usually, but not necessarily, if-then-else) of specific rules that can be applied directly to a problem. The only software needed is an inference engine with its associated user interface.

Section 11.7 Review Questions 1.

State two production rules that can represent the knowledge of repairing your car.

IF engine does not turn over AND headlights illuminate, THEN battery is good.

IF engine does not turn over AND headlights do not illuminate, THEN battery may be dead.

2.

Describe how ES perform inference.

Inference involves going through the knowledge base rule by rule. For each rule examined, the expert system determines whether the information in its antecedent (the IF part) makes it true or false. It then knows if its consequent (the THEN part, perhaps also the ELSE part) is true or false. It uses that information to work through other rules. The two approaches to choosing the order in which to examine and process rules, forward and backward chaining, are described in the answer to the next question.

3.

Describe the reasoning procedures of forward chaining and backward chaining.

The forward chaining approach is to determine what you know, apply that information to the rules, and see what conclusion(s) you reach. (This is also called data-driven reasoning.) It selects rules by determining that the system knows the truth or falsity of their antecedents (the IF parts). It then derives further facts from their consequents (the THEN parts). These new facts allow it to determine the truth or falsity of the antecedents of more rules. It can then evaluate those rules

9 .

and determine still more facts from their consequents. The process continues until the system reaches its overall conclusion.

The process of backward chaining is to establish a goal, determine what must be found out to reach that goal, and try to find that knowledge. (This is also called goal-directed or goal-driven reasoning.) An expert system that uses backward chaining selects rules by examining the variables contained in their consequents.

It then tries to determine the values of variables used in their antecedents to decide if the rule applies or not.

4.

List the three most popular methods to deal with uncertainty in ES. • •

•

5.

Evaluation is a broad concept. Its objective is to assess an ES’s overall value. In addition to assessing acceptable performance levels, it analyzes whether the system would be usable, efficient, and cost-effective.

Validation is the part of evaluation that deals with the performance of the system (e.g., as it compares to the expert’s). Simply stated, validation is building the right system (i.e., substantiating that a system performs with an acceptable level of accuracy).

Verification is building the system right or substantiating that the system is correctly implemented to its specifications.

Why do we need to incorporate uncertainty in ES solutions?

In the realm of ES, these activities are dynamic because they must be repeated each time the prototype is changed. In terms of the knowledge base, it is necessary to ensure that the right knowledge base (i.e., that the knowledge is valid) is used. It is also essential to ensure that the knowledge base has been constructed properly (i.e., verification).

6.

What are the ways by which ES justify their knowledge?

This feature offers a means of evaluating the integrity of the system when it is to be used by the experts themselves. Two basic types of explanations are the why and the how. Metaknowledge is knowledge about knowledge. It is a structure within the system using the domain knowledge to accomplish the system’s problem-solving strategy. Explanation in rule-based ES is usually associated with a way of tracing the rules that are fired during the course of a problem-solving session. It uses “why” and “how” explanations.

10 .

Section 11.8 Review Questions 1.

Describe a sample ES application for prediction.

One prediction area that is suitable for expert systems is weather prediction. This system could use rules about weather patterns such as “if there is a cold front over New York State and the wind is from the west, temperatures in Massachusetts will drop in three to six hours.” If the rules are derived from the knowledge of expert meteorologists, its predictions could be about as accurate as theirs.

2.

Describe a sample ES application for diagnosis.

One diagnosis area that is suitable for expert systems is fault diagnosis in complex mechanical or electrical equipment. It would use rules based on the knowledge of experienced repair personnel to narrow down the cause of a failure to the optimal replaceable unit (ORU). Such a system might have to rely on people to carry out some of the required physical tests and report the results.

3.

Describe a sample ES application for the rest of the generic ES categories.

Open-ended answer, depending on which of the eight categories is or are chosen.

Answers should resemble the above two in their general form.

Section 11.9 Review Questions 1.

Describe the major steps in developing rule-based ES. Developing rule-based expert systems typically involves six major steps: Step 1. Defining the nature and scope of the problem Step 2. Identifying proper experts Step 3. Acquiring knowledge Step 4. Selecting the building tools Step 5. Coding the system Step 6. Evaluating the system

2.

What are the necessary conditions for a good expert?

According to the text: a thorough understanding of problem solving in the domain, an understanding of the role of ES and of decision support technology, and good communication skills to transfer his/her knowledge. Students may find other (generally longer) lists in other sources.

11 .

3.

Compare three different types of tools for developing ES. Programming from scratch: The most work, but the only way to get a truly customized ES and also a good learning process for budding technical specialists.

In most business situations, not worth the effort over using a shell unless the ES is to be a very small part of a larger system.

Using an ES shell: A good choice if one can be found to meet the requirements, as will usually be possible. Little is lost in performance over custom programming, since the inference engines of shells are carefully hand-coded by experts in inference methods and have, for popular shells, been refined over several generations of software.

Using a turn-key package: usually the most practical choice if a suitable one exists, but it often will not.

4.

List the criteria for choosing a development tool.

Cost (total, not just purchase price); functionality, compatibility with other software already in use; and vendor reliability and support.

5.

What is the difference between verification and validation of an ES? Verification confirms that the system follows the rules obtained from the expert.

Validation confirms that the system solves the problem it was intended to solve.

Section 11.10 Review Questions (This section has no review questions.)

ANSWERS TO APPLICATION CASE QUESTIONS FOR DISCUSSION

 

Application Case 11.1: Giant Food Stores Prices the Entire Store (This application case has no discussion questions.)

Application Case 11.2: Expert System Helps in Identifying Sport Talents (This application case has no discussion questions.)

Application Case 11.3: Expert System Aids in Identification of Chemical, Biological, and Radiological Agents

12 .

1.

How can CBR Advisor assist in making quick decisions?

It provides logical step-by-step procedures to determine the identity of a toxic agent when little or no information is available, which is typical at the beginning of a terrorism incident. There are screens that give information on several action items; including immediate response actions.

2.

What characteristics of CBR Advisor make it an expert system?

It incorporates the knowledge of the U.S. Environmental Protection Agency, (EPA) along with Dr. Lawrence H. Keith, president of Instant Reference Sources, Inc., and others from an extensive team that incorporated their knowledge, experience, and expertise, plus information in publicly available EPA documents.

It was developed using Exsys, Inc.’s Corvid® software, which uses classical expert systems technology.

3.

What could be other situations where such expert systems can be employed?

Expert systems, like the one used in CBR, can be applied to medical diagnosis, system configuration, tax planning, loan evaluation, financial planning, and network management, among many other possibilities. Anywhere you can utilize rule-based logic is a potential application of this technology.

Application Case 11.4: Diagnosing Heart Diseases by Signal Processing 1.

List the major components involved in building SIPMES and briefly comment on them.

Digitized heart-sound algorithms gather the heart sounds via an electronic stethoscope. A signal processing module (SPM) segments the inputs into cycles and identifies cardiac events. A rule-based Exsys Corvid expert system provides expert knowledge and reasoning along with documents from the medical literature.

2.

Do expert systems like SIPMES eliminate the need for human decision making?

For automated decision systems, which can make decisions for you by replicating the reasoning of an expert using the same sort of reasoning as expert systems, the answer is yes. But for SIPMES and similar expert systems, the answer is probably no. This system is intended to assist general physicians in their decision making under emergency situations where expert cardiologists and advanced medical facilities are not readily available. To an extent it replaces a human (the cardiologist), but the decision making is still ultimately in the hands of a human (the general physician).

13 .

3.

How often do you think that the existing expert systems, once built, should be changed?

It should be changed periodically. There will be cases when the recommendations for a particular case won’t be correct, so new rules need to be generated as the knowledge base becomes more refined. Also, new items from the relevant literature will be added. Knowledge management involves a continuous process of refining the knowledge base.

Application Case 11.5: Clinical Decision Support System for Tendon Injuries 1.

Research other expert systems in other domains and list a few of them.

Two early expert systems were MYCIN (medicine) and DENDRAL (chemistry).

Expert systems have been applied to finance (e.g., mortgage loan ESs), law (CHIRON, JUDGE), business, airline scheduling, automated help desk (HelpDeskIQ,) and a wide variety of other domains. Even computer games (Earl Weaver Baseball, Tony La Russa Baseball) use rule-based expert systems. (This is an open-ended question with many possibilities.)

2.

Why is it important to evaluate the expert systems before they are put into use?

The decisions made by the expert system can have significant consequences, and the tendon injury CDSS is an example. Therefore it is important to experts’ needs to be evaluated for quality, including evaluation, validation, and verification.

ANSWERS TO END OF CHAPTER QUESTIONS FOR DISCUSSION 1.

  

Why are automated decision systems so important for business applications?

The knowledge captured in rule-based ADSs helps preserve expertise when the human experts are not available. The ability for ADSs to automatically make decisions, especially repetitive decisions, helps make the company more efficient and effective. ADSs can often make a decision much faster than a human.

2.

It is said that powerful computers, inference capabilities, and problem-solving heuristics are necessary but not sufficient for solving real problems. Explain.

They are not sufficient because specific problem-related knowledge is required as well. Each problem uses different knowledge and often requires its own approach.

14 .

3.

Explain the relationship between the development environment and the consultation (i.e., runtime) environment.

The development environment is used by the knowledge engineer to code experts’ knowledge into the expert system. This is done through a knowledge acquisition (KA) process, often aided by KA tools. The consultation environment is the software used by an end user (the non-expert receiving the advice). This is implemented by applying an inference engine (or reasoning engine) to the knowledge that had been coded via the development environment.

4.

Explain the difference between forward chaining and backward chaining and describe when each one is most appropriate.

Backward chaining tends to ask for less data than forward chaining, but to examine more rules. Forward chaining tends to ask for more data, but uses the data to eliminate possibilities and therefore to process fewer rules. (An expert system can combine both methods.)

The choice between backward and forward chaining should lean toward the one that narrows the tree of choices faster. When there are many rules that start from input data, only a few of which apply in any given situation, forward reasoning works well because the first step eliminates most of the possibilities. When rules near the top of the tree can be eliminated without reaching all the way down the tree, backward chaining works better.

Configuring a computer system to meet user requirements is a good forward chaining application. Here, all system components—processors, disk drives, cabinets, power supplies, communications gear—must be addressed, so every toplevel rule always applies and every major branch of the rule tree must be used. It is more efficient to start with the requirements. They will eliminate groups of bottom-level rules as irrelevant in a particular case.

Backward chaining, on the other hand, is good for problem diagnosis. Here, a few top-level questions (“Is the warning light on?” “Does your back hurt?” “Are there brown spots on its leaves?”) isolate the problem to a set of related possibilities.

The system can then ignore rules that have to do with other possibilities, along with all the inputs that those rules might require.

(This information is not from the text, which does not address this issue. If you assign this question, explain to your students that they will have to research the answer in other places.)

15 .

5.

What kind of mistakes might ES make and why? Why is it easier to correct mistakes in ES than in conventional computer programs?

ES make mistakes due to lack of “common sense,” encountering problems that are outside their range of expertise, and errors made by the original expert(s) in providing rules to the system. It is easier to correct mistakes in ES than in conventional computer programs because, in an ES, each rule is an independent “chunk of knowledge.” It can be changed without directly affecting other rules, much like a row in a database table can be changed without affecting the content of other rows. In a conventional program, by contrast, the sequence of execution moves through the code in a controlled way. Any change to the code can affect the sequence in unpredictable ways and thereby cause errors. Extensive testing is required to make sure that a change in one area did not affect other areas.

6.

An ES for stock investment is developed and licensed for $1,000 per year. The system can help identify the most undervalued securities on the market and the best timing for buying and selling the securities. Will you order a copy as your investment adviser? Explain why or why not.

Such a system is unlikely to be successful once it becomes generally available.

The reason is that stocks it identifies as undervalued will be bought, thus raising their price until they are no longer undervalued. The first few people to purchase a stock so identified would make money. The rest would not. Unless I thought I could get to the market ahead of all the other thousands or millions of people who use this system, it wouldn’t work. (I might consider buying it to know which stocks to sell short at the end of the day, thinking that many of the people who followed the system’s recommendation will dump the stock as soon as they realize what happened and thus depress its price from its artificially inflated value, but that’s a separate issue.)

ANSWERS TO END OF CHAPTER APPLICATION CASE QUESTIONS 1.

 

What is the key difference between the former tax collection system and the new system?

The former tax collection system had rigid and manual rules that took too long to implement and also required too many resources and personnel to be used. The existing method for tax collection employed a linear approach to identify, initiate, and collect delinquent taxes. This approach emphasized what should be done, rather than what could be done, by tax collection officers. A “one-size-fits-all” procedure for data collection was used within the constraints of allowable laws.

16 .

By contrast, the current system utilizes analytics and optimization processes, including Markov Decision Process (MDP), with a Constrained Reinforcement Learning (C-RL) method. The C-RL generates tax collection rules based on taxpayer characteristics. The newer system results in more adaptive and costeffective decisions regarding the processing of delinquent taxes.

2.

List at least three benefits that were derived from implementing the new system.

• • • • •

3.

tax agency could collect delinquent tax when needed (based on dynamic conditions) instead of when allowed (based on preconceived rules) fewer personnel were needed for tax collection average age of cases dropped the amount of dollars collected per field agent increased 7 percent increase in revenue

In what ways do analytics and optimization support the generation of an efficient tax collection system?

At CCED, they were used to help improve decisions of which delinquent cases to pursue. They do this by applying rule-based reasoning along with predictive and prescriptive models.

4.

Why was tax collection a target for decreasing budget deficit in the State of New York?

A key part of the state’s budget is revenue from tax collection, which forms about 40 percent of their yearly revenue. Tax collection mechanism was therefore seen as one key area that would help decrease the state’s budget deficit if improved.

The goal was to optimize tax collection in a very efficient way.

17 .

Chapter 12: Knowledge Management and Collaborative Systems

Learning Objectives for Chapter 12 1.

2.

3.

4.

Define knowledge and describe the different types of knowledge Describe the characteristics of knowledge management Describe the knowledge management cycle Describe the technologies that can be used in a knowledge management system (KMS) 5. Describe different approaches to knowledge management 6. Understand the basic concepts and processes of groupwork, communication, and collaboration 7. Describe how computer systems facilitate communication and collaboration in an enterprise 8. Explain the concepts and importance of the time/place framework 9. Explain the underlying principles and capabilities of groupware, such as group support systems (GSS) 10. Understand how the Web enables collaborative computing and group support of virtual meetings 11. Describe the role of emerging technologies in supporting collaboration

CHAPTER OVERVIEW In this chapter, we study two major IT initiatives related to decision support. First, we describe the characteristics and concepts of knowledge management. We explain how firms use information technology (IT) to implement knowledge management (KM) systems and how these systems are transforming modern organizations. Knowledge management, although conceptually ancient, is a relatively new business philosophy. The goal of knowledge management is to identify, capture, store, maintain, and deliver useful knowledge in a meaningful form to anyone who needs it, anyplace and anytime, within an organization. Knowledge management is about sharing and collaborating at the organization level. People work together, and groups make most of the complex decisions in organizations. The increase in organizational decision-making complexity increases the need for meetings and groupwork. Supporting groupwork, where team members may 1 .

be in different locations and working at different times, emphasizes the important aspects of communications, computer-mediated collaboration, and work methodologies.

Group support is a critical aspect of decision support systems (DSS). Effective computersupported group support systems have evolved to increase gains and decrease losses in task performance and underlying processes. So this chapter covers both knowledge management and collaborative systems. It consists of the following sections:

CHAPTER OUTLINE 12.1

OPENING VIGNETTE: EXPERTISE TRANSFER SYSTEM TO TRAIN FUTURE ARMY PERSONNEL  Questions for the Opening Vignette A. WHAT WE CAN LEARN FROM THIS VIGNETTE

12.2

INTRODUCTION TO KNOWLEDGE MANAGEMENT A. KNOWLEDGE MANAGEMENT CONCEPTS AND DEFINITIONS B. KNOWLEDGE C. EXPLICIT AND TACIT KNOWLEDGE  Section 12.2 Review Questions

12.3

APPROACHES TO KNOWLEDGE MANAGEMENT A. THE PROCESS APPROACH TO KNOWLEDGE MANAGEMENT B. THE PRACTICE APPROACH TO KNOWLEDGE MANAGEMENT C. HYBRID APPROACHES TO KNOWLEDGE MANAGEMENT D. KNOWLEDGE REPOSITORIES  Section 12.3 Review Questions

12.4

INFORMATION TECHNOLOGY (IT) IN KNOWLEDGE MANAGEMENT A. THE KMS CYCLE B. COMPONENTS OF KMS C. TECHNOLOGIES THAT SUPPORT KNOWLEDGE MANAGEMENT 1.

Artificial Intelligence 2.

Web 2.0  Section 12.4 Review Questions

12.5

MAKING DECISIONS IN GROUPS: CHARACTERISTICS, PROCESS, BENEFITS, AND DYSFUNCTIONS A. CHARACTERISTICS OF GROUPWORK B. THE GROUP DECISION-MAKING PROCESS C. THE BENEFITS AND LIMITATIONS OF GROUPWORK ⧫ Technology Insights 12.1: Benefits of Working in Groups and Dysfunctions of the Group Process  Section 12.5 Review Questions

2 .

12.6

SUPPORTING GROUPWORK WITH COMPUTERIZED SYSTEMS A. AN OVERVIEW OF GROUP SUPPORT SYSTEMS (GSS) B. GROUPWARE C. TIME/PLACE FRAMEWORK  Section 12.6 Review Questions

12.7

TOOLS FOR INDIRECT SUPPORT OF DECISION MAKING A. GROUPWARE TOOLS 1.

Synchronous Versus Asynchronous Products 2.

Virtual Meeting Systems B. GROUPWARE C. COLLABORATIVE WORKFLOW D. WEB 2.0 E. WIKIS F. COLLABORATIVE NETWORKS  Section 12.7 Review Questions

12.8

DIRECT COMPUTERIZED SUPPORT FOR DECISION MAKING: FROM GROUP DECISION SUPPORT SYSTEMS TO GROUP SUPPORT SYSTEMS A. GROUP DECISION SUPPORT SYSTEMS (GDSS) B. GROUP SUPPORT SYSTEMS C. HOW GDSS (OR GSS) IMPROVE GROUPWORK D. FACILITIES FOR GDSS 1.

Decision Rooms 2.

Internet/Intranet-Based Systems  Section 12.8 Review Questions

Chapter Highlights Key Terms Questions for Discussion Exercises Teradata UNIVERSITY NETWORK (TUN) and Other Hands-on Exercises ⧫ End of Chapter Application Case: Solving Crimes by Sharing Digital Forensic Knowledge  Questions for the Case References

TEACHING TIPS/ADDITIONAL INFORMATION

       

This chapter is somewhat free standing in its content. If you are pressed for time in your course and need to cut topics, it offers one possibility. Its content is not used later in the book. Should you go this route, you may still want your students to read some of the early part of this chapter and Section 12.7 so they’ll have at least vocabulary-level acquaintance with the concepts.

3 .

Knowledge management has fast become an integral part of many organizations’ IT strategy. It is important to relate the concept of knowledge management to its practical implementations. As students study this chapter, you could have them research different KMS implementations as individual or team projects. How do they differ? What issues did each organization encounter? The results can lead to interesting discussions.

An example of knowledge students can relate to is “What should I take next year?” Some answers are documented in official policies: “All business sophomores must take Accounting 101–2.” Some are part of a nominal program that can be varied: “MIS juniors normally take Marketing 301, but it can be taken in the fall of the senior year.”

Some are part of advisors’ personal knowledge: “Art 311, Web Site Design, is a good way for MIS majors to fulfill half their liberal arts distribution requirement.” And some is even less official: “Prof. Fink is a better MIS 301 instructor than Prof. Lavin; she grades easier, too.” Most topics in this chapter can be related to this example: how could this information be stored? How could it be retrieved? Does the culture encourage people to share it? What would it take to change that? How can the technologies of this chapter be applied to it?

The Defense Ammunition Center vignette that opens this chapter is a good preview of what is to come. You might want to point out some of the coming themes in it, such as the nature of the knowledge, the technologies used, and the organizational resistance to informal learning. Some of these are brought out in the review questions for the section, but they’re worth noting even if you don’t use those.

In Section 12.2, tacit knowledge is the tricky concept. How can you manage something if you don’t even know exactly what it is? One point is that a lot of tacit knowledge doesn’t have to be tacit, if we think of it the right way.

Section 12.3 continues in a nontechnical vein by discussing KM processes and approaches. Section 12.4 gets to what some of your students have undoubtedly been waiting for: how technology fits into all this. Several of the tools mentioned here, such as document management systems, were not originally designed for knowledge management but turned out to be useful in it nonetheless.

In Section 12.5, the discussion shifts to the second chapter topic, collaborative systems. As with knowledge management, the chapter begins by laying a foundation, describing the business processes the systems are meant to assist. Section 12.5 is a review of group decision making; students likely will have encountered these principles in a management course, so you can review them quickly. Sections 12.6 and 12.7 move back to technology, identifying the elements of group support systems and groupware. Section 12.8 wraps up the material on collaboration with a comparison of the group decision support system (GDSS) and the more broadly useful group support system (GSS). Remind students—as they have probably heard time and again—that in organizations today, much of the work is done by groups. Whether or not an organization has a sophisticated, dedicated KM system or GSS, its people will likely use many of the applications.

4 .

ANSWERS TO END OF SECTION REVIEW QUESTIONS

     

Section 12.1 Review Questions 1.

What are the key impediments to the use of knowledge in a knowledge management system?

These systems fail for two main reasons: inadequate representation and lack of integration into the department’s decision-making process.

2.

What features are incorporated in a knowledge nugget in this implementation?

Knowledge nuggets in this implementation have several features that make the KNs easy to find, more user-friendly, and more effective in classroom training.

The summary page delivers the main point along with links to the features. Users can link to the video (with captions) of the interview that was used to capture the knowledge. Another link takes the user to a transcript of the interview, and still another links to a causal map illustrating the interviewee’s thought process. A feature called “related links” takes the user to lists of related regulatory guidance and KNs on relevant topics. The KN also provides information about the interviewee and subject matter experts, as well as supporting images. Users can apply Web 2.0 tools such as the ability to rate the KNs and suggest additional tags to aid future searches. A feature called My URL lets trainers insert URLs into lessons to link the lesson to the KN.

3.

Where else could such a system be implemented?

This type of system could be implemented wherever an organization would benefit from capturing and sharing ideas that employees have devised. Some of the conditions in the case are relevant: significant turnover (service members leaving and being replaced by new recruits) and working conditions that require individuals to improvise and be innovative. It would be less useful where jobs are very routine. Any reasonable ideas from students would be correct. Examples would be sharing knowledge from salespeople at a company that requires creative selling, sharing repair solutions among repair technicians who encounter unusual problems, and sharing customer service solutions at a company with high standards for customer satisfaction.

Section 12.2 Review Questions 1.

Define knowledge management and describe its purposes.

Knowledge management (KM) is the systematic and active management of ideas, information, and knowledge residing in an organization’s employees.

5 .

Its purposes include effective and efficient problem solving, dynamic learning, strategic planning, and decision making. KM initiatives focus on identifying knowledge, explicating it in such a way that it can be shared in a formal manner, and leveraging its value through reuse.

2.

Distinguish between knowledge and data.

Knowledge and data are related through information, which is also defined in this answer.

Data are facts, measurements, and statistics.

Information is organized or processed data that is timely (i.e., inferences from the data are drawn within the time frame of applicability) and accurate (i.e., with regard to the original data).

Knowledge is information that is contextual, relevant, and actionable.

3.

Describe the knowledge-based economy.

A knowledge-based economy is one in which the primary resource used to produce economic benefits is knowledge—as distinct from land, capital, energy, or any other resource. One difference between knowledge and these other resources is that knowledge can be shared without diminishing, and can actually increase as a result of such sharing. Knowledge is also easier to transfer than are other resources, making laws and barriers difficult to apply and reducing the effect of location. Also, knowledge can have widely different values to different people, or even to the same person at different times.

4.

Define tacit knowledge and explicit knowledge.

Explicit knowledge is knowledge that has been or can be articulated, codified, and stored. It deals with objective, rational, and technical information in manuals, documents, procedures, software, and even stories (though this last is less common in business).

Tacit knowledge has been described as “know-how,” as opposed to the “knowwhat” and “know-why” of explicit knowledge. It is often highly personal and difficult to formalize, and deals with subjective, cognitive, and experiential learning. Some tacit knowledge could be made explicit if the person holding the knowledge recognized the value of doing so, but in other cases it is difficult to codify (such as knowing how to ride a bicycle). Capturing and sharing tacit knowledge is a more difficult knowledge management task than is capturing and sharing explicit knowledge.

5.

Define KMS and describe the capabilities of KMS. The term KMS, or knowledge management system(s), refers to the use of modern information technology to systematize, enhance, and expedite knowledge 6 .

management inside and outside a firm. (See the answer to Review Question 1 of this section above for “knowledge management.”)

KMS capabilities include capturing knowledge, storing it, allowing users to search the system for relevant knowledge, and making knowledge available to them. Some KMS use specific technologies for these purposes and some add additional features (e.g., the ability to contact a person who stored certain knowledge in the system with a question about it) but those are the basic conceptual elements.

Section 12.3 Review Questions 1.

Describe the process approach to knowledge management.

The process approach attempts to codify organizational knowledge through formalized controls, processes, and technologies. It frequently uses IT in forms such as intranets and data warehousing.

2.

Describe the practice approach to knowledge management.

The practice approach to knowledge management assumes that the vast majority of organizational knowledge is tacit and that formal controls and processes are not effective in their transfer. The focus of this approach is to build communities of practice.

3.

Why is a hybrid approach to KM desirable?

Because both the practice approach and the process approach have drawbacks. By using each where it is most appropriate, typically evolving from a practice approach to a process approach, the advantages of both can be obtained while minimizing the impact of their drawbacks.

4.

Define knowledge repository and describe how to create one.

A knowledge repository is a database in an informal sense, though not in the formal sense of database management systems. It is an electronic storehouse of knowledge, which is often text based and may have widely varying characteristics. Its structure will depend on its purpose, the type(s) of knowledge it stores, and the way that knowledge is represented.

There are three ways to create a knowledge repository: to develop a custom Internet-based system, to purchase a document management system, or to purchase a knowledge management package. Whichever of these is chosen, it is important to make contributing knowledge easy for the contributor and to have a good way to catalogue the knowledge it will contain.

7 .

Section 12.4 Review Questions 1.

Describe the KMS cycle.

The KMS cycle has the steps listed below.

1.

Create knowledge.

2.

Capture knowledge.

3.

Refine knowledge.

4.

Store knowledge.

5.

Manage knowledge.

6.

Disseminate knowledge.

2.

List and describe the components of KMS. Components of KMS include communication, collaboration, and storage and retrieval.

3.

Describe how AI and intelligent agents support knowledge management.

Artificial intelligence supports knowledge management by doing the following: • Assist in and enhance searching knowledge (e.g., intelligent agents in Web searches) • Help establish knowledge profiles of individuals and groups • Help determine the relative importance of knowledge when it is contributed to and accessed from the knowledge repository • Scan e-mail, documents, and databases to perform knowledge discovery, determine meaningful relationships, glean knowledge, or induce rules for expert systems • Identify patterns in data (usually through neural networks) • Forecast future results by using existing knowledge • Provide advice directly from knowledge by using neural networks or expert systems • Provide a natural language or voice command–driven user interface for a KMS Intelligent agents support knowledge management by learning what the user prefers to see and how the user organizes it. Then, the intelligent agent takes over to provide that information at the desktop, just as a good administrative assistant would.

4.

Relate Web 2.0 to knowledge management.

Web 2.0 is a collective term for interactive applications that allow communication and collaboration online. These applications include mashups, social networks, media-sharing sites, RSS, blogs, and wikis. They make it easy and natural to share knowledge, thereby giving knowledge management a strong boost. In fact, they have made the term knowledge management almost redundant because nontechnical people now can readily share their knowledge. The ultimate value of 8 .

Web 2.0 in knowledge management is its ability to foster greater responsiveness, better knowledge capture and sharing, and more effective collective intelligence.

Section 12.5 Review Questions 1.

Define groupwork.

Groupwork is work done by two or more people together.

2.

List five characteristics of groupwork.

The text gives the characteristics listed below. A correct answer can consist of any five: • A group performs a task (sometimes decision making, sometimes not).

• Group members may be located in different places.

• Group members may work at different times.

• Group members may work for the same organization or for different organizations.

• A group can be permanent or temporary.

• A group can be at one managerial level or can span several levels.

• There can be synergy (leading to process and task gains) or conflict in groupwork.

• There can be gains and/or losses in productivity from groupwork.

• The task may have to be accomplished very quickly.

• It may be impossible or too expensive for all the team members to meet in one place, especially when the group is called for emergency purposes.

• Some of the needed data, information, or knowledge may be located in many sources, some of which may be external to the organization.

• The expertise of non-team members may be needed.

• Groups perform many tasks; however, groups of managers and analysts frequently concentrate on decision making.

• The decisions made by a group are easier to implement if supported by all (or at least most) group members.

3.

Describe the process of a group meeting for decision making.

The text lists the items listed below as “activities and processes” that characterize meetings. Other answers, that are more descriptive of the meeting process, can also be correct. If you have specific requirements in this regard, it might be well to state them.

•

The decision situation is important, so it is advisable to make it in a group in a meeting.

9 .

• • • • • •

A meeting is a joint activity engaged in by a group of people typically of equal or nearly equal status.

The outcome of a meeting depends partly on the knowledge, opinions, and judgments of its participants and the support they give to the outcome.

The outcome of a meeting depends on the composition of the group and on the decision-making process the group uses.

Differences in opinions are settled either by the ranking person present or, often, through negotiation or arbitration.

The members of a group can be in one place, meeting face-to-face, or they can be a virtual team, in which case they are in different places while in a meeting.

The process of group decision making can create benefits as well as dysfunctions.

Section 12.6 Review Questions 1.

Why do companies use computers to support groupwork?

Because it can help a group follow a process, save cost, expedite decisions, support virtual teams, improve access to external experts, and improve the decision-making process overall.

2.

Describe the components of the time/place framework.

The two components of the time/place framework are time and place. Either can be the same or different. The four combinations that result (same time/place, same time/different place, different time/same place, different time/place) define the framework.

Section 12.7 Review Questions 1.

List the major groupware tools and divide them into synchronous and asynchronous types.

Synchronous (real-time) groupware tools include audio teleconferencing, videoconferencing, data conferencing, Web conferencing, whiteboards, screen sharing, and instant video.

Asynchronous groupware tools include blogs, wikis, discussion groups, autoresponders, workflow software, interactive portals, and online workspaces.

Other asynchronous communication tools, such as e-mail, can be useful to groupwork but are not true “groupware.”

2.

Identify specific tools for Web conferencing and their capabilities.

Web-based systems allow improved, electronically supported virtual meetings.

Sources of online meetings and presentation tools include webex.com, 10 .

gotomeeting.com, Adobe.com, and Skype.com. Microsoft Office has a built-in capability for virtual meetings. These systems feature Webinars, screen sharing, audioconferencing, videoconferencing, polling, and question-and-answer sessions. Mobile phones allow live meetings through applications such as Facetime.

3.

Define wiki and wikilog.

A wiki is a Web site in which users can freely create and edit content, using any Web browser, via special server-based software. A wiki supports hyperlinks and has a simple text syntax for creating new pages and cross-links between internal pages on-the-fly. It is especially suited for collaborative writing. (The word wiki means “quick” in Hawaiian.)

A wikilog is essentially a blog that allows everyone to participate as a peer (a combination of wiki and blog). Anyone may add, delete, or change content.

4.

Define collaborative hub.

A collaborative hub is used by members of a B2B supply chain to improve effectiveness among manufacturing companies, their suppliers, and contract producers by reducing inventory, improving flexibility, and increasing supplychain transparency through the Internet.

Section 12.8 Review Questions 1.

Define GDSS and list the limitations of the initial GDSS software.

A GDSS is “an interactive computer-based system that facilitates the solution of semistructured and unstructured problems by a group of decision makers.” Its goal is “to improve the productivity of decision-making meetings, either by speeding up the decision-making process, by improving the quality of the resulting decisions, or both.”

The initial GDSS software had these three major limitations: 1.

Its use was limited to face-to-face meetings.

2.

Its use depended on a special “decision room” facility. This was expensive.

3.

It was designed to support a single, clearly defined, narrow task.

2.

Define GSS and list its benefits.

A group support system (GSS) is “any combination of hardware and software that enhances groupwork either in direct or indirect support of decision making. GSS is a generic term that includes all forms of collaborative computing.”

11 .

GSS benefits are improving the productivity and effectiveness of meetings by streamlining and speeding up the decision-making process (efficiency) and/or improving the quality of the decision (effectiveness). This is achieved by providing support for group members in generating and exchanging ideas, opinions and preferences, using specific features such as parallelism and anonymity.

The book provides a list of twelve GSS support activities. Students may be attracted to this list to answer this question. This would be incorrect, since activities are not benefits.

3.

List process gain improvements made by GSS. GSS addressed limitations of earlier-generation GDSS by adding support for virtual teams and by providing indirect support as well as decision-making tools.

Along with providing structure to decision making and ready access to information, a GSS supports activities such as idea generation, consensus building, conflict resolution, voting, and anonymous ranking of alternatives. A complete GSS is considered a specially designed information system, but many capabilities today are embedded in standard productivity tools. Modern systems also have been made easy to use, thanks to a Windows-based graphical user interface or Web browser interface. GSS are building up their capabilities to operate in both asynchronous and synchronous modes.

4.

Define decision room.

A decision room is an electronic meeting room with networked PCs and a large public screen at the front of the room. It may have software that runs over a LAN. A server PC is attached to a large-screen projection system and connected to the network to display the work at individual workstations and aggregated information from the facilitator’s workstation.

5.

Describe Web-based GSS. Web-based GSS is a system in which groupware allows group members to work from any location at any time, using their computers or mobile devices. It often includes audioconferencing and videoconferencing.

ANSWERS TO END OF CHAPTER QUESTIONS FOR DISCUSSION 1.

  

Why is the term knowledge so difficult to define?

There is a vast amount of literature about what knowledge and knowing means in epistemology, social sciences, philosophy, and psychology. Although business is more pragmatic, there is no uniform definition or consensus on what knowledge 12 .

and knowledge management specifically mean. We suspect that there are so many definitions because each field approaches knowledge differently, and each has different sets of assumptions.

2.

Describe and relate the different characteristics of knowledge to one another.

Unlike other assets, knowledge has: (see Gray 1999, cited in the chapter references) • Extraordinary leverage and increasing returns. Knowledge is not subject to diminishing returns. When it is used, it is not consumed. Its consumers can add to it, thus increasing its value. It can improve.

• Fragmentation, leakage, and the need to refresh. As knowledge grows, it branches and fragments. Knowledge is dynamic; it is information in action. Thus, an organization must continually refresh its knowledge base to keep it as a source of competitive advantage.

• Uncertain value. It is difficult to estimate the impact of an investment in knowledge.

• Uncertain value sharing. Similarly, it is difficult to estimate the value of sharing the knowledge, or even who will benefit most.

3.

Explain why it is important to capture and manage knowledge.

Knowledge can have incredible impact on an organization when it is captured, managed, and most importantly reused. The leveraging of knowledge is clear from the opening vignette. An investment of less than $8 million in KM contributed, presumably significantly, to a sales increase of $122 million during two years. While we don’t have figures tying the revenue increase to a profit increase, the positive impact of these incremental sales on the bottom line is almost surely over $8 million.

4.

Compare and contrast tacit knowledge and explicit knowledge.

See answer to Review Question 4 of Section 12.2. While it is more desirable to capture tacit knowledge because it is more valuable, explicit knowledge is typically more readily available.

5.

Explain why organizational culture must sometimes change before knowledge management is introduced.

If there is no culture of sharing, it must be developed or any KM effort is doomed to failure.

6.

How does knowledge management attain its primary objective?

The objective of KM is to identify knowledge, explicate it so it can be shared, and leverage its value through reuse. KM attains its objectives by providing a means to capture knowledge and then make it available throughout an organization.

13 .

7.

How can employees be motivated to contribute to and use KMS? Real reward systems and recognition seem to work best. People really like recognition (putting their names at the bottom as contributors). It can help if raises and promotions are tied to knowledge contribution and use.

8.

What is the role of a knowledge repository in knowledge management?

The knowledge repository is the storage mechanism for the knowledge. It is like a database in function, but is not a database as the term database is usually understood.

9.

Explain the importance of communication and collaboration technologies to the processes of knowledge management.

Without communication, there is no mechanism to share knowledge.

Collaboration, a fancy word for people working together, is at the heart of any type of sharing—including knowledge.

10.

List three top technologies most frequently used for implementing KMS and explain their importance.

The three top technologies most frequently used for implementing KMS are communication technologies, collaboration technologies, and storage/retrieval technologies. Without all of these, the objectives of knowledge management cannot be achieved.

11.

Explain why it is useful to describe groupwork in terms of the time/place framework.

Because the type of information system needed to support the groupwork depends on where the work is in terms of this framework.

12.

Describe the kinds of support that groupware can provide to decision makers.

The support can be divided into three categories: General (can be either synchronous or asynchronous) • Built-in e-mail, messaging system • Browser interface • Joint Web-page creation • Sharing of active hyperlinks • File sharing (graphics, video, audio, or other) • Built-in search functions (by topic or keyword) • Workflow tools • Use of corporate portals for communication, collaboration, and search • Shared screens • Electronic decision rooms • Peer-to-peer networks Synchronous (same-time) 14 .

• Instant messaging (IM) • Videoconferencing, multimedia conferencing • Audio conferencing • Shared whiteboard, smart whiteboard • Instant video • Brainstorming • Polling (voting), and other decision support (consensus builder, scheduler) Asynchronous (different times) • Workspaces • Threaded discussions • Users can receive/send e-mail, SMS • Users can receive activity notification alerts, via e-mail or SMS • Users can collapse/expand discussion threads • Users can sort messages (by date, author, or read/unread) • Auto responder • Chat session logs • Bulletin boards, discussion groups • Use of blogs, wikis, and wikilogs • Collaborative planning and/or design tools • Use of bulletin boards 13.

Explain why most groupware is deployed today over the Web. Group members can work from any location at any time. Also, the software is relatively inexpensive, and it can run on computers and mobile devices that now offer sufficient power at a low cost.

14.

Explain why meetings can be so inefficient. Given this, explain how effective meetings can be run.

Almost all of the reasons involve people. Some don't take meetings seriously.

Other issues are that a large amount (perhaps as high as 90 percent) of the information discussed at a meeting is either not remembered or remembered incorrectly. This could become an open-ended answer if you let the students describe their meeting experiences.

How can effective meetings be run? Some items that might be included are: have an agenda, keep to the subject, record the minutes of each meeting, send out a meeting notice, and ask meeting participants to summarize results.

15.

Explain how GDSS can increase some of the benefits of collaboration and decision making in groups and eliminate or reduce some of the losses.

GDSS provides support to reduce the negative effects of group decision making and increase the positive effects. For example, anonymity means more equal participation, whereas parallelism saves time. By moving quickly, people get the urge to contribute. There might be less free-riding. There are many more benefits.

15 .

16.

The original term for group support system (GSS) was group decision support system (GDSS).Why was the word decision dropped? Does this make sense?

Why or why not?

Because GSS can help even when there is no decision to be made. For instance, it can help group members communicate. Many meetings are not held to make a formal decision. Researchers noticed this and changed the name of the technology to reflect reality.

ANSWERS TO END-OF-CHAPTER APPLICATION CASE QUESTIONS 1.

 

Why should digital forensics information be shared among law enforcement communities?

Law enforcement agencies seeking evidence face a staggering number and variety of digital devices to search, and each type of hardware and software requires the creation of new methods of search. As new techniques are developed and tested in the courts, applying the accepted methods would save an enormous amount of time and effort. Although little time is available for sharing through formal means such as briefings, not sharing causes duplication of effort and failure to take consistent advantage of precedent rulings that could strengthen the chance a court will accept the use of a given process.

2.

What does egocentric theory suggest about knowledge sharing?

Egocentric theory says that within a group, members identify experts through interaction, and knowledge remains primarily tacit. Group members use informal means of communication to transfer knowledge from one person to another. The informality of knowledge transfer results in local pockets of expertise and redundant effort across the larger community.

3.

What behavior did the developers of NRDFI observe in terms of use of the system?

The developers saw that more than 1,000 users quickly joined, but their usage pattern was almost entirely unidirectional. Typically, a user would log on, download tools and documents, and then wait to return until after the content on the site had be extensively refreshed. For the most part, the tools for communities to share information with one another were unused. The culture of law enforcement values the protection of information, rather than the sharing of knowledge. The developers have been exploring and trying ways to promote greater communication and knowledge sharing.

16 .

4.

What additional features might enhance the use and value of such a KMS? Answers will vary; accept any reasonable suggestions, including features described in other chapters, if they enhance knowledge sharing. Tools and technologies mentioned in the chapter include artificial intelligence, Web 2.0 (for example, social networking, media sharing, and blogs), groupwork applications to improve the efficiency and effectiveness of meetings, virtual meeting systems, and group support systems.

17 .

Chapter 13: Big Data and Analytics

Learning Objectives for Chapter 13 1. Learn what Big Data is and how it is changing the world of analytics 2. Understand the motivation for and business drivers of Big Data analytics 3. Become familiar with the wide range of enabling technologies for Big Data analytics 4. Learn about Hadoop, MapReduce, and NoSQL as they relate to Big Data analytics 5. Understand the role of and capabilities/skills for data scientist as a new analytics profession 6. Compare and contrast the complementary uses of data warehousing and Big Data 7. Become familiar with the vendors of Big Data tools and services 8. Understand the need for and appreciate the capabilities of stream analytics 9. Learn about the applications of stream analytics

CHAPTER OVERVIEW

              

Big Data, which means many things to many people, is not a new technological fad. It is a business priority that has the potential to profoundly change the competitive landscape in today’s globally integrated economy. In addition to providing innovative solutions to enduring business challenges, Big Data and analytics instigate new ways to transform processes, organizations, entire industries, and even society all together. Yet extensive media coverage makes it hard to distinguish hype from reality. This chapter aims to provide a comprehensive coverage of Big Data, its enabling technologies, and related analytics concepts to help understand the capabilities and limitations of this emerging paradigm. The chapter starts with a definition and related concepts of Big Data followed by the technical details of the enabling technologies including Hadoop, MapReduce, and NoSQL. After describing “data scientist” as a new, fashionable organizational role/job, we provide a comparative analysis between data warehousing and Big Data analytics.

The last part of the chapter is dedicated to stream analytics, which is one of the most promising value propositions of Big Data analytics.

1 .

This chapter contains the following sections:

CHAPTER OUTLINE 13.1

OPENING VIGNETTE: BIG DATA MEETS BIG SCIENCE AT CERN  Questions for the Opening Vignette A. WHAT WE CAN LEARN FROM THIS VIGNETTE

13.2

DEFINITION OF BIG DATA A. THE VS THAT DEFINE BIG DATA 1.

Volume ⧫ Technology Insights 13.1: The Data Size Is Getting Big, Bigger, and Bigger 2.

Variety 3.

Velocity 4.

Veracity 5.

Variability 6.

Value Proposition ⧫ Application Case 13.1: Big Data Analytics Helps Luxottica Improve Its Marking Effectiveness  Section 13.2 Review Questions

13.3

FUNDAMENTALS OF BIG DATA ANALYTICS A. BUSINESS PROBLEMS ADDRESSED BY BIG DATA ANALYTICS ⧫ Application Case 13.2: Top 5 Assessment Bank Achieves Single Source of the Truth  Section 13.3 Review Questions

13.4

BIG DATA TECHNOLOGIES A. MAPREDUCE B. WHY USE MAPREDUCE? C. HADOOP D. HOW DOES HADOOP WORK? E. HADOOP TECHNICAL COMPONENTS 1.

Hive 2.

Pig 3.

HBase 4.

Flume 5.

Oozie 6.

Ambari 7.

Avro 8.

Mahout 9.

Sqoop 10.

HCatalog F. HADOOP: THE PROS AND CONS 2 .

⧫ G.

NOSQL ⧫ 

Technology Insights 13.2: A Few Demystifying Facts about Hadoop Application Case 13.3: eBay’s Big Data Solution Section 13.4 Review Questions

13.5

DATA SCIENTIST A. WHERE DO DATA SCIENTISTS COME FROM? ⧫ Technology Insights 13.3: A Typical Job Post for Data Scientists ⧫ Application Case 13.4: Big Data and Analytics in Politics  Section 13.5 Review Questions

13.6

BIG DATA AND DATA WAREHOUSING A. USE CASES FOR HADOOP B. USE CASES FOR DATA WAREHOUSING C. THE GRAY AREAS (ANY ONE OF THE TWO WOULD DO THE JOB) D. COEXISTENCE OF HADOOP AND DATA WAREHOUSE  Section 13.6 Review Questions

13.7

BIG DATA VENDORS ⧫ Application Case 13.5: Dublin City Council Is Leveraging Big Data to Reduce Traffic Congestion ⧫ Technology Insights 13.4: How to Succeed with Big Data ⧫ Application Case 13.6: Creditreform Boosts Credit Rating Quality with Big Data Visual Analytics  Section 13.7 Review Questions

13.8

BIG DATA AND STREAM ANALYTICS A. STREAM ANALYTICS VERSUS PERPETUAL ANALYTICS B. CRITICAL EVENT PROCESSING C. DATA STREAM MINING  Section 13.8 Review Questions

13.9

APPLICATIONS OF STREAM ANALYTICS A. E-COMMERCE B. TELECOMMUNICATIONS ⧫ Application Case 13.7: Turning Machine-Generated Streaming Data into Valuable Business Insights C. LAW ENFORCEMENT AND CYBER SECURITY D. POWER INDUSTRY E. FINANCIAL SERVICES F. HEALTH SCIENCES G. GOVERNMENT  Section 13.9 Review Questions 3 .

Chapter Highlights Key Terms Questions for Discussion Exercises Teradata University Network (TUN) and Other Hands-On Exercises ⧫ End-of-Chapter Application Case: Discovery Health Turns Big Data into Better Healthcare  Questions for the Case References

TEACHING TIPS/ADDITIONAL INFORMATION

       

This chapter is about Big Data and its ramifications for the field of analytics. Here we are talking about new and emerging trends. Big Data technologies are considered “cutting edge,” and this should be exciting for your students to consider. Have them imagine what it would be like to be a “data scientist.” The data scientist characteristics are things they can reflect on; this will give the chapter a personal relevance.

If your course has a technical component, you may want to consider giving some examples and/or exercises utilizing the Hadoop and MapReduce APIs. These are Javabased, and could be incorporated into a curriculum that includes Java programming.

It is important for students to understand the differences and complementary potentials that exist between the traditional RDBMS data warehouse type of data and the largely unstructured (or multi-structured) data that you find with Big Data. It is also important for them to understand that the very term “Big Data” has many interpretations.

Stress the six Vs often in your lecture.

The concept of stream analytics is exciting, and presents a new paradigm to the concept of “data” for students to consider. So far, they’ve considered data analytics in the context of permanently stored data, whether in a database or in a (preserved) e-mail thread or in a (preserved) social media post or blog. With stream analytics, you are considering data that streams in and is not permanently stored, which changes the nature of the analytic processing that can take place. This is a great topic of conversation to have in the classroom.

ANSWERS TO END OF SECTION REVIEW QUESTIONS

     

Section 13.1 Review Questions 1.

What is CERN? Why is it important to the world of science?

4 .

CERN is the European Organization for Nuclear Research. It plays a leading role in fundamental studies of physics. It has been instrumental in many key global innovations and breakthrough discoveries in theoretical physics and today operates the world’s largest particle physics laboratory, home to the Large Hadron Collider (LHC).

2.

How does Large Hadron Collider work? What does it produce?

The LHS houses the world’s largest and the most sophisticated scientific instruments to study the basic constituents of matter—the fundamental particles.

These instruments include purpose-built particle accelerators and detectors.

Accelerators boost the beams of particles to very high energies before the beams are forced to collide with each other or with stationary targets. Detectors observe and record the results of these collisions, which are happening at or near the speed of light. This process provides the physicists with clues about how the particles interact, and provides insights into the fundamental laws of nature.

3.

What is the essence of the data challenge at CERN? How significant is it?

Collision events in LHC occur 40 million times per second, resulting in 15 petabytes of data produced annually at the CERN Data Centre. CERN does not have the capacity to process all of the data that it generates, and therefore relies on numerous other research centers all around the world to access and process the data. Processing all the data of their experiments requires an enormously complex distributed and heterogeneous computing and data management system. With such vast quantities of data, both structured and unstructured, information discovery is a big challenge for CERN.

4.

What was the solution? How did Big Data address the challenges?

CMS’s data management and workflow management (DMWM) created the Data Aggregation System (DAS), built on MongoDB (a Big Data management infrastructure) to provide the ability to search and aggregate information across this complex data landscape. MongoDB’s non-schema structure allowed flexible data structures to be stored and indexed.

5.

What were the results? Do you think the current solution is sufficient?

DAS is used 24 hours a day, seven days a week, by CMS physicists, data operators, and data managers at research facilities around the world. The performance of MongoDB has been outstanding, with an ability to offer a free text query system that is fast and scalable. Without help from DAS, information lookup would have taken orders of magnitude longer. The current solution is outstanding, but more improvements can be made and CERN is looking to apply big data approached beyond CMS.

5 .

Section 13.2 Review Questions 1.

Why is Big Data important? What has changed to put it in the center of the analytics world?

As more and more data becomes available in various forms and fashions, timely processing of the data with traditional means becomes impractical. The exponential growth, availability, and use of information, both structured and unstructured, brings Big Data to the center of the analytics world. Pushing the boundaries of data analytics uncovers new insights and opportunities for the use of Big Data.

2.

How do you define Big Data? Why is it difficult to define?

Big Data means different things to people with different backgrounds and interests, which is one reason it is hard to define. Traditionally, the term “Big Data” has been used to describe the massive volumes of data analyzed by huge organizations such as Google or research science projects at NASA. Big Data includes both structured and unstructured data, and it comes from everywhere: data sources include Web logs, RFID, GPS systems, sensor networks, social networks, Internet-based text documents, Internet search indexes, detailed call records, to name just a few. Big data is not just about volume, but also variety, velocity, veracity, and value proposition.

3.

Out of the Vs that are used to define Big Data, in your opinion, which one is the most important? Why? Although all of the Vs are important characteristics, value proposition is probably the most important for decision makers’ “big” data in that it contains (or has a greater potential to contain) more patterns and interesting anomalies than “small” data. Thus, by analyzing large and feature rich data, organizations can gain greater business value that they may not have otherwise. While users can detect the patterns in small data sets using simple statistical and machine-learning methods or ad hoc query and reporting tools, Big Data means “big” analytics. Big analytics means greater insight and better decisions, something that every organization needs nowadays. (Different students may have different answers.)

4.

What do you think the future of Big Data will be like? Will it lose its popularity to something else? If so, what will it be?

Big Data could evolve at a rapid pace. The buzzword “Big Data” might change to something else, but the trend toward increased computing capabilities, analytics methodologies, and data management of high volume heterogeneous information will continue. (Different students may have different answers.)

6 .

Section 13.3 Review Questions 1.

What is Big Data analytics? How does it differ from regular analytics?

Big Data analytics is analytics applied to Big Data architectures. This is a new paradigm; in order to keep up with the computational needs of Big Data, a number of new and innovative analytics computational techniques and platforms have been developed. These techniques are collectively called high-performance computing, and include in-memory analytics, in-database analytics, grid computing, and appliances. They differ from regular analytics which tend to focus on relational database technologies.

2.

What are the critical success factors for Big Data analytics?

Critical factors include a clear business need, strong and committed sponsorship, alignment between the business and IT strategies, a fact-based decision culture, a strong data infrastructure, the right analytics tools, and personnel with advanced analytic skills.

3.

What are the big challenges that one should be mindful of when considering implementation of Big Data analytics?

Traditional ways of capturing, storing, and analyzing data are not sufficient for Big Data. Major challenges are the vast amount of data volume, the need for data integration to combine data of different structures in a cost-effective manner, the need to process data quickly, data governance issues, skill availability, and solution costs.

4.

What are the common business problems addressed by Big Data analytics?

Here is a list of problems that can be addressed using Big Data analytics: • • • • • • • • • •

Process efficiency and cost reduction Brand management Revenue maximization, cross-selling, and up-selling Enhanced customer experience Churn identification, customer recruiting Improved customer service Identifying new products and market opportunities Risk management Regulatory compliance Enhanced security capabilities

Section 13.4 Review Questions

7 .

1.

What are the common characteristics of emerging Big Data technologies?

They take advantage of commodity hardware to enable scale-out, parallel processing techniques; employ nonrelational data storage capabilities in order to process unstructured and semistructured data; and apply advanced analytics and data visualization technology to Big Data to convey insights to end users.

2.

What is MapReduce? What does it do? How does it do it?

MapReduce is a programming model that allows the processing of large-scale data analysis problems to be distributed and parallelized. The MapReduce technique, popularized by Google, distributes the processing of very large multistructured data files across a large cluster of machines. High performance is achieved by breaking the processing into small units of work that can be run in parallel across the hundreds, potentially thousands, of nodes in the cluster. The map function in MapReduce breaks a problem into sub-problems, which can each be processed by single nodes in parallel. The reduce function merges (sorts, organizes, aggregates) the results from each of these nodes into the final result.

3.

What is Hadoop? How does it work?

Hadoop is an open source framework for processing, storing, and analyzing massive amounts of distributed, unstructured data. It is designed to handle petabytes and exabytes of data distributed over multiple nodes in parallel, typically commodity machines connected via the Internet. It utilizes the MapReduce framework to implement distributed parallelism. The file organization is implemented in the Hadoop Distributed File System (HDFS), which is adept at storing large volumes of unstructured and semistructured data.

This is an alternative to the traditional tables/rows/columns structure of a relational database. Data is replicated across multiple nodes, allowing for fault tolerance in the system.

4.

What are the main Hadoop components? What functions do they perform?

Major components of Hadoop are the HDFS, a Job Tracker operating on the master node, Name Nodes, Secondary Nodes, and Slave Nodes. The HDFS is the default storage layer in any given Hadoop cluster. A Name Node is a node in a Hadoop cluster that provides the client information on where in the cluster particular data is stored and if any nodes fail. Secondary nodes are backup name nodes. The Job Tracker is the node of a Hadoop cluster that initiates and coordinates MapReduce jobs or the processing of the data. Slave nodes store data and take direction to process it from the Job Tracker.

Querying for data in the distributed system is accomplished via MapReduce. The client query is handled in a Map job, which is submitted to the Job Tracker. The Job Tracker refers to the Name Node to determine which data it needs to access to complete the job and where in the cluster that data is located, then submits the 8 .

query to the relevant nodes which operate in parallel. A Name Node acts as facilitator, communicating back to the client information such as which nodes are available, where in the cluster certain data resides, and which nodes have failed.

When each node completes its task, it stores its result. The client submits a Reduce job to the Job Tracker, which then collects and aggregates the results from each of the nodes.

5.

What is NoSQL? How does it fit into the Big Data analytics picture?

NoSQL, also known as “Not Only SQL,” is a new style of database for processing large volumes of multi-structured data. Whereas Hadoop is adept at supporting large-scale, batch-style historical analysis, NoSQL databases are mostly aimed at serving up discrete data stored among large volumes of multi-structured data to end-user and automated Big Data applications. NoSQL databases trade ACID (atomicity, consistency, isolation, durability) compliance for performance and scalability.

Section 13.5 Review Questions 1.

What is a data scientist? What makes them so much in demand?

Data scientists use a combination of their business and technical skills to investigate Big Data, looking for ways to improve current business analytics practices (from descriptive to predictive and prescriptive) and hence to improve decisions for new business opportunities. One of the biggest differences between a data scientist and a business intelligence user—such as a business analyst—is that a data scientist investigates and looks for new possibilities, while a BI user analyzes existing business situations and operations. Data scientist is an emerging profession, and there is no consensus on where data scientists come from or what educational background a data scientist has to have. But there is a common understanding of what skills and qualities they are expected to possess, which involve a combination of soft and hard skills.

2.

What are the common characteristics of data scientists? Which one is the most important?

One of the most sought-out characteristics of a data scientist is expertise in both technical and business application domains. Data scientists are expected to have soft skills such as creativity, curiosity, communication/interpersonal skills, domain expertise, problem definition skills, and managerial skills as well as sound technical skills such as data manipulation, programming/hacking/scripting, and knowledge of Internet and social media/networking technologies. Data scientists are supposed to be creative and curious, and should be excellent communicators, with the ability to tell compelling stories about their data.

9 .

3.

Where do data scientists come from? What educational backgrounds do they have?

Data scientist is an emerging profession, and there is no consensus on where data scientists come from or what educational background a data scientist has to have.

Master of Science (or Ph.D.) in Computer Science, MIS, Industrial Engineering, of postgraduate analytics are common examples. But many data scientists have advanced degrees in other disciplines, like the physical or social sciences, or more specialized fields like ecology or system biology.

4.

What do you think is the path to becoming a great data scientist?

Becoming a great data scientist requires you to delve deeply into developing quantitative and technical skills, as well as interpersonal and communication skills. In addition, you will need to gain significant domain knowledge (e.g., in business). This effort will most likely require an advanced degree. It also requires a continuous thirst for knowledge and an intense curiosity; you will always be learning in this profession. In addition to meticulous analytical skills, it also requires creativity and imagination. (Students will vary in their answers to this question.)

Section 13.6 Review Questions 1.

What are the challenges facing data warehousing and Big Data? Are we witnessing the end of the data warehousing era? Why or why not?

What has changed the landscape in recent years is the variety and complexity of data, which made data warehouses incapable of keeping up. It is not the volume of the structured data but the variety and the velocity that forced the world of IT to develop a new paradigm, which we now call “Big Data.” But this does not mean the end of data warehousing. Data warehousing and RDBMS still bring many strengths that make them relevant for BI and that Big Data techniques do not currently provide.

2.

What are the use cases for Big Data and Hadoop?

In terms of its use cases, Hadoop is differentiated two ways: first, as the repository and refinery of raw data, and second, as an active archive of historical data. Hadoop, with their distributed file system and flexibility of data formats (allowing both structured and unstructured data), is advantageous when working with information commonly found on the Web, including social media, multimedia, and text. Also, because it can handle such huge volumes of data (and because storage costs are minimized due to the distributed nature of the file system), historical (archive) data can be managed easily with this approach.

10 .

3.

What are the use cases for data warehousing and RDBMS? Three main use cases for data warehousing are performance, integration, and the availability of a wide variety of BI tools. The relational data warehouse approach is quite mature, and database vendors are constantly adding new index types, partitioning, statistics, and optimizer features. This enables complex queries to be done quickly, a must for any BI application. Data warehousing, and the ETL process, provide a robust mechanism for collecting, cleaning, and integrating data.

And, it is increasingly easy for end users to create reports, graphs, and visualizations of the data.

4.

In what scenarios can Hadoop and RDBMS coexist?

There are several possible scenarios under which using a combination of Hadoop and relational DBMS-based data warehousing technologies makes sense. For example, you can use Hadoop for storing and archiving multi-structured data, with a connector to a relational DBMS that extracts required data from Hadoop for analysis by the relational DBMS. Hadoop can also be used to filter and transform multi-structural data for transporting to a data warehouse, and can also be used to analyze multi-structural data for publishing into the data warehouse environment. Combining SQL and MapReduce query functions enables data scientists to analyze both structured and unstructured data. Also, front end query tools are available for both platforms.

Section 13.7 Review Questions 1.

What is special about the Big Data vendor landscape? Who are the big players?

The Big Data vendor landscape is developing very rapidly. It is in a special period of evolution where entrepreneurial startup firms bring innovative solutions to the marketplace. Cloudera is a market leader in the Hadoop space. MapR and Hortonworks are two other Hadoop startups. DataStax is an example of a NoSQL vendor. Informatica, Pervasive Software, Syncsort, and MicroStrategy are also players. Most of the growth in the industry is with Hadoop and NoSQL distributors and analytics providers. There is still very little in terms of Big Data application vendors. Meanwhile, the next-generation data warehouse market has experienced significant consolidation. Four leading vendors in this space— Netezza, Greenplum, Vertica, and Aster Data—were acquired by IBM, EMC, HP, and Teradata, respectively. Mega-vendors Oracle and IBM also play in the Big Data space, connecting and consolidating their products with Hadoop and NoSQL engines.

11 .

2.

How do you think the Big Data vendor landscape will change in the near future?

Why? As the field matures, more and more traditional data vendors will incorporate Big Data into their architectures. We already saw something similar with the incorporation of XML data types and XPath processing engines in relational database engines. Also, the Big Data market will be increasingly cloud-based, and hosting services will include Big Data data storage options, along with the traditional MySql and SqlServer options. Vendors providing Big Data applications and services, for example in the finance domain or for scientific purposes, will begin to proliferate. (Different students will have different answers.)

3.

What is the role of visual analytics in the world of Big Data?

Visual analytics help organizations uncover trends, relationships, and anomalies by visually shifting through very large quantities of data. Many vendors are developing visual analytics offerings, which have traditionally applied to structured data warehouse environments (relational and multidimensional), for the Big Data space. To be successful, a visual analytics application must allow for the coexistence and integration of relational and multistructured data.

Section 13.8 Review Questions 1.

What is a stream (in the Big Data world)?

A stream can be thought of as an unbounded flow or sequence of data elements, arriving continuously at high velocity. Streams often cannot be efficiently or effectively stored for subsequent processing; thus Big Data concerns about Velocity (one of the six Vs) are especially prevalent when dealing with streams.

Examples of data streams include sensor data, computer network traffic, phone conversations, ATM transactions, web searches, and financial data.

2.

What are the motivations for stream analytics?

In situations where data streams in rapidly and continuously, traditional analytics approaches that work with previously accumulated data (i.e., data at arrest) often either arrive at the wrong decisions because of using too much out-of-context data, or they arrive at the correct decisions but too late to be of any use to the organization. Therefore it is critical for a number of business situations to analyze the data soon after it is created and/or as soon as it is streamed into the analytics system. It is no longer feasible to “store everything.” Otherwise, analytics will either arrive at the wrong decisions because of using too much out-of-context data, or at the correct decisions but too late to be of any use to the organization.

Therefore it is critical for a number of business situations to analyze the data as soon as it is streamed into the analytics system.

12 .

3.

What is stream analytics? How does it differ from regular analytics?

Stream analytics is the process of extracting actionable information from continuously flowing/streaming data. It is also sometimes called “data in-motion analytics” or “real-time data analytics.” It differs from regular analytics in that it deals with high velocity (and transient) data streams instead of more permanent data stores like databases, files, or web pages.

4.

What is critical event processing? How does it relate to stream analytics?

Critical event processing is a method of capturing, tracking, and analyzing streams of data to detect events (out of normal happenings) of certain types that are worthy of the effort. It involves combining data from multiple sources to infer events or patterns of interest. An event may also be defined generically as a “change of state,” which may be detected as a measurement exceeding a predefined threshold of time, temperature, or some other value. This applies to stream analytics because the events are happening in real time.

5.

Define data stream mining. What are the additional challenges that are posed?

Data stream mining is the process of extracting novel patterns and knowledge structures from continuous, rapid data records. Processing data streams, as opposed to more permanent data storages, is a challenge. Traditional data mining techniques can process data recursively and repetitively because the data is permanent. By contrast, a data stream is a continuous flow of ordered sequence of instances that can only be read once and must be processed immediately as they come in.

Section 13.9 Review Questions 1.

What are the most fruitful industries for stream analytics?

Many industries can benefit from stream analytics. Some prominent examples include e-commerce, telecommunications, law enforcement, cyber security, the power industry, health sciences, and the government.

2.

How can stream analytics be used in e-commerce?

Companies such as Amazon and eBay use stream analytics to analyze customer behavior in real time. Every page visit, every product looked at, every search conducted, and every click made is recorded and analyzed to maximize the value gained from a user’s visit. Behind the scenes, advanced analytics are crunching the real-time data coming from our clicks, and the clicks of thousands of others, to “understand” what it is that we are interested in (in some cases, even we do not know that) and make the most of that information by creative offerings.

13 .

3.

In addition to what is listed in this section, can you think of other industries and/or application areas where stream analytics can be used?

Stream analytics could be of great benefit to any industry that faces an influx of relevant real-time data and needs to make quick decisions. One example is the news industry. By rapidly sifting through data streaming in, a news organization can recognize “newsworthy” themes (i.e., critical events). Another benefit would be for weather tracking in order to better predict tornados or other natural disasters. (Different students will have different answers.)

4.

Compared to regular analytics, do you think stream analytics will have more (or fewer) use cases in the era of Big Data analytics? Why? Stream analytics can be thought of as a subset of analytics in general, just like “regular” analytics. The question is, what does “regular” mean? Regular analytics may refer to traditional data warehousing approaches, which does constrain the types of data sources and hence the use cases. Or, “regular” may mean analytics on any type of permanent stored architecture (as opposed to transient streams). In this case, you have more use cases for “regular” (including Big Data) than in the previous definition. In either case, there will probably be plenty of times when “regular” use cases will continue to play a role, even in the era of Big Data analytics. (Different students will have different answers.)

ANSWERS TO APPLICATION CASE QUESTIONS FOR DISCUSSION

 

Application Case 13.1: Big Data Analytics Helps Luxottica Improve Its Marketing Effectiveness 1.

What does Big Data mean to Luxottica?

For Luxottica, Big Data includes everything they can find about their customer interactions (in the form of transactions, click streams, product reviews, and social media postings). They see this as constituting a massive source of business intelligence for potential product, marketing, and sales opportunities.

2.

What were their main challenges?

Because Luxottica outsourced both data storage and promotional campaign development and management, there was a disconnect between data analytics and marketing execution. Their competitive posture and strategic growth initiatives were compromised for lack of an individualized view of their customers and an inability to act decisively and consistently on the different types of information generated by each retail channel.

14 .

3.

What was the proposed solution, and the obtained results?

Luxottica deployed the Customer Intelligence Appliance (CIA) from IBM Business Partner Aginity LLC. This product is built on IBM PureData System for Analytics. This solution helps Luxottica highly segment customer behavior and provide a platform and smart database for marketing execution systems, such as campaign management, e-mail services, and other forms of direct marketing.

Anticipated benefits include a 10% improvement in marketing effectiveness, identifying the highest valued customers, and the ability to target customers based on preference and history.

Application Case 13.2: Top 5 Investment Bank Achieves Single Source of the Truth 1.

How can Big Data benefit large-scale trading banks?

Big Data can potentially handle the high volume, high variability, continuously streaming data that trading banks need to deal with. Traditional relational databases are often unable to keep up with the data demand.

2.

How did MarkLogic infrastructure help ease the leveraging of Big Data?

MarkLogic was able to meet two needs: (1) upgrading existing Oracle and Sybase platforms, and (2) compliance with regulatory requirements. Their solution provided better performance, scalability, and faster development for future requirements than competitors. Most importantly, MarkLogic was able to eliminate the need for replicated database servers by providing a single server providing timely access to the data.

3.

What were the challenges, the proposed solution, and the obtained results?

The Bank’s legacy system was built on relational database technology. As data volumes and variability increased, the legacy system was not fast enough to respond to growing business needs and requirements. It was unable to deliver real-time alerts to manage market and counterparty credit positions in the desired timeframe. Big data offered the scalability to address this problem. The benefits included a new alert feature, less downtime for maintenance, much faster capacity to process complex changes, and reduced operations costs.

Application Case 13.3: eBay’s Big Data Solution 1.

Why is Big Data a big deal for eBay?

15 .

eBay is the world’s largest online marketplace, and its success requires the ability to turn the enormous volumes of data it generates into useful insights for customers. Big Data is essential for this effort.

2.

What were the challenges, the proposed solution, and the obtained results? eBay was experiencing explosive data growth and needed a solution that did not have the typical bottlenecks, scalability issues, and transactional constraints associated with common relational database approaches. The company also needed to perform rapid analysis on a broad assortment of the structured and unstructured data it captured. eBay’s solution includes NoSQL via Apache Cassandra and DataStax Enterprise. It also uses integrated Apache Hadoop analytics that come with DataStax. The solution incorporates a scale-out architecture that enables eBay to deploy multiple DataStax Enterprise clusters across several different data centers using commodity hardware. Now, eBay can more cost effectively process massive amounts of data at very high speeds. The new architecture serves a wide variety of new use cases, and its reliability and fault tolerance has been greatly enhanced.

3.

Can you think of other e-commerce businesses that may have Big Data challenges comparable to that of eBay? Any large company with a significant online presence will have similar challenges as eBay. Amazon and Walmart are two of the largest.

Application Case 13.4: Big Data and Analytics in Politics 1.

What is the role of analytics and Big Data in modern-day politics?

Big Data and analytics have a lot to offer to modern-day politics. The main characteristics of Big Data, namely volume, variety, and velocity (the three Vs), readily apply to the kind of data that is used for political campaigns. Big Data Analytics can help predict election outcomes as well as targeting potential voters and donors, and have become a critical part of political campaigns.

2.

Do you think Big Data Analytics could change the outcome of an election?

It may well have changed the outcome of the 2008 and 2012 elections. For example, in 2012, Barack Obama had a data advantage and the depth and breadth of his campaign’s digital operation, from political and demographic data mining to voter sentiment and behavioral analysis, reached beyond anything politics had ever seen. So there was a significant difference in expertise and comfort level with modern technology between the two parties. The usage and expertise gap between the party lines may disappear over time, and this will even the playing field. But new data regarding voter sympathies may itself change the thinking and 16 .

ideological directions of the major parties. This in itself could shift election outcomes. (Answers will vary by student.)

3.

What do you think are the challenges, the potential solution, and the probable results of the use of Big Data Analytics in politics?

One of the big challenges is to ensure some sort of reliability in news media coverage of election issues. For example, most people, including the so-called political experts (who often rely on gut feelings and experiences), thought the 2012 presidential election would be very close. They were wrong. On the other hand, data analysts, using data-driven analytical models, predicted that Barack Obama would win easily with close to 99 percent certainty. The results will be an ever-increasing use of Big Data analytics in politics, both by the parties and candidates themselves and by the news media and analysts who cover them.

Application Case 13.5: Dublin City Council Is Leveraging Big Data to Reduce Traffic Congestion 1.

Is there a strong case to make for large cities to use Big Data Analytics and related information technologies? Identify and discuss examples of what can be done with analytics beyond what is portrayed in this application case.

For the Dublin case, Big Data Analytics were used primarily to ease traffic problems and better understand the traffic network. Municipalities could use these technologies for many other governmental tasks. For example, providing social services is a very complex and difficult process that can be benefit from Big Data technologies. Other areas include tax collection, sanitation services, environmental management, crime prevention, and management of police and fire departments.

2.

How can big data analytics help ease the traffic problem in large cities?

They can help get a better sense of the “traffic health” by identifying traffic congestion in its early stages. By integrating geospatial data from buses and data on bus timetables into a central geographic information system, you can create a digital map of the city. Then, using the dashboard screen, operators can drill down to see the number of buses that are on time or delayed on each route. With big data analytics, users can produce detailed reports on areas of the network where buses are frequently delayed, and take prompt action to ease congestion. Data and analytics can also assist with future planning of roads, infrastructure, and public transportation in order to further ease traffic problems.

3.

What were the challenges Dublin City was facing; what were the proposed solution, initial results, and future plans?

17 .

The major problem was the difficulty in getting a good picture of traffic in the city from a high-level perspective. The proposed solution was to team up with IBM Research, and especially their Smarter Cities Technologies Centre. Using IBM InfoSphere Streams and mapping software, IBM researchers created a digital map of the city, overlaid with the real-time positions of Dublin’s 1,000 buses.

This gave operators the ability to see the system as a whole instead of just individual corridors. The new system gave insight to the operators and managers.

They could now start answering questions such as: “Are the bus lane start times correct?” and “Where do we need to add additional bus lanes and bus-only traffic signals?” For the future, the Dublin City Council and IBM plan to enhance the system by incorporating meteorological data, under-road sensors, and bicycleusage data into their predictive analytics.

Application Case 13.6: Creditreform Boosts Credit Rating Quality with Big Data Visual Analytics 1.

How did Creditreform boost credit rating quality with Big Data and visual analytics?

Using Big Data and visual analytics, executives can make quicker, better decisions with instant access, via PC or tablet, to insights based on the latest data.

Creditreform employees can use the software to make smart decisions without needing extensive training. This makes Creditreform better at providing the highest quality financial information and credit ratings to its client businesses.

2.

What were the challenges, proposed solution, and initial results?

Creditreform makes more than 24 million credit reports from 26 countries in Europe and from China that are available around the clock. Creditreform wants to maintain its leadership and widen its market lead. Big data and visual analytics are seen as principal technologies to help with this effort. Creditreform decided to use SAS Visual Analytics to simplify the analytics process. SAS Visual Analytics is a high-performance, in-memory solution for exploring massive amounts of data very quickly. This made it faster and easier for their analysts to detect correlations in the business data, which improved the quality and forecasting accuracy of their credit ratings.

Application Case 13.7: Turning Machine-Generated Streaming Data into Valuable Business Insights 1.

Why is stream analytics becoming more popular?

18 .

Stream analytics is becoming increasingly more popular for two main reasons.

First, time-to-action has become an ever decreasing value, and second, we have the technological means to capture and process the data while it is being created.

2.

How did the telecommunications company in this case use stream analytics for better business outcomes? What additional benefits can you foresee?

Mainly they used stream analytics to improve their service delivery in the following areas: application troubleshooting, operations, compliance, and security. For application troubleshooting, stream analytics helps to quickly diagnose the causes of 404 errors encountered by users on their Web site. In terms of operations, stream analytics helps to avoid critical downtime by anticipating potential infrastructure failures before they happen. Stream analytics is helping the company’s audit process, which assists with meeting compliance requirements. Finally, stream analytics enhances the company’s ability to detect system vulnerabilities and attacks.

3.

What were the challenges, proposed solution, and initial results?

This telecommunication company encounters over a billion daily events running on a distributed hardware/software infrastructure supporting millions of cable, online, and interactive media customers. It was overwhelming to gather and view this data in one place, and to perform any diagnostics, or hone in on the real-time intelligence that lives in the machine-generated data. Using time-consuming and error-prone traditional search methods, the company’s roster of experts would shuffle through mountains of data to uncover issues threatening data integrity, system stability, and applications performance. They needed a better way to do this. They decided to work with Splunk, one of the leading stream analytics service providers. The insights gained from the analysis and information provided

by Splunk helped improve in several areas: application troubleshooting, operations, compliance, and security.

ANSWERS TO END OF CHAPTER QUESTIONS FOR DISCUSSION 1.

  

What is Big Data? Why is it important? Where does Big Data come from?

Traditionally, the term “Big Data” has been used to describe the massive volumes of data analyzed by huge organizations like Google or research science projects at NASA. But for most businesses, it’s a relative term: “Big” depends on an organization’s size. In general, Big Data exceeds the reach of commonly used hardware environments and/or capabilities of software tools to capture, manage, and process it within a tolerable time span for its user population. Big Data has become a popular term to describe the exponential growth, availability, and use of information, both structured and unstructured. Big data is important because it is 19 .

there and because it is rich in information and insight that, if effectively tapped, can lead to better business decisions and improved company performance. Much has been written on the Big Data trend and how it can serve as the basis for innovation, differentiation, and growth. Big Data includes both structured and unstructured data, and it comes from everywhere: data sources include Web logs, RFID, GPS systems, sensor networks, social networks, Internet-based text documents, Internet search indexes, and detailed call records, to name just a few.

Big data is not just about volume, but also variety, velocity, veracity, and value proposition.

2.

What do you think the future of Big Data will be? Will it lose its popularity to something else? If so, what will it be?

Big Data could evolve at a rapid pace. The buzzword “Big Data” might change to something else, but the trend toward increased computing capabilities, analytics methodologies, and data management of high volume heterogeneous information will continue. (Different students may have different answers.)

3.

What is Big Data analytics? How does it differ from regular analytics?

Big Data analytics is analytics applied to Big Data architectures. This is a new paradigm; in order to keep up with the computational needs of Big Data, a number of new and innovative analytics computational techniques and platforms have been developed. These techniques are collectively called high-performance computing, and include in-memory analytics, in-database analytics, grid computing, and appliances. They differ from regular analytics which tend to focus on relational database technologies.

4.

What are the critical success factors for Big Data analytics?

Critical factors include a clear business need, strong and committed sponsorship, alignment between the business and IT strategies, a fact-based decision culture, a strong data infrastructure, the right analytics tools, and personnel with advanced analytic skills.

5.

What are the big challenges that one should be mindful of when considering implementation of Big Data analytics?

Traditional ways of capturing, storing, and analyzing data are not sufficient for Big Data. Major challenges are the vast amount of data volume, the need for data integration to combine data of different structures in a cost-effective manner, the need to process data quickly, data governance issues, skill availability, and solution costs.

6.

What are the common business problems addressed by Big Data analytics?

Here is a list of problems that can be addressed using Big Data analytics:

20 .

• • • • • • • • • • 7.

Process efficiency and cost reduction Brand management Revenue maximization, cross-selling, and up-selling Enhanced customer experience Churn identification, customer recruiting Improved customer service Identifying new products and market opportunities Risk management Regulatory compliance Enhanced security capabilities

What is a data scientist? What makes them so much in demand?

Data scientists use a combination of their business and technical skills to investigate Big Data, looking for ways to improve current business analytics practices (from descriptive to predictive and prescriptive) and hence to improve decisions for new business opportunities. One of the biggest differences between a data scientist and a business intelligence user—such as a business analyst—is that a data scientist investigates and looks for new possibilities, while a BI user analyzes existing business situations and operations. Data scientist is an emerging profession, and there is no consensus on where data scientists come from or what educational background a data scientist has to have. But there is a common understanding of what skills and qualities they are expected to possess, which involve a combination of soft and hard skills.

8.

What are the common characteristics of data scientists? Which one is the most important?

One of the most sought-out characteristics of a data scientist is expertise in both technical and business application domains. Data scientists are expected to have soft skills such as creativity, curiosity, communication/interpersonal skills, domain expertise, problem definition skills, and managerial skills as well as sound technical skills such as data manipulation, programming/hacking/scripting, and knowledge of Internet and social media/networking technologies. Data scientists are supposed to be creative and curious, and should be excellent communicators, with the ability to tell compelling stories about their data.

9.

In the era of Big Data, are we about to witness the end of data warehousing?

Why? What has changed the landscape in recent years is the variety and complexity of data, which made data warehouses incapable of keeping up. It is not the volume of the structured data but the variety and the velocity that forced the world of IT to develop a new paradigm, which we now call “Big Data.” But this does not mean the end of data warehousing. Data warehousing and RDBMS still bring

21 .

many strengths that make them relevant for BI and that Big Data techniques do not currently provide.

10.

What are the use cases for Big Data/Hadoop and data warehousing/RDBMS? In terms of its use cases, Hadoop is differentiated two ways: first, as the repository and refinery of raw data, and second, as an active archive of historical data. Hadoop, with their distributed file system and flexibility of data formats (allowing both structured and unstructured data), is advantageous when working with information commonly found on the Web, including social media, multimedia, and text. Also, because it can handle such huge volumes of data (and because storage costs are minimized due to the distributed nature of the file system), historical (archive) data can be managed easily with this approach.

Three main use cases for data warehousing are performance, integration, and the availability of a wide variety of BI tools. The relational data warehouse approach is quite mature, and database vendors are constantly adding new index types, partitioning, statistics, and optimizer features. This enables complex queries to be done quickly, a must for any BI application. Data warehousing, and the ETL process, provide a robust mechanism for collecting, cleaning, and integrating data.

And, it is increasingly easy for end users to create reports, graphs, and visualizations of the data.

11.

What is stream analytics? How does it differ from regular analytics?

Stream analytics is the process of extracting actionable information from continuously flowing/streaming data. It is also sometimes called “data in-motion analytics” or “real-time data analytics.” It differs from regular analytics in that it deals with high velocity (and transient) data streams instead of more permanent data stores like databases, files, or web pages).

12.

What are the most fruitful industries for stream analytics? What is common to those industries?

Many industries can benefit from stream analytics. Some prominent examples include e-commerce, telecommunications, law enforcement, cyber security, the power industry, health sciences, and the government.

13.

Compared to regular analytics, do you think stream analytics will have more (or fewer) use cases in the era of Big Data analytics? Why? Stream analytics can be thought of as a subset of analytics in general, just like “regular” analytics. The question is, what does “regular” mean? Regular analytics may refer to traditional data warehousing approaches, which does constrain the types of data sources and hence the use cases. Or, “regular” may mean analytics on any type of permanent stored architecture (as opposed to transient streams). In this case, you have more use cases for “regular” (including Big Data) than in the previous definition. In either case, there will probably be plenty of times when 22 .

“regular” use cases will continue to play a role, even in the era of Big Data analytics. (Different students will have different answers.)

ANSWERS TO END OF CHAPTER APPLICATION CASE QUESTIONS 1.

 

How big is Big Data for Discovery Health?

Discovery Health serves 2.7 million members, and its claims system generates a million new rows of data each day. For analytics purposes, they want to use three years of data, which comes to over a billion rows of claims data alone.

2.

What big data sources did Discovery Health use for their analytic solutions?

The data sources include members’ claims data, pathology results, members’ questionnaires, data from pharmacies, hospital admissions, and billing data, to name a few. Looking to the future, Discovery is also starting to analyze unstructured data, such as text-based surveys and comments from online feedback forms.

3.

What were the main data/analytics challenges Discovery Health was facing?

The challenge faced by Discovery Health was similar to that faced by any company operating in a Big Data environment. Given the huge amount of data that Discovery Health was generating, how could they identify the key insights that their business and their members’ health depend on? To find the needles of vital information in the big data haystack, the company not only needed a sophisticated data-mining and predictive modeling solution, but also an analytics infrastructure with the power to deliver results at the speed of business.

4.

What were the main solutions they have produced?

To achieve this transformation in its analytics capabilities, Discovery worked with BITanium, an IBM Business Partner with deep expertise in operational deployments of advanced analytics technologies. The solution included use of IBM’s SSPS and PureData System for Analytics.

5.

What were the initial results/benefits? What do you think will be the future of Big Data analytics at Discovery?

Discovery Health is now able to run three years’ worth of data for their 2.7 million members through complex statistical models to deliver actionable insights in a matter of minutes. This helps them to better predict and prevent health risks, and to identify and eliminate fraud. The results have been transformative. From an analytics perspective, the speed of the solution gives Discovery more scope to 23 .

experiment and optimize its models. From a broader business perspective, the combination of SPSS and PureData technologies gives Discovery the ability to put actionable data in the hands of its decision makers faster.

24 .

Chapter 14: Business Analytics: Emerging Trends and Future Impacts

Learning Objectives for Chapter 14 1. Explore some of the emerging technologies that may impact analytics, BI, and decision support 2. Describe how geospatial and location-based analytics are assisting organizations 3. Describe how analytics are powering consumer applications and creating a new opportunity for entrepreneurship for analytics 4. Describe the potential of cloud computing in business intelligence 5. Understand Web 2.0 and its characteristics as related to analytics 6. Describe organizational impacts of analytics applications 7. List and describe the major ethical and legal issues of analytics implementation 8. Understand the analytics ecosystem to get a sense of the various types of players in the analytics industry and how one can work in a variety of roles CHAPTER OVERVIEW This chapter introduces several emerging technologies that are likely to have major impacts on the development and use of business intelligence applications. Many other interesting technologies are also emerging, but we have focused on some trends that have already been realized and others that are about to impact analytics further. Using a crystal ball is always a risky proposition, but this chapter provides a framework for analysis of emerging trends. We introduce and explain some emerging technologies and explore their current applications. We then discuss the organizational, personal, legal, ethical, and societal impacts of support systems that may affect their implementation. We conclude with a description of the analytics ecosystem. This section should help readers appreciate different career possibilities within the realm of analytics. This chapter contains the following sections:

CHAPTER OUTLINE 14.1

OPENING VIGNETTE: OKLAHOMA GAS AND ELECTRIC EMPLOYS ANALYTICS TO PROMOTE SMART ENERGY USE  Questions for the Opening Vignette 1 .

A.

WHAT WE CAN LEARN FROM THIS VIGNETTE

14.2

LOCATION-BASED ANALYTICS FOR ORGANIZATIONS A. GEOSPATIAL ANALYTICS ⧫ Application Case 14.1: Great Clips Employs Spatial Analytics to Shave Time in Location Decisions B. REAL-TIME LOCATION INTELLIGENCE ⧫ Application Case 14.2: Quiznos Targets Customers for Its Sandwiches  Section 14.2 Review Questions

14.3

ANALYTICS APPLICATIONS FOR CONSUMERS ⧫ Application Case 14.3: A Life Coach in Your Pocket  Section 14.3 Review Questions

14.4

RECOMMENDATION ENGINES  Section 14.4 Review Questions

14.5

WEB 2.0 AND ONLINE SOCIAL NETWORKING A. REPRESENTATIVE CHARACTERISTICS OF WEB 2.0 B. SOCIAL NETWORKING C. A DEFINITION AND BASIC INFORMATION D. IMPLICATIONS OF BUSINESS AND ENTERPRISE SOCIAL NETWORKS 1.

Using Twitter to Get a Pulse of the Market  Section 14.5 Review Questions

14.6

CLOUD COMPUTING AND BI A. SERVICE-ORIENTED DSS B. DATA-AS-A-SERVICE (DAAS) C. INFORMATION-AS-A-SERVICE (INFORMATION ON DEMAND) (IAAS) D. ANALYTICS-AS-A-SERVICE (AAAS)  Section 14.6 Review Questions

14.7

IMPACTS OF ANALYTICS IN ORGANIZATIONS: AN OVERVIEW A. NEW ORGANIZATIONAL UNITS B. RESTRUCTURING BUSINESS PROCESSES AND VIRTUAL TEAMS C. THE IMPACTS OF ADS SYSTEMS D. JOB SATISFACTION E. JOB STRESS AND ANXIETY F. ANALYTICS’ IMPACT ON MANAGERS’ ACTIVITIES AND THEIR PERFORMANCE  Section 14.7 Review Questions

14.8

ISSUES OF LEGALITY, PRIVACY, AND ETHICS 2 .

A. B.

C. D.

14.9

LEGAL ISSUES PRIVACY 1.

Collecting Information about Individuals 2.

Mobile User Privacy 3.

Homeland Security and Individual Privacy RECENT TECHNOLOGY ISSUES IN PRIVACY AND ANALYTICS ETHICS IN DECISION MAKING AND SUPPORT  Section 14.8 Review Questions

AN OVERVIEW OF THE ANALYTICS ECOSYSTEM A. ANALYTICS INDUSTRY CLUSTERS B. DATA INFRASTRUCTURE PROVIDERS C. DATA WAREHOUSE INDUSTRY D. MIDDLEWARE INDUSTRY E. DATA AGGREGATORS/DISTRIBUTORS F. ANALYTICS-FOCUSED SOFTWARE DEVELOPERS G. REPORTING/ANALYTICS H. PREDICTIVE ANALYTICS I. PRESCRIPTIVE ANALYTICS J. APPLICATION DEVELOPERS OR SYSTEM INTEGRATORS: INDUSTRY SPECIFIC OR GENERAL K. ANALYTICS USER ORGANIZATIONS L. ANALYTICS INDUSTRY ANALYSTS AND INFLUENCERS M. ACADEMIC PROVIDERS AND CERTIFICATION AGENCIES  Section 14.9 Review Questions

Chapter Highlights Key Terms Questions for Discussion Exercises Teradata University Network (TUN) and Other Hands-On Exercises ⧫ End of Chapter Application Case: Southern States Cooperative Optimizes Its Catalog Campaign  Questions for the Case References

TEACHING TIPS/ADDITIONAL INFORMATION

       

How to wrap up your course is a personal decision. A book can support your approach, but specific content is less important at this point in the course than it was earlier.

The primary purpose of this chapter is to introduce several emerging technologies that will provide new opportunities for application and extension of business intelligence and analytics techniques. In particular, the impact of cloud and mobile computing are prominently discussed, as well as location-based geospatial analytics technologies.

3 .

Another main theme is the increasing use of analytics for consumers, including recommendation engines. Students will find this relevant, since much of their on-line lives involve their roles as consumers, so they have real-world experience with these engines.

This chapter also explores the individual, organizational, and societal impacts of these technologies, especially the ethical, legal, and privacy concerns related to analytics implementation. After describing many of the emerging technologies or application domains, we will focus on organizational issues.

Also, many of the students in the typical undergraduate BI course will start their professional careers a few weeks after you cover this chapter. They care deeply about how Internet and mobile-based analytics will affect them and will enjoy an active discussion of this topic. (Once the graduating seniors break the ice, students who don’t fit that category will chime in, too.)

ANSWERS TO END OF SECTION REVIEW QUESTIONS

     

Section 14.1 Review Questions 1. Why perform consumer analytics?

Consumer analytics helps a company’s customers make better purchasing and usage decisions. OG&E is using it to help conserve on energy usage and ultimately to delay their need to build new fossil fuel generation plants. This helps both OG&E (by reducing costs), its customers (by reducing their energy costs), and the environment as a whole (by reducing pollution from fossil fuels).

2. What is meant by dynamic segmentation?

Dynamic segmentation refers to real-time or near-real-time customer segmentation analytics that will enhance their understanding about individuals’ responses to the price signals and identify the best customers to be targeted with specific marketing campaigns.

3. How does geospatial mapping help OG&E? Using geospatial mapping and visual analytics, OG&E views a near–real-time version of data about its energy-efficient prospects spread over geographic areas and comes up with marketing initiatives that are most suitable for these customers. Geospatial mapping gives OG&E an easy way to narrow down to the specific customers in a geographic region based on their meter usage. In addition, OG&E can find noncommunicating smart meters, track the outages, and weather overlay of their services.

4. What types of incentives might the consumers respond to in changing their energy use?

4 .

OG&E has started working on consumer-oriented efficiency programs to shift the customer’s usage out of peak demand cycles. OG&E is targeting customers with its smart hours plan. This plan encourages customers to choose a variety of rate options sent via phone, text, or e-mail. These rate options offer attractive summer rates for all other hours apart from the peak hours of 2 p.m. to 7 p.m. OG&E is making an investment in customers by supplying a communicating thermostat that will respond to the price signals sent by OG&E and help customers in managing their utility consumption. OG&E also educates its customers on their usage habits by providing 5-minute interval data every 15 minutes to the demand-responsive customers.

Section 14.2 Review Questions 1. How does traditional analytics make use of location-based data?

Traditional analytics produce visual maps that are geographically mapped and based on the traditional location data, usually grouped by the postal codes. The use of postal codes to represent the data is a somewhat static approach for achieving a higher level view of things.

2. How can geocoded locations assist in better decision making?

They help the user in understanding “true location-based” impacts, and allow them to view at higher granularities than that offered by the traditional postal code aggregations. Addition of location components based on latitudinal and longitudinal attributes to the traditional analytical techniques enables organizations to add a new dimension of “where” to their traditional business analyses, which currently answer questions of “who,” “what,” “when,” and “how much.” By integrating information about the location with other critical business data, organizations are now creating location intelligence (LI).

3. What is the value provided by geospatial analytics?

Geospatial analysis gives organizations a broader perspective and aids in decision making. Location intelligence (LI) is enabling organizations to gain critical insights and make better decisions by optimizing important processes and applications. By incorporating demographic details into locations, retailers can determine how sales vary by population level and proximity to other competitors; they can assess the demand and efficiency of supply chain operations. Consumer product companies can identify the specific needs of the customers and customer complaint locations, and easily trace them back to the products. Sales reps can better target their prospects by analyzing their geography.

4. Explore the use of geospatial analytics further by investigating its use across various sectors like government census tracking, consumer marketing, and so forth.

5 .

Students’ answers will vary.

Section 14.3 Review Questions 1. What are the various options that CabSense provides to users?

CabSense provides an interactive map based on current user location obtained from the mobile phone’s GPS locator to find the best street corners for finding an open cab.

It also provides a radar view that automatically points the right direction toward the best street corner. The application also allows users to plan in advance, set up date and time of travel, and view the best corners for finding a taxi. Furthermore, CabSense distinguishes New York’s Yellow Cab services from the for-hire vehicles and readily prompts the users with relevant details of private service providers that can be used in case no Yellow Cabs are available.

2. Explore more transportation applications that may employ location-based analytics.

Another app that is mentioned in the text is one deployed in Pittsburgh, Pennsylvania, and developed in collaboration with Carnegie Mellon University. This app, called ParkPGH, includes predictive capabilities to estimate parking availability. It calculates the number of spaces available in downtown Pittsburgh parking lots and garages. (Student answers will vary.)

3. Briefly describe how the data are used to create profiles of users.

Profiles are often built based on users’ behavior. The behavior is tracked by the system based on user clicks, mobile locations, purchase histories, email connections, social network actions, etc. In this way, companies can construct a profile of the user’s habits, interests, spending patterns, and favorite locations.

4. What other applications can you imagine if you were able to access cell phone location data? Do a search on location-enabled services.

Students’ answers will vary.

Section 14.4 Review Questions 1. List the types of approaches used in recommendation engines.

Two basic approaches that are employed in the development of recommendation systems are collaborative filtering and content filtering.

2. How do the two approaches differ?

6 .

In collaborative filtering, the recommendation system is built based on the individual user’s past behavior by keeping track of the previous history of all purchased items.

Collaborative filtering involves aggregating the user-item profiles to generate user ratings matrices, and can take a user-based approach in which the users take the main role. Collaborative filtering systems require huge amounts of existing data on useritem preferences to make appropriate recommendations, and so are subject to the “cold start” problem.

By contrast, content filtering approaches first consider specifications and characteristics of items. Then content-based individual user profiles are built to store the information about the characteristics of specific items that the user has rated in the past. Content-based filtering involves using information tags or keywords in fetching detailed information about item characteristics and restricts this process to a single user, unlike collaborative filtering, which looks for similarities between various user profiles.

In short, collaborative filtering makes recommendations based on what other consumers like you chose, whereas content filtering makes recommendations based on similar items you chose in the past.

3. Can you identify specific sites that may use one or the other type of recommendation system?

Amazon, Facebook, and LinkedIn all use collaborative filtering. Pandora uses content-based filtering.

Section 14.5 Review Questions 1. Define Web 2.0.

Web 2.0 is the popular term for describing advanced Web technologies and applications, including blogs, wikis, RSS, mashups, user-generated content, and social networks. A major objective of Web 2.0 is to enhance creativity, information sharing, and collaboration.

2. List the major characteristics of Web 2.0.

The following are representative characteristics of the Web 2.0 environment: • • •

The ability to tap into the collective intelligence of users. The more users who contribute, the more popular and valuable a Web 2.0 site becomes.

Data is made available in new or never-intended ways. Web 2.0 data can be remixed or “mashed up,” often through Web service interfaces, much the way a dance-club DJ mixes music.

Web 2.0 relies on user-generated and user-controlled content and data.

7 .

• • • • • •

Lightweight programming techniques and tools let nearly anyone act as a Web site developer.

The virtual elimination of software-upgrade cycles makes everything a perpetual beta or work-in-progress and allows rapid prototyping, using the Web as an application development platform.

Users can access applications entirely through a browser.

An architecture of participation and digital democracy encourages users to add value to the application as they use it.

A major emphasis on social networks and computing.

Strong support of information sharing and collaboration.

3. What new business model has emerged from Web 2.0?

New business models stress collaboration with customers, partners, and suppliers, as well as among internal users. Businesses are also much more data-driven utilizing metrics on behaviors and sentiments of the participants. Agility (perpetual beta) and global reach are other Web 2.0 contributors to the evolving business models.

4. Define social network.

A social network is a place where people create their own space, or homepage, on which they write blogs (Web logs); post pictures, videos, or music; share ideas; and link to other Web locations they find interesting. In addition, members of social networks can tag the content they create and post it with keywords they choose themselves, which makes the content searchable.

5. List some major social network sites.

Facebook, LinkedIn, Orkut, Google+.

Section 14.6 Review Questions 1. Define cloud computing. How does it relate to PaaS, SaaS, and IaaS? Cloud computing offers the possibility of using software, hardware, platform, and infrastructure, all on a service-subscription basis. Cloud computing enables a more scalable investment on the part of a user. Like PaaS, etc., cloud computing offers organizations the latest technologies without significant upfront investment.

In some ways, cloud computing is a new name for many previous related trends: utility computing, application service provider grid computing, on-demand computing, software as a service (SaaS), and even older centralized computing with dumb terminals. But the term cloud computing originates from a reference to the Internet as a “cloud” and represents an evolution of all previous shared/centralized computing trends.

8 .

2. Give examples of companies offering cloud services.

Companies offering such services include 1010data, LogiXML, and Lucid Era. These companies offer feature extract, transform, and load capabilities as well as advanced data analysis tools. Other companies, such as Elastra and Rightscale, offer dashboard and data management tools that follow the SaaS and DaaS models.

3. How does cloud computing affect business intelligence?

Cloud-computing-based BI services offer organizations the latest technologies without significant upfront investment.

4. What are the three service models that provide the foundation to service-oriented DSS? The three service models are data-as-a-service (DaaS), information-as-a-service (IaaS), and analytics-as-a-service (AaaS).

5. How does DaaS change the way data is handled?

In the DaaS model, the actual platform on which the data resides doesn’t matter. Data can reside in a local computer or in a server at a server farm inside a cloud-computing environment. With DaaS, any business process can access data wherever it resides.

Customers can move quickly thanks to the simplicity of the data access and the fact that they don’t need extensive knowledge of the underlying data.

6. What is MaaS? What does it offer to businesses?

MaaS stands for models-as-services. This subset of IaaS provides a collection of industry-specific business processes, reports, dashboards, and other service models for key industries (e.g., banking, insurance, and financial markets) to accelerate enterprise business initiatives for business process optimization and multi-channel transformation.

7. Why is AaaS cost effective?

AaaS in the cloud has economies of scale and scope by providing many virtual analytical applications with better scalability and higher cost savings. The capabilities that a service orientation (along with cloud computing, pooled resources, and parallel processing) brings to the analytic world enable cost-effective data/text mining largescale optimization, highly-complex multi-criteria decision problems, and distributed simulation models.

8. Why is MapReduce mentioned in the context of AaaS?

9 .

As we learned from Chapter 13, MapReduce is a programming model that allows the processing of large-scale data analysis problems to be distributed and parallelized.

High performance is achieved by breaking the processing into small units of work that can be run in parallel across the hundreds, potentially thousands, of nodes in the cluster. This fits very well with the AaaS model, based on a service orientation of analytics functionality in a cloud platform involving highly distributed computing.

Section 14.7 Review Questions 1. List the impacts of analytics on decision making.

Analytics can change the manner in which many decisions are made and can consequently change managers’ jobs. They can help managers gain more knowledge, experience, and expertise, and consequently enhances the quality and speed of their decision making. In particular, information gathering for decision making is completed much more quickly when analytics are in use. This affects both strategic planning and control decisions, changing the decision-making process and even decision-making styles.

2. List the impacts of analytics on other managerial tasks.

Less expertise (experience) is required for making many decisions. Faster decision making is possible because of the availability of information and the automation of some phases in the decision-making process. Less reliance on experts and analysts is required to provide support to top executives. Power is being redistributed among managers. (The more information and analysis capability they possess, the more power they have.) Support for complex decisions allows decisions to be made faster and of better quality. Information needed for high-level decision making is expedited or even self-generated. Automation of routine decisions or phases in the decisionmaking process (e.g., for frontline decision making and using ADS) may eliminate some managers, especially middle level managers. Routine and mundane work can be done using an analytic system, freeing up managers and knowledge workers to do more challenging tasks.

3. Describe new organizational units that are created because of analytics.

One change in organizational structure is the possibility of creating an analytics department, a BI department, or a knowledge management department in which analytics play a major role. This special unit can be combined with or replace a quantitative analysis unit, or it can be a completely new entity.

4. How can analytics affect restructuring of business processes?

When a company introduces a data warehouse and BI, the information flows and related business processes (e.g., order fulfillment) are likely to change because 10 .

information flows change. For example, before IBM introduced e-procurement, it restructured all related business processes, including decision making, searching inventories, reordering, and shipping.

5. Describe the impacts of ADS systems.

Automated decision support (ADS) applications will probably have the following impacts: • reduction of middle management • empowerment of customers and business partners • improved customer service • increased productivity of help desks and call centers 6. How can analytics affect job satisfaction?

Analytics has positive or negative impacts. Although many jobs may be substantially enriched by analytics, other jobs may become more routine and less satisfying. But a study by Davenport and Harris found that employees using ADS systems, especially those who were empowered, were more satisfied with their jobs.

Section 14.8 Review Questions 1. List some legal issues of analytics.

• • • • •

What is the value of an expert opinion in court when the expertise is encoded in a computer?

Who is liable for wrong advice (or information) provided by an intelligent application?

What happens if a manager enters an incorrect judgment value into an analytic application and the result is damage or a disaster?

Who owns the knowledge in a knowledge base?

Can management force experts to contribute their expertise?

2. Describe privacy concerns in analytics.

In general, privacy is the right to be left alone and the right to be free from unreasonable personal intrusions. The Internet, in combination with large-scale databases, has created an entirely new dimension of accessing and using data. The inherent power in systems that can access vast amounts of data can be used for the good of society. For example, by matching records with the aid of a computer, it is possible to eliminate or reduce fraud, crime, government mismanagement, tax evasion, welfare cheating, family-support filching, employment of illegal aliens, and so on. The same is true on the corporate level. Private information about employees may aid in better decision making, but the employees’ privacy may be affected.

Similar issues are related to information about customers.

11 .

3. Explain privacy concerns on the Web. The Internet offers a number of opportunities to collect private information about individuals. Here are some of the ways it can be done: • • • • • • •

By reading an individual’s newsgroup postings By looking up an individual’s name and identity in an Internet directory By reading an individual’s e-mail By wiretapping wireline and wireless communication lines and listening to employees By conducting surveillance on employees By asking an individual to complete Web site registration By recording an individual’s actions as he or she navigates the Web with a browser, using cookies or spyware

The implications for online privacy are significant. The ability of law enforcement agencies to authorize installation of pen registers and trap-and-trace devices has increased. The U.S. PATRIOT Act also broadens the government’s ability to access student information and personal financial information without any suspicion of wrongdoing by attesting that the information likely to be found is pertinent to an ongoing criminal investigation.

4. List ethical issues in analytics.

Representative ethical issues that could be of interest in MSS implementations include the following: • • • • • • • • • • • • •

Electronic surveillance Ethics in DSS design Software piracy Invasion of individuals’ privacy Use of proprietary databases Use of intellectual property such as knowledge and expertise Exposure of employees to unsafe environments related to computers Computer accessibility for workers with disabilities Accuracy of data, information, and knowledge Protection of the rights of users Accessibility to information Use of corporate computers for non-work-related purposes How much decision making to delegate to computers

12 .

Section 14.9 Review Questions 1. Identify the nine clusters in the analytics ecosystem.

1.

2.

3.

4.

5.

6.

7.

8.

9.

Data Infrastructure Providers Data Warehouse Industry Middleware Industry Data-Aggregated Distributers Analytics-Focused Software Developers Analytics Industry Analysts and Influencers Academic Providers and Certification Agencies Analytics User Organizations Application Developers: Industry Specific or General

2. Which clusters represent technology developers?

Most involve developers to some degree. The data infrastructure providers, data warehouse industry, middleware industry, and data aggregators all develop technologies that enable analytics applications. Then there are applications developers, especially those focused on analytics applications. Although the user organizations, academic providers, and influencers will include many non-developers, even these clusters will also have some developers in them.

3. Which clusters represent technology users?

The most prominent one is analytics user organizations. But others will include analysts and influencers as well as academic providers and certification agencies.

Indeed, all stakeholders are users of some sort.

4. Give examples of an analytics professional moving from one cluster to another.

One prominent example is people who move into the analysts and influencers cluster.

These will have been either users or developers or perhaps from other clusters as well. A similar migration can bring people into the Academic Provider category, where experts in the field from some of these other clusters become prominent members of this cluster and thereby bring knowledge and expertise to a wider audience. (This answer will vary by student.)

ANSWERS TO APPLICATION CASE QUESTIONS FOR DISCUSSION

 

Application Case 14.1: Great Clips Employs Spatial Analytics to Shave Time in Location Decisions 1.

How is geospatial analytics employed at Great Clips?

13 .

Great Clips depends on a growth strategy that is driven by rapidly opening new stores in the right locations and markets. They use geospatial analysis to help analyze the locations based on the requirements for a potential customer base, demographic trends, and sales impact on existing franchises in the target location.

They use their Alteryx-based solution to evaluate each new location based on demographics and consumer behavior data, aligning with existing Great Clips customer profiles and the potential revenue impact of the new site on the existing sites.

2.

What criteria should a company consider in evaluating sites for future locations?

Major criteria include potential customer base, demographic trends, and sales impact on existing franchises in the target location.

3.

Can you think of other applications where such geospatial data might be useful?

Geospatial data can be used to help customers find the right location (for example, the closest Great Clips location). It is certainly relevant for other companies in a variety of industries. Analyzing customer profiles and applying these to geographic information can assist with many retail firms. Another possibility is utilizing geospatial analysis to find locations for manufacturing facilities; in this case you would be looking for supplier and raw materials’ locations more than customer locations. In the consumer market, geospatial analysis can help users in a variety of applications; for example finding the best locations for restaurants or stores catering to the customer’s desires. (Student answers will vary.)

Application Case 14.2: Quiznos Targets Customers for Its Sandwiches 1.

How can location-based analytics help retailers in targeting customers?

Location-based behavioral targeting can help to narrow the characteristics of users who are most likely to utilize a retailer’s services or products. This sort of analytics would typically target the tech-savvy and busy consumers of the company in question.

2.

Research similar applications of location-based analytics in the retail domain.

Student answers will vary.

Application Case 14.3: A Life Coach in Your Pocket 1.

Search online for other applications of consumer-oriented analytical applications.

Student answers will vary.

14 .

2.

How can location-based analytics help individual consumers?

If a user on a smart phone enters data, the location sensors of the phone can help find others in that location who are facing similar circumstances, as well as local companies providing services and products that the consumer desires. The user can thus see what others in their location are choosing, and the opportunities for meeting his or her needs. Conversely, the user’s behaviors and choices can then contribute information to other consumers in the same location.

3.

How can smartphone data be used to predict medical conditions?

Via smartphones, users can enter data such as gender, age, weight, height, and the location where he or she lives. From this, an app can create a behavior profile compared with health data from the CDC. Predictive analytics can calculate life expectancy of the user, who can begin discovering health opportunities. Also, most smartphones are equipped with accelerometers and gyroscopes to measure jerk, orientation, and sense motion. Muscle motions may be used to predict the progression of disorders such as Parkinson’s disease, as well as tracking exercise activities.

4.

How is ParkPGH different from a “parking space-reporting” app?

ParkPGH does more than just report current parking-space availability. It is also capable of predicting future parking availability. Depending on historical demand and current events, the app is able to provide information on which lots will have free space by the time the driver gets to the destination. The app’s underlying algorithm uses data on current events around the area—for example, a basketball game—to predict an increase in demand for parking spaces later that day, thus saving commuters valuable time searching for parking spaces in the busy city.

ANSWERS TO END OF CHAPTER QUESTIONS FOR DISCUSSION

  

1. What are the potential benefits of using geospatial data in analytics? Give examples.

Geospatial analytics gives organizations a broader perspective and aids in decision making. Geospatial data helps companies with managing operations, targeting customers, and deciding on promotions. It also helps consumers directly, making use of integrated sensor technologies and global positioning systems installed in their smartphones. Using geospatial data, companies can identify the specific needs of the customers and customer complaint locations, and easily trace them back to the products. Another example is in the telecommunications industry, where geospatial analysis can enable communication companies to capture daily transactions from a network to identify the geographic areas experiencing a large number of failed connection attempts of voice, data, text, or Internet.

15 .

2. What type of new applications can emerge from knowing locations of users in real time? What if you also knew what they have in their shopping cart, for example?

One prominent application is in the emerging area of reality mining, which uses location-enabled devices for finding nearby services, locating friends and family, navigating, tracking of assets and pets, dispatching, and engaging in sports, games, and hobbies. Adding shopping cart knowledge will enhance the application’s ability to provide targeted information to a customer; for example, the app could find prices for similar products in nearby stores.

3. How can consumers benefit from using analytics, especially based on location information?

Consumer-oriented analytics based on location information fit into two major categories: (a) GPS navigation and data analysis, and (b) historic and current location demand analysis. Consumers benefit from analytics-based applications in many areas, including fun and health, as well as enhanced personal productivity.

4. “Location-tracking-based profiling (reality mining) is powerful but also poses privacy threats.” Comment.

Students’ answers will differ. Privacy threats relate to user-profiling, intrusive use of personal information, and not being able to control what is being collected.

5. Is cloud computing “just an old wine in a new bottle”? How is it similar to other initiatives? How is it different?

In some ways, cloud computing is a new name for many previous related trends: utility computing, application service provider grid computing, on-demand computing, software as a service (SaaS), and even older centralized computing with dumb terminals. But the term cloud computing originates from a reference to the Internet as a “cloud” and represents an evolution of all previous shared/centralized computing trends.

Cloud computing offers the possibility of using software, hardware, platform, and infrastructure, all on a service-subscription basis. Cloud computing enables a more scalable investment on the part of a user. Like PaaS, etc., cloud-computing offers organizations the latest technologies without significant upfront investment.

6. Discuss the relationship between mobile devices and social networking.

Mobile social networking enables social networking where members converse and connect with one another using cell phones or other mobile devices.

16 .

7. Some say that analytics in general, and ES in particular, dehumanize managerial activities, and others say they do not. Discuss arguments for both points of view.

Students’ answers will differ. A common criticism of traditional data-processing systems is their negative effects on people’s individuality. Such systems are criticized as being impersonal: They may dehumanize and depersonalize activities that have been computerized because they reduce or eliminate the human element that was present in noncomputerized systems. Some people feel a loss of identity; they feel like just another number. On the bright side, one of the major objectives of analytics is to create flexible systems and interfaces that allow individuals to share their opinions and knowledge and work together with computers. Despite all these efforts, some people are still afraid of computers, so they are stressed; others are mostly afraid of their employers watching what they do on the computer.

8. Diagnosing infections and prescribing pharmaceuticals are the weak points of many practicing physicians (according to E. H. Shortliffe, one of the developers of MYCIN). It seems, therefore, that society would be better served if MYCIN (and other ES, see Chapter 11) were used extensively, but few physicians use ES. Answer the following questions: a. Why do you think such systems are little used by physicians?

Students’ answers will differ. Some reasons are: physicians do not understand and therefore do not trust the ES; malpractice insurance does not cover recommendations made by MYCIN; administrators will not invest in it; physicians fear they will be replaced or earn less. b. Assume that you are a hospital administrator whose physicians are salaried and report to you. What would you do to persuade them to use ES? Students’ answers will differ. Students should identify a combination of positive (more money) and negative (calm their fears) motivators c. If the potential benefits to society are so great, can society do something that will increase doctors’ use of such analytic systems?

Probably not--or at least not yet.

9. Discuss the potential impacts of ADS systems on various types of employees and managers.

Automation of routine decisions or phases in the decision-making process (e.g., for frontline decision making and using ADS) may eliminate some managers. In general, it has been found that the job of middle managers is the most likely job to be automated. Midlevel managers make fairly routine decisions, which can be fully automated.

17 .

In general, it has been found that the job of middle managers is the most likely job to be automated. Managers at lower levels do not spend much time on decision making.

However, even if we completely automate their decisional role, we could not automate their jobs.

10. What are some of the major privacy concerns in employing analytics on mobile data?

Loss of privacy is a key concern in employing analytics on mobile data. If someone can track the movement of a cell phone, the privacy of that customer is a big issue.

Some of the app developers claim that they only need to gather aggregate flow information, not individually identifiable information. But many stories appear in the media that highlight violations of this general principle. Sometimes, retailers provide information on their customers to the federal government, in violation of their stated privacy policies.

Legally, the right of privacy is not absolute. The public’s right to know is superior to the individual’s right to privacy. For example, the USA PATRIOT Act broadens the government’s ability to access student information and personal financial information without any suspicion of wrongdoing. Location information from devices has been used to locate victims and criminals, so provides a social good. But at what point is the information not the property of the individual?

11. How can one move from a technology provider cluster to a user cluster?

Individuals in a technology provider cluster move into a user cluster simply by utilizing the technology and tools for solving their own business problems or meeting their consumer/personal needs. In effect, everyone is potentially in the user cluster.

ANSWERS TO END OF CHAPTER APPLICATION CASE QUESTIONS

 

1. What is the main business problem faced by Southern States Cooperative?

Southern States Cooperative is one of the largest farmer-owned cooperatives in the United States, with over 300,000 farmer members being served at over 1,200 retail locations across 23 states. The firm realized the need to continually optimize marketing activities by gaining insights into its customers. It wanted to maintain and extend its success by better targeting the right customers in its direct-marketing campaigns. Specifically, the main business challenges included determining the right set of customers to be targeted for mailing the catalogs, choosing the right combination of storage keeping units (SKUs) to be included in the catalog, cutting down mailing costs, and increasing customer response.

18 .

2. How was predictive analytics applied in the application case?

SSC first built a predictive model to determine which catalogs the customer was most likely to prefer. The data for the analysis included Southern States’ historical customer transaction data analyzed on the basis of recency, frequency, and monetary value of customer transactions (known as RFM analysis). Sample data was analyzed via logistic regression models, a decision tree, and a random forest model. Based on the predictive modeling results, an incremental revenue model based on linear regression was used to estimate revenue per customer. In sum, the predictive model aimed to determine the customer’s catalog use in purchasing selected items.

3. What problems were solved by the optimization techniques employed by Southern States Cooperative?

An optimization model was developed to maximize the total income from mailing the catalogs to customers. This model was solved using genetic algorithms, aiming to maximize the combined selection of the catalog SKUs and the customers to whom the catalog should be sent to result in increased response, at the same time increasing the revenues and cutting down the mailing costs. In sum, the prescriptive model helped the marketing department prepare the customized catalogs containing the SKUs that suited the targeted customer needs, resulting in better revenue generation.

19 .