Page 1

In this final part of my series of articles on integrating Business Intelligence (BI) into the enterprise, we’ll investigate event-driven, on-demand BI processing–the fourth and final technique defined at the beginning of this series. These techniques: 1. Integrate analytical applications with applications using an enterprise portal for access and exploitation by internal and external users 2. Embed analytics in operational applications during application development 3. Introduce Web services to dynamically integrate analytical processing with internal and external operational applications for supporting collaborative commerce 4. Deploy event-driven, on-demand BI processing for user alerts, real-time recommendations, and automated actions. This approach includes Business Activity Monitoring (BAM) Most companies today have a significant investment in BI to measure business performance. However, almost all BI systems are built to support decision-making in specific functional areas, such as marketing, finance, human resources, etc., rather than being built around core processes that span the enterprise and govern how a business operates. An airline, telecommunications provider, manufacturer or bank, for example, may measure sales or analyze costs but probably isn’t using BI to monitor events in its business processes as it operates. So, generally, companies have no active, real-time element to their use of BI. The consequences are that there’s nothing supporting the business to automatically act immediately when problems occur or opportunities arise. Also, there’s nothing to assist employees (i.e., no guidance) in real-time operational job functions in making the right decisions, at the right time, so they can contribute to operational performance in their own job functions. The same applies to business events. Most companies have nothing to continually, automatically monitor events to predict whether problems lie ahead. Also, there’s no automatic actions (or flagging of alerts to take actions) that may avoid unnecessary costs, business disruption, operational mistakes, and unhappy customers in the future. This lack of ”involvement” of BI systems in daily operations often means executives see BI as passive and with no active role in managing operational performance to optimize real-time operations. To implement operational performance management, companies have to extend their BI systems, adding new technology components to help integrate BI with operational business processes. These components can be used to automatically guide people and optimize operations. Examples include: On-demand requests from operational applications for automatic recommendations Automatic monitoring of events in operational business process activities as processes execute to see if any immediate action needs to be taken Event- or timer-driven automatic analysis of data using predictive analytics with little or no human involvement in the analysis process Automatic, rule-based actions where technology can continually test for specific business conditions and changes in state in operations; then, if necessary, alert people, recommend, send messages to other applications or invoke whole business processes

Techniques for Integrating BI into the Enterprise – Part 4 © Intelligent Business Strategies Ltd


Automatic alerting upon detection of an actual business problem or opportunity (e.g., a cancelled cargo order has just occurred that will leave a ship half empty; immediately alert sales to resell the new capacity) Automatic alerting to draw attention to a specific, predicted business problem or opportunity (e.g., given the current sales rate, mineral water inventory will run out in three days in the 5th Ave., New York store) Automatic action messages sent to other applications and processes to prompt response based on detected problems or opportunities Automatic, dynamic, self-tuning “decision” rules, which can be achieved by automatically comparing the performance and accuracy of existing rules to alternative rules and automatically selecting the best rule (rules would be automatically replaced, if necessary, with more accurate ones discovered during this procedure). One key type of event-driven BI processing is BAM. Business Activity Monitoring BAM is the ability to automatically monitor events associated with specific activities in an executing process. Given that business process management software manages process execution, BAM technology needs to integrate with that software to monitor process activities. Figure 1 shows an example of an order entry, fulfilment and tracking process consisting of several process activities. BAM lets companies select specific events to monitor that are associated with specific activities in a process. Using the example in Figure 1, a company may choose to monitor events associated with orders, package assembly, shipping and payments.

What is BAM? – Monitoring Events Associated With Specific Process Activities Order Entry, Fulfilment and Tracking Process

E.g. Order Entry Events New order Cancelled order Changed order Need to decide what activity EVENTS to monitor with BAM technology

Figure 1

Techniques for Integrating BI into the Enterprise – Part 4 © Intelligent Business Strategies Ltd


Reasons for this might be to: Detect any changes in orders, cancelled orders, or large orders and alert the sales force and other operations staff to respond when cancelled orders occur Check orders and predicted demand against inventory to optimize a supply chain by reordering, delaying or cancelling supply of inventory Detect or predict bottlenecks in the package assembly activity Detect or predict delays in shipments for valuable customers Detect or predict cash flow problems because of late payments. To implement BAM, companies need technology to let them: Listen for and capture specific event(s) as a process executes Integrate a necessary set of data using event-driven data integration technology as soon as each specific event being monitored occurs, then make this data available for analysis. This data may come from data warehouses, BI reports, cubes, systems, external data sources or some combination of these Automatically analyze the selected integrated set of data to produce intelligence Use rules to automatically determine whether to take automated action on the intelligence and, if necessary, actually take those actions. Note that actions include firing alerts to draw one or more people's attention to a problem or opportunity, alert with recommendations on what to do next, alert with guided analytics automatically sending messages to one or more internal or external applications to cause something to happen, invocation of a whole process (i.e., a whole chain reaction), etc. Figure 2 shows how a BAM service works to monitor events associated with activities in executing processes. When a monitored event occurs (e.g., a cancelled order), a BAM service is triggered to automatically integrate data, analyze that data, then automatically trigger alerts (e.g., via e-mail or instant message) or action messages to other applications if any action is needed.

Techniques for Integrating BI into the Enterprise – Part 4 Š Intelligent Business Strategies Ltd


BAM - Event Driven Automated Real-Time Processing Process modelling tool

E-mail alerts Real-time Automated BAM actions Dashboard n Front-office (Rules engine) Actio XML) ( systems s e g a actionable Mess rules Intelligence Application Automated Event-driven XML data LLES analysis Integration App events integration Low latency (mining) platform event store Executing BAM Service


Back-office systems DBMS Replication



Other Raw BI Data events operational data

CPM objectives & forecasts

Figure 2 The business benefits of BAM are potentially massive because this kind of technology can prevent business disruption problems from arising that otherwise may cause significant unexpected and additional operational costs, delays, penalties, upset customers, etc. If such events do occur, BAM makes a business much more responsive so it can seize an opportunity, maintain or improve performance and minimize the impact on costs of problems that arise. For example, if a last-minute order cancellation on Monday causes half the cargo capacity of an aircraft scheduled for departure on Friday to become available, then BAM helps the business detect this and alert the necessary sales and cargo operations people to respond rapidly so they can: Resell the available cargo capacity before the flight leaves Optimize labor resources needed to manage the cargo handling on that flight. Without BAM, sales and operations people may not even know about the cancellation and not realize the impact. This may result in excessive labor costs if too many people have been hired for unnecessary cargo handling, a missed sales opportunity and a lower yield by having to fly on Friday with only half the cargo capacity occupied. As shown in Figure 2, three main software components need to be included as part of a BI system to make operational performance management using BAM possible: Event-driven and on-demand data integration Automated analysis (scoring) via deployed data-mining models Automated actions via a rules engine. In addition, these components need to be linked together via workflow.

Techniques for Integrating BI into the Enterprise – Part 4 Š Intelligent Business Strategies Ltd


Looking at the above three components, many Extract, Transform and Load (ETL) tools are already event-driven because they can plug into any Java Messaging Service (JMS)-based, Enterprise Application Integration (EAI) message-queuing software to subscribe to messages. Messages are applications events. Several ETL tools let you invoke mining models and rules engines, from within an ETL workflow, to respectively score integrated data and take automatic action. Therefore, all three of the above components can be executed in a sequence, as part of an event-driven ETL workflow. Alternatively, as long as all three components (ETL data integration job, mining model and rules in a rules engine) can be published as Web services, they can be linked in a "BAM process," using any business process modeling tool, and then attached to a process activity you want to monitor. Another alternative is to use one of the powerful new event-driven BAM server products available from any one of several vendors. These BAM server products often include all the above components in a workflow as part of their product. To maximize value, companies need to identify where in their processes they should deploy BAM and also the activities that would benefit from on-demand recommendations and alerts. An obvious example is to use on-demand recommendations in the customer contact center or any front-office customer touch-point where individual customer intelligence could potentially help contribute to increasing sales, stopping churn and improving customer satisfaction. Also beneficial would be monitoring events in supply and distribution chains to facilitate just-in-time delivery at minimum cost. To take advantage of BAM, companies need to appoint process owners, define their core processes and identify which process activity events to monitor that will yield maximum ROI. These processes would then execute under the control of business process management and application integration software that’s integrated with event-driven data integration and predictive analytics to support BAM. The trend here is toward agents where many BAM jobs can be built with each one becoming a monitoring "agent." Layers of these intelligent BAM agents can then be deployed to help us monitor and manage operations at different enterprise levels (see Figure 3).

Techniques for Integrating BI into the Enterprise – Part 4 Š Intelligent Business Strategies Ltd


Trends – BAM - Layers of Intelligent Agents Will Emerge To Manage Business Operations At Different Levels events

monitoring agent

monitoring agent

monitoring agent


monitoring agent

monitoring agent

monitoring agent

monitoring agent


Figure 3 Ultimately, however, this is about self-tuning business where BAM agents keep their finger on the pulse of business to keep everything running smoothly and optimally. If any events indicate change or problems, then the idea is that the business automatically adjusts itself to re-optimize operations (see Figure 4).

Where Are We Heading With BAM? - Event Driven Self-Tuning Business Automation Actions/ Recommendations Operational data

CPM objectives & forecasts

FrontFront-office Async E-business Q systems

BackBack-office/ office/ Partner E-business systems


Analysis Server

Low latency Data

Q Async

Rules Engine

Data integration

Integ Broker


DBMS Replication

Operational data

Actions/ Recommendations

Figure 4

Techniques for Integrating BI into the Enterprise – Part 4 © Intelligent Business Strategies Ltd


Lou Gerstner, former CEO of IBM, once said in a keynote speech that business should be like the human body. I run up stairs, my heart beats faster, I breathe more heavily and I sweat a little. Nothing tells my body to do this; it just happens. The same thing needs to happen for business. That’s what we’re trying to achieve with BAM. On-Demand Recommendations In addition to BAM, automatic recommendations can help business people. The same three components defined above can also be used to deliver on-demand, automated recommendations. Any ETL workflow that can do data integration, execute a mining model and call a rules engine, could be used in supporting recommendations. For example, if an ETL job that does this can be published as a Web service, then an application can invoke the entire ETL workflow in real-time by simply calling that service. This would cause the whole ETL workflow to be invoked as a service to automatically integrate data, automatically analyze it and call a rules engine to automatically make a recommendation. Similarly, a BAM server with a Web service interface could also do this. Either way, you’re getting more value for money out of the same technology components used for event-driven BAM processing. Generally, BAM is event-driven while live recommendations are requested on-demand (i.e., a request for a recommendation needs to be made from another application). So application requests for live recommendations tend to be client/server-based rather than event-driven. BAM and automated recommendations could be used together as shown in Figure 5. Trends – Guided Analytics to “Steer” The Business User on What Analytics To Look At and What Action(s) To Take

events Event monitoring BAM agent


1. 2. 3. 4.

____ ____ ____ …….

Guided Analytics and recommended action list

Guides the user by telling them to follow an analytical process and/or a recommended action process consisting on 1 or more steps

Figure 5 Here, the monitoring BAM agent detects an event, analyzes the necessary data and alerts the user with a recommended action list to assist and steer the user in making the right decision. This decision should let the user contribute in their small way to one or more strategic objectives. Many small actions taken in this way could add up to a large business benefit. The point is that this kind of BI processing makes it possible for the whole business to start to

Techniques for Integrating BI into the Enterprise – Part 4 © Intelligent Business Strategies Ltd


execute on a common strategy so everyone in the enterprise is contributing to achieving strategic business goals. When everyone pulls in the same direction, it can be amazing how agile even the largest companies can become. If we’re going to achieve this vision, we must redefine what business performance management means. It isn’t just about scorecards and dashboards that the BI vendors are selling. It’s about putting BI systems in the centre of the enterprise and connecting them to processes to empower everyone in the company and turn it into an intelligent business.

Mike Ferguson is Managing Director of Intelligent Business Strategies Limited, a leading information technology analyst and consulting company. As an analyst and consultant he specialises in enterprise business intelligence, enterprise business integration, and intelligent business. He can be contacted at +44 1625 520700 or e-mail at

Techniques for Integrating BI into the Enterprise – Part 4 © Intelligent Business Strategies Ltd


Techniques for Integrating BI into the Enterprise - Part 4  

In this final part of my series of articles on integrating Business Intelligence (BI) into the enterprise, we’ll investigate event-driven, o...