Issuu on Google+

Intelligent business analytics — a tool to build decision-support systems for eBusinesses B Azvine, D Nauck and C Ho

An important principle in managing any business is ‘What can’t be measured, can’t be managed’. The complexity of businesses today means that in order to measure business performance one needs to perform considerable analysis of data gathered in vast quantities on a regular basis. Therefore data analysis is at the heart of decision making in all business applications. There is, however, a significant degree of manual intervention in preparing, presenting and analysing business data. Recent advances in intelligent software technology have produced a number of novel techniques to model the human decision-making process. Data analysis tools have in the past been used by businesses mainly as a reactive tool. The pace of change and increased competition means that those businesses that can turn data into information and then into action quickly will have a better chance to survive and out-manoeuvre their rivals. This requires a fundamental change in the way data is used in the enterprise, from a reactive manner to a proactive one. The implications of recent changes can be significant on the level of skilled resources required as well as the cost of such operations. Intelligent software can play an important role in automating the analysis process and up-skill the business users. In this paper we will describe the intelligent business analytics (IBA) platform and two applications developed using it. The paper will focus on soft computing as an emerging technology suitable for incorporation into business analytics applications to model hidden patterns in data and to explain such patterns automatically.


Past and future generations of business analytics

Business analytics systems monitor data generated in business operations so as to analyse performance based on key indicators and present the analysis results to a wide range of users in a format that can be grasped intuitively. One can divide the development of such systems into four generations. The first-generation business information systems were based on centralised batch processing. Analysis of data contained in such systems was difficult and exclusive to experts. The second-generation data warehousing systems based on client/server computing were a major step forward, but if the information was not readily available in the warehouse it could not be found easily by business users. The third-generation systems saw the introduction of OLAP, data mining and Web deployment. Packaged analytic applications in third-generation systems

accelerated deployment. However, detailed analysis of the data still required specialist skills and a high level of familiarity with the available components. As competition requires companies to respond faster to customer needs and revise their internal and external processes, real-time closed-loop processing of data is becoming a necessity. The integration of business analytics into the overall business process can be achieved by building a closed-loop decision-making system in which the output of business analytics is used by operational managers in the form of recommended actions. The vision of intelligent business analytics is extending this closed-loop process to the automatic adjustment of business operations based on decisions made through analysis of available data in real-time. A closed-loop enterprise analytics system that can support real-time processing represents a fourth-generation of business analytics software. BT Technology Journal • Vol 21 No 4 • October 2003


Intelligent business analytics — a tool to build decision-support systems for eBusinesses

In order to achieve this vision we set out to develop a platform that would facilitate the integration of latest intelligent technology within business analytics applications. The objective was to reduce the skill barrier, reduce development time and cost, and take advantage of economies of scale by reusing the platform to develop many applications. These issues will be discussed in more detail in the following sections.


data warehouses

intelligent techniques soft computing fuzzy neuro-fuzzy NN decision trees probablistic models

Motivation for intelligent business analytics (IBA)

The key question that motivated this project was how can the latest intelligent software technology be incorporated in business analytics applications more efficiently than before. To answer this question one must understand how the technology transfer cycle works, where the bottlenecks are, and how they can be removed (see Fig 1). The new technology life cycle consists of three phases [1]. The R&D life cycle is characterised by the technology being immature and the requirement for technology push by those working with the technology. The new application life cycle is characterised by the technology gaining maturity, although there is still the requirement for technology push by those working with it. The adoption life cycle is characterised by the change from a technology push to a market pull with the technology being recognised as mature. There have been many new developments in algorithms and architectures through the latest academic or industrial research that have been found difficult to translate into business applications. One of the problems has been lack of tools to quickly and efficiently try out such algorithms and architectures both in terms of the viability of the technology and early market testing. The problem is most visible in the value chain at the applied research and prototyping stages. This is the void that IBA aims to fill (see Fig 2). IBA is a software platform that supports the later stages of the R&D life cycle as well as the early stages of the new application life cycle. It adds value in the R&D life cycle by enabling more efficient applied research and reducing time required to build and trial prototypes. What differentiates it from other software platforms is that it is focused on incorporating the latest intelligent techniques into business analytics applications. The platform consists of libraries of standard components for database

IBA connection analysis visualisation

modelling explanation optimisation

Fig 2

Schematic diagram of the intelligent business analytics platform.

connectivity, intelligent analysis routines and advanced visualisation components. The aim of the platform is to enable rapid transfer of new technology from research into business applications (see Fig 3).


Why is it intelligent?

Currently, business decisions are made mainly by humans based on available information. Intelligent decision making is to a large extent an exclusive characteristic of humans. Latest developments in artificial intelligence (AI) technology have brought us much closer to modelling human reasoning and learning from examples. A new branch of artificial intelligence known as soft computing recognises that human reasoning is based on imprecise and uncertain information, an ability that has been difficult to incorporate into computer programmes. In traditional computing, the prime desiderata are precision, certainty, and rigour. By contrast, in soft computing the principal notion is that precision and certainty carry a cost and that computation, reasoning, and decision-making should exploit (wherever possible) the tolerance for imprecision, uncertainty, approximate reasoning, and partial truth for obtaining low-cost solutions. This leads to the remarkable human ability of understanding distorted speech, deciphering sloppy handwriting, comprehending the nuances of natural

new application life cycle R&D life cycle

basic research

Fig 1


applied research

adoption life cycle


product development

sales and marketing


The AI application value chain and new technology life cycle — IBA is aimed at adding value to the highlighted activities.

BT Technology Journal • Vol 21 No 4 • October 2003

Intelligent business analytics — a tool to build decision-support systems for eBusinesses classical computing

soft computing

precise models

approximate models

symbolic logic reasoning

classical numerical and search methods

Fig 3

functional approximation and random search

approximate reasoning

Models in classical AI and soft computing paradigms [2].

language, summarising text, recognising and classifying images, driving a vehicle in dense traffic and, more generally, making rational decisions in an environment of uncertainty and imprecision. The challenge, then, is to exploit the tolerance for imprecision by devising methods of computation that lead to an acceptable solution at low cost. This, in essence, is the guiding principle of soft computing [3, 4]. Soft computing (SC) is a term coined by Zadeh, the father of fuzzy logic [5]. Soft computing comprises technologies such as fuzzy logic, neural networks, probabilistic computing, evolutionary computation and their combinations, for example, neuro-fuzzy systems [6]. Because these areas are all based on numeric methods they are also subsumed under the notion ‘computational intelligence’ — a term suggested by Bezdek to characterise numeric approaches to modelling intelligent behaviour in contrast to artificial intelligence, where mostly methods based on symbol manipulation are considered [7]. Real-world problems are typically ill-defined, difficult to model and with large-scale solution spaces [2]. In these cases precise models are impractical, too expensive or nonexistent. Therefore there is a need for approximate modelling techniques capable of handling imperfect and sometimes conflicting information. Soft computing offers a framework for integrating such techniques [8]. Soft computing is a consortium of methodologies that works synergistically and provides a flexible information processing capability for handling real-life ambiguous situations [9]. Its aim is to exploit the tolerance for imprecision, uncertainty, approximate reasoning, and partial truth in order to achieve tractability, robustness, and low-cost solutions. The guiding principle is to find an acceptable solution at low cost by seeking for an approximate solution to an imprecisely/precisely formulated problem. Learning and adaptation are important features of SC techniques. The neuro-fuzzy approach, which provides flexible information processing for representation and recognition of real-life ambiguous situations, forms, at this juncture, a key component of soft computing. Neuro-fuzzy techniques take advantage of

efficient neural network learning algorithms to learn fuzzy rules and adjust their parameters [10].


Soft computing and intelligent data analysis

In order to provide data analysis technology to a broader user spectrum, i.e. also to managers and not only to experts, complex methods must be provided in the form of easy-to-use tools. Today’s data mining tools do not focus on simplicity but are usually obsessed with accuracy, denying the fact that most real-world problems cannot be solved with 100% accuracy anyway and that an imperfect solution that can be easily and readily applied has a much higher benefit. Intelligent data analysis (IDA) goes one step further than today’s data-mining approaches and also considers the suitability of the created solutions in terms such as usability, comprehension, simplicity and cost. The term ‘intelligent’ is used, because expert knowledge is integrated in the analysis process, and knowledge-based methods are used for analysis, while new knowledge is created and communicated by the analysis process. Our platform has a plug-in interface for intelligent data analysis methods. According to the area in which an application based on our platform is deployed, appropriate IDA methods are selected and configured to run automatically without user intervention. We think that soft computing methods are particularly useful, because they can provide the differentiating factor to build solutions that can be understood by the user more easily, can be developed faster and at lower cost. Fuzzy technology, for example, despite having a sound mathematical basis, uses simple expressions taken from everyday language to describe relationships between variables. Fuzzy systems reduce the complexity of a problem by refraining from unnecessarily discriminating very similar values (data compression). Because they are rule-based, fuzzy systems can easily make use of prior knowledge. Sophisticated learning algorithms, taken from neural network theory, are able to automatically refine a fuzzy system or can even create it completely from scratch. Fuzzy systems endowed with such learning capabilities are called neuro-fuzzy systems [10]. Fuzzy technology can also be conveniently used for combining BT Technology Journal • Vol 21 No 4 • October 2003


Intelligent business analytics — a tool to build decision-support systems for eBusinesses

different models that provide partial solutions. Fuzzy rules can be set up to describe the way to combine models and ultimately neuro-fuzzy technology can be used to learn these rules. Neuro-fuzzy systems have the potential to revolutionise IDA and form the basis of our approach to platform and solution development.


The system architecture

For the design of the system architecture the following features were relevant to us:

if the same application, i.e. the same business logic, is used for another customer the code should not change,

• •

any SQL database can be used to store the data,

it should be possible to exchange the business logic, but still retain a lot of the code,

an application should be Web-based so that thin clients can be used, but it should also be able to run as a stand-alone version that does not require network access,

according to the requirements of the application domain, suitable IDA methods must easily plug in,

it should provide a number of reusable GUI modules (maps, charts, etc).

The system that we have built (see Fig 4) has a threetier architecture. A database server holds the operational data and the data structures of the business logic. A Web server uses a number of Java servlets to implement the business logic, to run data analysis, and to provide a generic database interface. The client side is implemented in the form of Java applets that provide the GUI to the application. When we set up an application for a new domain we have to specify a set of generic data structures (tables) in the database that store preprocessed operational data. The business logic that makes use of those tables is distributed between a number of Java

classes and an XML configuration file that stores all required queries and configuration parameters. Some parts of the business logic are generic and can be reused in different application domains (e.g. attribute value grouping, regression, classification). Specific parts of the business logic make use of the generic parts and have to be implemented according to the application requirements (e.g. travel time estimation, fraud detection, decision optimisation). We can also easily exchange different IDA methods in order to tailor the data analysis to the specific domain or customer. For example, for a travel-time estimation system a regression model is required. For one customer it may be appropriate to use a linear model while for another customer a nonlinear model like a neuro-fuzzy model or a neural network may be better suited. On the client side the GUI has to be built according to the specification of the application. The GUI has to present the results of the data analysis in the form of reports, tables and graphical visualisations, e.g. maps and charts. Here we have also implemented a number of reusable modules, such as, for instance, a module that visualises data on geographical maps. The flexibility of the system allows us to easily set up new applications in different areas and also to tailor an application in a specific area to the operation of a new customer. To deploy the same application, i.e. the same business logic, for a different customer we do the following:

map the operational data to the generic data structure, e.g. by changing the XML configuration file,

if required, replace some of the IDA methods, e.g. by either changing the XML configuration file or replacing some Java classes,

if required, switch off some of the available functionality, by either changing the XML configuration file or removing some Java classes (if the customer is not supposed to switch them back on),

queries and configuration (XML)


IDA plug-ins

Fig 4


BT Technology Journal • Vol 21 No 4 • October 2003


generic DB connector business logic


GUI (applets) visualisation analysis results explanations what-if scenarios log-in dialogue

Web server

System architecture.

Intelligent business analytics — a tool to build decision-support systems for eBusinesses


brand the GUI, e.g. by replacing image files or editing configuration information.


Using the generic platform described in the previous section we have implemented two applications — ITEMS (Intelligent Travel Time Estimation and Management System) and DecTOP (Decision Table Optimisation).



ITEMS predicts, manages, visualises and explains travel patterns of a mobile workforce. Service industries such as telecommunications, gas, water, electricity, etc, have to schedule jobs for large mobile workforces. Successful scheduling requires suitable estimates of inter-job times that are mainly determined by travel time. However, it is not sufficient to simply use routing software, because that cannot estimate the time that is required to find a parking space, to gain access to the site, etc. It is also impossible for technicians to log detailed information about the routes they have taken — they only log their actual travel (inter-job) time. Thus, it is not possible to compare travel data with recommendations from the routing software. Travel times that are estimated from historic travel data are more reliable, because they reflect the actual travel behaviour of the workforce, even if only point-topoint information is available. However, point-to-point information is also the only kind of data a scheduler can suitably process, because it would be intractable for a scheduler to analyse different routes while computing a job schedule. Recorded inter-job times automatically reflect features of the areas between which the journey took place. Areas where it is difficult to find a parking space, for example, will automatically result in higher values for the estimated travel time. Because workforce travel displays distinct local patterns, only a fraction of all possible combinations of areas has to be considered by a model for travel estimation. If for some areas there is no travel data available, default estimates are used. For areas where travel occurs regularly, estimates will improve over time. Improved estimates for travel will improve job scheduling, which will finally result in a reduction of unnecessary travel and can thus create huge savings. In addition to reliable estimates, workforce managers also regularly need to analyse the travel behaviour of their workforce in order to determine if improvements are required. ITEMS provides a colour-coded geographical visualisation of travel patterns. Managers can easily identify areas where travel is slow and can assess the performance of technicians on a weekly, daily and individual basis. Self-motivating teams can also use the information from individual travel patterns for improving their travel performance.

ITEMS contains an explanation facility based on decision trees and neuro-fuzzy systems that display rulebased information about individual journeys. The rules derived from travel data explain why a certain journey may have been late. Managers can use the information provided by those rules to improve the overall system behaviour. ITEMS has a learning component that constantly builds new models for travel-time prediction. It compares the new model with the performance of the model currently used by the scheduler and recommends updating the scheduler if the new model performs significantly better. A customised version of ITEMS has been deployed within BT [11]. Because of the nature of the scheduler used by BT, an estimation model in the form of a database table is required. Therefore the IDA module, used to compute the travel time estimates, employs local adaptive regression models. The explanation facility is not required by BT at this time and is therefore not contained in the deployed application. ITEMS has been configured as a Web-based application using an Oracle database.



DecTOP is a decision table evaluation and optimisation tool. Many organisations use decision models in the form of tables or trees to make operational decisions. Call centre operators usually use a decision model in the form of a prescribed interview. Based on the answers given by customers the operator navigates through the decision model to reach an assessment. In order to maintain customer satisfaction and operational excellence, it is very important to constantly monitor the performance of a decision model not only on an overall level, but also on the level of individual decisions (see Fig 5). DecTOP represents a decision model in the form of a table. The software enables the user to identify individual decisions of poor performance and to optimise them manually or automatically. The user can study any number of variants of the decision model in parallel and compare their performance. By changing individual decisions, the user can conduct a what-if analysis. The automatic optimisation of the decision model can be based on accuracy as well as on cost. For this application we use IDA modules for optimisation based on accuracy and cost, and for providing statistical information. DecTOP has been configured as a stand-alone application for Windows PCs using an MSAccess database. Currently the analysis and modification of process data within the multitude of test systems available is largely done manually. DecTOP brings all the benefits of automation to these systems. It frees up expert resources and provides the facility to handle the majority of cases automatically so that the experts can be better utilised on the hard problems. DecTOP also brings the benefits of BT Technology Journal • Vol 21 No 4 • October 2003


Intelligent business analytics — a tool to build decision-support systems for eBusinesses

Fig 5


economies of scale as it aims to use the same basic analysis platform across all systems, thereby reducing development time and training time for the eventual users. We have applied DecTOP to analyse the fault-clearing process of BT Retail. Based on data collected from 600 000 records, our initial results show that DecTOP can increase the accuracy of correct decisions by 13% and reduce the average cost of incorrect decisions by 49%. Ultimately, DecTOP aims to reduce the average time to clear faults — one of the factors that will play an important part in reducing customer dissatisfaction.



This paper describes the development of an intelligent business analytics platform. The aim of the platform was to create a software environment where the latest algorithms and architecture developed within industrial or academic research can be incorporated efficiently into real business applications. Two applications, based on operational needs from within BT businesses, were successfully developed using the platform and have been deployed. The platform has removed some of the bottle-necks in the technology transfer process and has enabled our teams to build experimental systems in a fraction of the time required to do so in the past.

References 1 New technology life cycle — 2 Bonissone P P: ‘Soft Computing: the Convergence of Emerging Reasoning Technologies’, Journal of Research in Soft Computing, 1, No 1 (1997). 3 Zadeh L A: ‘Fuzzy Logic, Neural Networks and Soft Computing’, Comm ACM, 37, No 3, pp 77—84 (1994). 4 Zadeh L A: ‘Soft Computing and Fuzzy Logic’, IEEE Software, 11, No 6, pp 48—58 (1994).


BT Technology Journal • Vol 21 No 4 • October 2003

5 Zadeh L A: ‘Fuzzy Sets, Information and Control’, Volume 8, pp 338— 353, (1965). 6 Azvine B, Azarmi N and Tsui K C: ‘Soft computing — a tool for building intelligent systems’, BT Technol J, 14, No 4, pp 37—45 (October 1996). 7 Bezdek J: ‘What is computational intelligence?’, in Zurada J, Marks R and Robinson C (Eds): ‘Computational Intelligence: Imitating Life’, IEEE Press, Piscataway, pp 1—12 (1994). 8 Azvine B and Wobcke W: ‘Human-centred intelligent systems and soft computing’, BT Technol J, 16 No 3, pp 125—134, (July 1998). 9 Zadeh L A: ‘Foreword’, to Medsker L R (Ed): ‘Hybrid Intelligent Systems’, Kluwer Academic Publishers (1995). 10 Nauck D, Klawonn F and Kruse R: ‘Foundations of Neuro-Fuzzy Systems’, Wiley, Chichester (1997). 11 Azvine B et al: ‘Estimating travel times of field engineers’, BT Technol J, 21, No 4, pp 33—38 (October 2003).

Ben Azvine holds a BSc in Mechanical Engineering, an MSc in Control Engineering, a PhD in Intelligent Control Systems from Manchester University and an MBA from Imperial College, London. Having held research fellowship and lectureship posts in several universities, he joined BT in 1995 to set up a research programme to develop and exploit soft computing and computational intelligence techniques within BT. Since then he has held senior, principal and chief research scientist posts in BT Exact where he currently leads the computational intelligence research group. He has co-edited two books, is an inventor on 20 patents, holds a visiting fellowship at Bristol University and has won several technology awards including the BCS gold medal. His current research interests include the application of soft computing to intelligent data analysis and intelligent information management. His current projects include building a soft computing platform for intelligent data analysis, developing a method-ology for customer satisfaction modelling, developing decision support tools for universal service management, building intelligent information retrieval capability for future contact centres and research into automatic identification of abnormal patterns from sensor data for health care.

Intelligent business analytics — a tool to build decision-support systems for eBusinesses

Detlef Nauck is a Chief Research Scientist leading an international team of researchers in intelligent data analysis in BT Exact’s Intelligent Systems Lab. He holds a Masters degree in Computer Science (1990) and a PhD in Computer Science (1994) both from the University of Braunschweig. He also holds a Venia Legendi (Habilitation) in Computer Science from the Otto-vonGuericke University of Magdeburg (2000), where he is a Visiting Senior Lecturer. He has published seven books and more than 70 papers and he is a regular member of programme committees for conferences on computational intelligence. He is an Associate Editor of IEEE Transactions on Systems, Men and Cybernetics — Part B. His research interests are in the area of computational intelligence, machine learning and data mining. He has developed several neuro-fuzzy learning algorithms that are able to derive linguistically interpretable rules from data. Since he joined BT in 1999, he has worked on several intelligent data analysis projects and a project to create autonomous machine learning systems.

Colin Ho is a Senior Research Scientist at BT Exact’s Intelligent Systems Lab. He received his PhD degree in Computer Science from the University of Essex in 1999. His research interests include data preprocessing, data visualisation, machine learning and data mining especially as they relate to the analysis of very large data sets.

BT Technology Journal • Vol 21 No 4 • October 2003


Business Intelligence paper