However, based on these traditional consolidation approaches data integration has numerous moving parts that must be synchronized -- including the schema, the mappings, the ETL scripts, and more. Aligning these properly slows solution delivery. In fact, TDWI confirms this lack of agility: their recent study stated the average time needed to add a new data source to an existing BI application was 8.4 weeks in 2009, 7.4 weeks in 2010, and 7.8 weeks in 2011. And, 33% of the organizations needed more than 3 months to add a new data source.
ment data abstraction. From an enterprise architecture point of view, data virtualization provides a semantic abstraction (or data services) layer supporting multiple consuming applications. This middle layer of reusable views (or data services) decouples the underlying source data and consuming solutions, providing the flexibility required to deal with each source and consumer in the most effective manner, as well as the agility to work quickly across sources and consumers as applications, schemas, or underlying data sources change.
Data Abstraction Addresses Data Diversity
Pain Relief at Last!
The schema is the key: this is where the various data silos are rationalized from the various source system taxonomies into common business ontologies. A common term for this activity is data abstraction -the process of transforming data from its native structure and syntax into reusable objects that business applications and consumers can understand. Some data abstraction approaches used today work better than others. For example, some organizations build data abstraction by hand in Java or use business process management (BPM) tools. Unfortunately, these are often constrained by brittleness and inefficiencies. Further, these tools are not effective for large data sets since they lack the robust federation and query optimization functions required to meet data consumers’ rigorous performance demands.
Analytics and BI can make bigger business impact when they can access more data. Data warehouse schemas can also provide data abstraction. Data modeling strategies for dimensions, hierarchies, facts, and more are well documented. Also well understood is the high cost and lack of agility in the data warehousing approach. However, data warehouse based schemas often don’t include the many new classes of data (big data, cloud data, external data services, and more) that reside outside the data warehouse. Data virtualization (DV) is a third way to imple-
Analytics and BI can make bigger business impact when they can access more data. With additional data now required from cloud and big data silos, accessing and integrating these new sources can be a challenge for enterprises accustomed to a traditional enterprise data warehouse centric data integration approach. Data abstraction, when implemented using data virtualization, simplifies and accelerates integration of cloud, big data, and enterprise sources by bridging the gap between diverse business needs and ontologies and even more diverse data sources and taxonomies. Done right, the benefits can be significant, including: • Simplified information access – Bridge business and IT terminology and technology so both can succeed; • Common business view of the data – Gain agility, efficiency, and reuse across applications via an enterprise information model or “Canonical” model; • More accurate data – Consistently apply data quality and validation rules across all data sources; • More secure data – Consistently apply data security rules across all data sources and consumers via a unified security framework; • End-to-end control – Use a data virtualization platform to consistently manage data access and delivery across multiple sources and consumers; and • Business and IT change insulation – Insulate consuming applications and users from changes in the source and vice versa. Business users and applications developers work with a more stable view of the data, and IT can make ongoing changes and relocation of physical data sources without impacting information users. For more information on Composite Software’s Data Abstraction Reference Architecture, please read the Data Abstraction Best Practices White Paper >. rediscoveringBI Magazine • #rediscoveringBI •
16