for interoperability and exchange between “boxes,” interposing or otherwise – foremost in mind. Gimme a Break(down) The DM industry’s response to big data has been more of the sameold, same-old. In most cases, this means a bouillabaisse of proprietary, stack-centric big data “solutions,” self-serving technological or architectural prescriptions, and not-yet-ready-for-prime-time front-end tools. A few vendors even dangle that most tantalizing of tchotchkes: Big Data-in-a-Box! The thing is, there’s something altogether unprecedented about “big data:” it outstrips the conventional containers into which we like to segment or bin IT technologies. Big data is inescapably multi-disciplinary: it presupposes interconnectedness – interoperability and exchange: commerce –
seen through the lens of big data, no product is an island entire of itself. This doesn’t preclude the development of big data-oriented products that target very specific use cases, nor that of generalized “big data-themed” products which address process-, domain-, or function-specific practices. Nor does it invalidate an entire class of products as in some sense “preBig Data.” And it doesn’t necessarily preclude the development of a stack-ish, platform-like offering that consolidates multiple tools or products into a kind of omnibus “big data” product. But this isn’t what the IBMs, Oracles, and SAPs – to say nothing of the Clouderas, MapRs, or even Hortonworkses – are doing. Instead, they’re developing and marketing “Big Data-ina-Platform” products. There’s the IBM “Big Data Platform,” which comes with its own distribution of Hadoop. Teradata has its “Unified Data Architecture.” SAP has, well,
these “solutions” has in common is a product-centric model: each aims to insert or implant itself – as an interposing box – into a process. Each interposing box introduces latency and increases complexity and fragility. What’s more, each interposing box has its own infrastructure. This includes, crucially, its own vendor-specific support staff with its own esoteric knowledge-base. At best, this means recruiting armies of Java or Pig Latin programmers, or training-up DBAs and SQL programmers in the intricacies of HQL. It means figuring out kludges or hacks to compensate for the primitivity of HCatalog, Hadoop’s still-gestating metadata catalog. At worst, this means investing significant amounts of time and money to develop platform-specific knowledge-bases. WhereScape was founded chiefly to address this dysfunction. With 3D and RED, we’ve focused on
there’s something altogether unprecedented about “big data:” it OUTSTRIPS THE CONVENTIONAL containers into which we like to segment or bin IT technologies. between and among domains. It is holistic in scope in precisely the way that data management is not. From a product perspective, then, a big data-aware tool must operate in a context in which problems, practices, and processes are by definition multi-disciplinary. This means that no product is completely self-sufficient; no product is isolated, siloed, or alone: as Parson Donne might have put it:
HANA-über-alles. Meanwhile, Oracle, Dell, and Microsoft are all pushing their own, platform-specific spins on big data. And don’t forget the Hadoop platform players: Cloudera and MapR continue to pursue increasingly proprietary Hadoop strategies; DataMeer and Platfora, to pitch Hadoop-based stacks as lock-stock replacements for traditional decision support systems. The one thing each of
automating the practices and processes that support and enable a data warehouse environment, such as scoping, warehouse creation, ongoing management, and periodic refactoring. RED even automates the creation and management of warehouse documentation, diagrams, and lineage information. It does this by completely eliminating hand-coding. This means no hand-coding in SQL – nor in esorediscoveringBI Special Edition • #rediscoveringBI • 19