Mobile Big Data
Figure 4.2: Analyst’s Preferred Analytics Processing Platform(s) Source: Several Recent Surveys As shown in Figure 4.2 analytics users prefer their analytics processing be performed in various locations with a growing number preferring that Hadoop be included. The group indicating a preference for a Hadoop based analysis platform typically prefer this approach for effectively handling disparate data types. Also note that many of today’s intelligence analysts consider having multiple analytics processing platforms to be acceptable approach. Dark Data, Active Archive and High Density Processing Most enterprises now have a vast amount of available data located at various points of the organization and kept on a variety of storage systems. The issue is that storing all this data is on Tier1 devices can be cost prohibitive and may lead to totally off line storage (inaccessible for short-term requirements). Unfortunately this valuable data resource then becomes “dark data” that for most purposes is dead. The other concern is that of the Tier1 stored data today, approximately 80% hasn’t been accessed in the last six months.Without an effective means to use Tier 2 (secondary disk based) or Tier 3 (typically active tape) the ever growing data storage investment CAPX issues quickly lead to ever increasing dark data. The Big Data answer has increasingly become active archive. With an active archive approach data is actively managed and assigned to the most appropriate storage type and location. Interestingly, none of the active archive managed locations are truly off line, although some clearly have a longer recovery cycle, so nothing ever goes dark and thus becomes of no value. Preserving the value of data may be the 4th V of data. Another characteristic of major data volumes is that in order to process them, for analytics or whatever, they may have to be segmented due to processor platform data size limitations. This data segmentation can be highly inefficient by requiring multiple combinational processing, etc. The Big Data response is through high density processing (actually high data density) platforms. Such processing platforms now approach 100 TB of data for single pass under one operating system, true “first time final” type processing. In effect this high density approach enables valuable application consolidation that can greatly facilitate big load analytics (huge image files, etc.) A good example of the value of these advanced Big Data capabilities was the recent case where the enterprise was able able to reduce a major 200 hour processing job to about 20 minutes, indeed a clear business advantage. Fabric-Like InfiniBand Topology Fusion Labs, Inc
Page 27