Manage Volumes of Machine Generated Data at Very Low Cost

Gartner has stated that data is growing by 650% between 2009 and 2014 and the majority of the growth is a by-product of machine generated data (MGD). According to a Wikipedia entry, almost all machine-generated data is unstructured but then derived into a common more readable structure. Typically, these derived structures contain many data pointers or columns. With these data points, the challenge lies mostly with analyzing the data. Given high performance requirements along with large data sizes, traditional database indexing and partitioning limits the size and history of the dataset for processing. RainStor directly addresses the need to manage this data at scale.

Specific industry MGD includes financial stock trades, network logs, location data, utility smart-grid sensor readings or building management systems, medical devices and of course the broadening range of communications records which includes SMS/MMS content, call detail records (CDR’s) and internet protocol data records (IPDR’s), all of which are growing at a faster rate than human-generated records.

The characteristics of MGD are very well suited to RainStor’s specialized OLDR solution because the data is generally:

• Voluminous
• Highly repetitive & once created never changes
• Created at very fast rates

RainStor’s compression and de-duplication capabilities significantly reduce the storage footprint of these large data sets. For example, one petabyte of CDR’s can be reduced to about 50 terabytes which requires 15x less the number of servers and is therefore much more cost effective.

Industry Specific use-cases include:

• Data retention for network log data across all sectors
• Retention of communications subscriber usage data beyond the billing lifecycle for query and further analysis
• Store for Utility smart-grid data for query and compliance
• Collection and storage of stock trade and tick data for ongoing query and audit

Key Benefits include:

• Lower cost per terabyte stored
• Continuously online and query-able
• Ability to store data for longer time periods to satisfy future business demands
• Fast deployment and very low-admin