Which database to use for structured - high volume - insert+read+summerize data?

Question

Requirements are as follows :

• High volume, structured, real time data.
• data is insert only (no updates)
• Data needs to be summarized- real time.
• data needs to be analyzed - real-time.
• Transactions wont matter (since data is read-only)
• data structure will not be modified frequently. (almost never)

To elaborate a little more...Data is generated by a software...One user can have many instance of this software and our system supports multiple users.

Our System >> Users >> their software (all relations are one to many - left to right)

Answer 1

For Data storage , use Hadoop HDFS (Hadoop File system)

If data is insert only (data structure not getting modified) , use Hive

Use Sqoop as a connector between HDFS and Hive.

If data needs to be search ( something like Search engine ) , use Solr

Lastly for maintaining log data, try Flume

Answer 2

Seriously, the only way to do real-time is to not store it at all beyond maybe writing it to a dated log file for recovery. Use stream processing for the analysis, not a database.