CODEWITHSUNDEEP
mongodbtime-series
Yaron Naveh
Yaron Naveh

Reputation: 24436

Is mongodb a good fit for general purpose time series data?

I need to collect various time series metrics that look like this

{event_type, event_time, data_point1, data_point2}

Then I need to perform ad hoc queries like "all datapoint1 values of event X in 5 min interval if datapoint2=7" or "avg of datapoint1-datapoint2 for event Y in this hour".

I've seen some material on modelling time series in mongo but it seems to aggregate data up front based on the queries you will ask. Is it still possible to save in mongo all data points and perform free querying?

Upvotes: 3

Views: 2008

Answers (1)

Sammaye
Sammaye

Reputation: 43884

I may be missing something here however I believe you want a collection of:

{event_type, event_time, data_point1, data_point2}

So that you can perform your queries. This document should satisfy both your queries quite easily, the latter being done with the aggregation framework ($avg).

This will give you full free form querying.

I think you may have just been confused by the linked post ( http://www.quora.com/Time-Series/What-is-the-best-way-to-store-time-series-data-in-MongoDB ). It shows there a form of pre-aggregation which will form up top level results of your data that would normally be too hard and difficult to do on-demand.

Of course, as you have noticed, this method restricts querying ability however a time series can grow impossibly big and I am unsure whether or not the aggregation framework for more complex queries on your free form data set. This is where pre-aggregation comes in. It gives some top level data to your low level data, effectively making aggregation much easier.

So in that post you can see that @Jared actually shows the very first collection he has which is a detail collection of:

{ 
    timestamp: "Sun May 02 2010 19:07:40 GMT-0700", 
    metric1: 10,
    metric2: 20,
}

This would be your detail collection containing:

{event_type, event_time, data_point1, data_point2}

A collection which you can query on in free form but then @Jared goes on to show how you can create to level data to that detail collection to make other aggregation easier.

So I am unsure what the problem is, the detail collection stores all data_points and allows you to perform free querying, provided you have the right indexes of course.

Upvotes: 2

Related Questions