CODEWITHSUNDEEP
mongodblogback
Erik
Erik

Reputation: 5119

how to store User logs on a large server best practice

From my experience this is what i come up with.

Im currently saving Users and Statistic classes into the MongoDb and everything works grate.

But how do i save the log each user generate?

Was thinking to use the LogBack SiftingAppender and delegate the log information
to separate MongoDb Collections. Like every MongoDb Collection have the id of the user.
That way i don't have to create advanced mapreduce query's since logs are neatly stacked.

Or use SiftingAppender with a FileAppender so each user have a separate log file.

I there is a problem with this if the MongoDB have one million Log Collections each one named with the User Id. (is it even possible btw)

If everything is stored in the MongoDb the MongoDb master-slave replication makes it easy if a master node dies.

What about the FileAppender approach. Feels like there will be a hole lot of log files to administrate. One could maybe save them in folders according to Alphabet. Folder A for user/id with names/id starting with A.

What are other options to make this work?

Upvotes: 2

Views: 478

Answers (2)

Sridhar Nanjundeswaran
Sridhar Nanjundeswaran

Reputation: 754

On your qn of 1M collections, the default namespace file for a db is 16MB which allows about 24000 namespaces (12000 collections + their _id indexes). More info on this website

And you can set maximum .ns (namespace) file size to 2GB with --nssize option, which will allow probably 3072000 namespaces.

Upvotes: 2

j_mcnally
j_mcnally

Reputation: 6968

Make use of Embedded Documents and have one Document for each user with an array of embedded documents containing log files. You can also benefit from sharding if collections get large.

Upvotes: 1

Related Questions