CODEWITHSUNDEEP
pythongoogle-app-enginelogging
JohnGB
JohnGB

Reputation: 2016

Custom logs in AppEngine

I have an application for which I would like to record some statistics for later analysis but which are not at all needed within the application itself. i.e. The data will never need to be read by the application. Whatever system is used for storing these statistics, I just need to be able to take periodic dumps of the data.

I could use the logging built into AppEngine, but then it will display in the application logs. I would like to keep the application logs for debugging the application without having to see other custom information every time I check them.

How would you do this?

Edit: More details on the stats. Each time a user completed a particular task (1-3 times a day) we need to store details about that task. Such as time and order of events. These would be stored in a single string per session - 1-2kB.

Upvotes: 2

Views: 237

Answers (2)

Ibrahim Arief
Ibrahim Arief

Reputation: 9106

I had a similar logging in my app, and what I did is to set up a simple servlet in a self-hosted server that receives the log string and level and store them in our local database. Each time I need this kind of log, I use asynchronous URLFetch to send the data from our app to our logging server.

I could store the log data in the datastore, but then I would lose the option to do full text search of my log strings, which in my experience is invaluable when you're sifting through log files. I could also store the log in an external file and grep only the lines that I need.

Upvotes: 3

Nick Johnson
Nick Johnson

Reputation: 101149

Your best option is to store this data in the datastore. If you're concerned about latency, you can either use asynchronous operations, and start the write as soon as possible, or you can use the task queue to do the writes offline.

Upvotes: 2

Related Questions