What are some good techniques for logging your applications?

Question

Logging can get complicated, quickly. Considering that you have some code, how do you add logging to it? What library(ies) do you use?

What are some good code techniques for getting the most out of your logging statements while having minimal impact on your application?

Answer 1

There are few things to consider when you decide your logging philosophy to make it performance conscious. First let us breakdown the resources spent on logging.

• CPU time spent on appending or assembling a particular log line & garbage collection
• Working memory spent on appending or assembling a particular log line
• CPU time spent on writing to the destination (I/O), can be file, console, network etc.
• Wall time spent on waiting or fighting on a shared resource or shared destination. This may be a file on disk or a synchronous method inside the logging framework.
• Disk space or any other storage occupied to persist the log file

Following are the things that you can do to minimize the above.

• Have controls over logging. Have proper log levels so can only INFO,WARN,ERROR will be used in production, and DEBUG, TRACE will be used in development. Most importantly levels should not control writing the logs to the destination, but it should also control assembling log lines as well. To achieve this, use parameterized logging ( https://www.slf4j.org/faq.html#logging_performance ) or simply check the log level is enabled before assembling the log line. Using a lambda is an alternative as well.
• Avoid costly formatting. If pretty printing is needed write a log extraction tool convert raw log lines to pretty log lines, which will happen offline, and on demand. There are lot of useful things that can be easily done if the logs are forwarded a tool like Splunk or ElasticSearch.
• Minimize log size. Use shorthand codes to minimize the size of a log lines while preserving the readability.
• Minimize log frequency. When deciding to log something, roughly estimate on how much log size/transaction or how many log lines/ transaction. If it is more than 1:100, then that is too much logging, well it depends.
• Try asynchronous logging. In other words do not write to the destination (I/O) in the transaction thread, rather add it to a buffer and tackle it in the background. RandomAccessFileAppender is such facility provided by Log4J
• Divide and conquer. Have the logs segmented. Later in production, this can be used to spread the load on destinations (I/O) by having different destinations (files) for different logs

Answer 2

Some aspects I would like to add about logging practices.

1. Make it a practice to separate functional logging from development logging. In a production environment, functional logging may involve a DBMS or some other resource. We can achieve this by writing a wrappers on actual logger.
2. In a multi-user environment, log statements become difficult to read, have unique user session-id and log it during development phase. Then a simple perl script can filter out the sections, making debugging simpler. Easy to do using wrapper classes as described above
3. Wrapper classes also free you from being tied to one specific logging api.
4. Try aspect oriented approach to logging where possible, this is essentially done using point cuts (method entry/exit points can be easily covered). Cleaning up code will be easier.

Answer 3

There has been quite some discussion on this in other questions. If you a C# man take a look at What is the best logging solution for a c# net 3.5 project or What’s your logging philosophy?

Search for Logging and you will find many more. :)

Answer 4

Tool for viewing Log4X logs: ChainSaw

Answer 5

Libraries: Log4J and Log4Net (for Java and .NET respectively)

From Log4J site:

Inserting log statements into your code is a low-tech method for debugging it. It may also be the only way because debuggers are not always available or applicable. This is often the case for distributed applications.

On the other hand, some people argue that log statements pollute source code and decrease legibility. (We believe that the contrary is true). In the Java language where a preprocessor is not available, log statements increase the size of the code and reduce its speed, even when logging is turned off. Given that a reasonably sized application may contain thousands of log statements, speed is of particular importance.

With log4j it is possible to enable logging at runtime without modifying the application binary. The log4j package is designed so that these statements can remain in shipped code without incurring a heavy performance cost. Logging behavior can be controlled by editing a configuration file, without touching the application binary.

Logging equips the developer with detailed context for application failures. On the other hand, testing provides quality assurance and confidence in the application. Logging and testing should not be confused. They are complementary. When logging is wisely used, it can prove to be an essential tool.