The Python logging module provides a flexible framework for tracking different activities or events when the application is running. The logging module can save the events in log files or any other format that allows you to track when a particular event occurred.
Logging is very useful for your applications because it can help to have clear information to debug when crashes occur. The logging records collected can also provide insights into your application’s performance and where it needs improvement.
Without the information offered by logging, it can be very difficult to know what’s wrong with your application when it goes down.
In this article, you’ll learn everything you need to know to start logging in Python, including the standard logging library, basics of logging in Python, and best practices.
Intro to logging in Python
Part of the standard Python library, the Python logging module tracks events and writes log messages to a user-configured output. There are multiple ways to format log messages, ensuring that you’re able to include all the information you need. A log message can store events that occur during the normal operation of the application, errors within the application, and warnings regarding a specific event.
Logging in Python has many use cases:
- Debugging: If something goes wrong, reading the log messages around the error will help you figure out what really happened, as well as point you towards potential solutions.
- Performance insights: Logging can provide insights about your application’s performance.
For example, you can track activities in your application such as request rates, load performance, and response time. The details you collect can help you identify operations with unnecessarily long run times that can cause problems in the future, and can inform improvements to the overall performance of your application. - Warnings: Logging can be used to record events and send a warning to a logging file, email address, or web server if something unexpected is happening in your application. For example, you might choose to get an email if you’re close to running out of disk space on your server.
Standard logging library
Python comes with a built-in logging module, so you don’t need to install any packages to implement logging in your application. All you need to do is to import the logging module, then set up a basic configuration by using the logging.basicConfig() method. You use logging.{level}(output) to show the log message.
The above example does the following:
- Imports the logging module.
- Creates a basic configuration for the default logger.
- Sets the threshold logging to info.
- Sets a file called
log_file.logwhere log messages will be saved.
The log messages in the log_file.log will read as follows:
INFO:root:The application is running task number 5
The first part of the log message is the logging level (info), followed by the default logger (root), and finally the log message.
Understand logging levels
In the above example, you set the logging level to INFO. Logging levels are labels used to show the importance of the given log message. The Python logging library supports five different logging levels, each associated with a constant value that shows the severity of the log entry.
- Debug (10): The lowest level of log messages, this level shows information that can help you to diagnose a particular problem in your application.
- Info (20): This level shows information that indicates the application is running as expected. For example, “new user has registered”.
- Warning (30): This level shows information that is indicative of future problems. This is not an error, but requires your attention. For example, “Too many requests from IP address 126.45.67.3”, or “Low disk space.”
- Error (40): This level shows that the application has failed to perform some tasks or functions, such as “File failed to upload.”
- Critical (50): This level shows information that indicates serious, urgent errors that can cause the application to stop working.
The lowest severity level is debug, and the highest severity level is critical. The default severity level is warning, which means only events at this level and above will be tracked.
The example below shows how you can configure the logging level to error:
In the example above, the logging level is configured to error. This means that only events at the level of error or above will be tracked.
Configuring Python loggers
Loggers are objects that you can use to create and configure different types of log messages that you want to implement in your application. For example, you can configure one logger to send log messages to a remote machine, and another to send logs to a file. You can also set these loggers at different severity levels, such as debug and warning.
The following example shows how you can create and configure different Python loggers:
The above example does the following:
- Creates the first logger, which is named
first logger. - Sets the threshold logging level of the logger to debug.
- Creates a file handler that sends log messages to a disk file named
first_logger.log. - Creates the second logger named
second logger. - Sets the threshold logging level of the logger to WARNING.
- Creates a socket-based handler that sends log messages to a network socket whose address is given by host and port.
Understanding logging handlers
Logging handlers are responsible for sending the log messages created by loggers to their specified destinations. Logging handlers can also be called targets or writers, depending on the platform.
The Python logging.handlers module has several handlers that you can use in your application.
Filehandler
Filehandler saves log messages directly to a file. In the following example, the file_handler object will save log messages to a file called log_file.log.
Httphandler
Httphandler lets you send log messages to a web server using either GET or POST semantics.
Sockethandler
SendHandler lets you send log messages to TCP/IP sockets.
Streamhandler
Streamhandler lets you send log messages to streams such as sys.stdout and sys.stderr. In the following example, log messages will be sent to the console.
Streamhandler is the recommended handler when you’re debugging your application.
Smtphandler
Smtphandler lets you send log messages to an email address via SMTP. In the following example, the stmp_handler object will send a log message via email with the subject of “Alert!” to [email protected].
This logging handler is especially useful, as it allows you to send urgent log messages directly to the person responsible for handling them. This person might be a system administrator, software developer, or security engineer.
Understanding the formatter
Formatter objects are responsible for specifying the layout of log messages in the final output. It’s up to you to decide what you want the output of the log messages to look like. The minimum recommendation for this is to include the date, time, and logging level in your output format.
The logging module provides various log-record attributes that you can implement in your formatter.
In the following example, you will learn how to specify the final output of the log message by including the date, time, name of the logger, and the specified logging level.
The formatter in the above example sets several parameters.
- %(asctime)s: A human-readable timestamp showing when the log message was created.
- %(name)s: The name of the logger object used to log the call.
- %(levelname)s: The logging level for the message in a text format.
- %(message)s: The log message itself.
Here is the output in the console:
Best practices: timestamps, rotations
To get the most out of your Python logs, it’s important to adhere to logging best practices.
Timestamps
Timestamps can include both the date and time, and are important to include in your log messages. Timestamps make it easy to see when an event occurred, and make accessing historical logs much easier.
For example, if you receive a lot of log messages indicating a warning due to high application traffic, the timestamp will allow you to identify if the high traffic is caused by intermittent spikes, or if traffic consistently goes up only at a specific time of day. This will help you to come up with a solution to handle the situation.
Rotating log files
While at first it may seem easier to save all your logs to a single file, it’s considered best practice to spread your logs across multiple files, especially if you have extensive logs. Having a single massive log file can lead to poor performance, since the system needs to open and close the file each time it records a new log message. A 500 MB log file will take significantly longer to open and close than one that’s limited to 2 MB.
Using a logging handler called RotatingFilehandler can help you rotate your log files. Rotating log files is when you save log messages to a file, and when that file hits a predetermined size, a new file is created. The handler files.RotatingFileHander will rotate log files based on a user-configured maximum size. To enable this, you need to configure the values of two parameters, maxBytes and backupCount.
The maxBytes value is the maximum size of a log file. When the log file is about to reach maxBytes, that file is closed, and a new file is silently opened to receive new log messages. The backupCount value sets names for the new log files when rotating. For example, when you set backupCount to five and a base log file name of logging_file.log, you would get logging_file.log, logging_file.log.1, and so on, up through logging_file.log.5.
The example below shows you how to use and configure RotatingFilehandler in your application.
In the above example, the RotatingFilehandler has a maxBytes value of two thousand bytes, (2 KB) and the backupCount is ten. This means you can have a total of eleven log files, each with a maximum size of 2 KB.
Final thoughts
Tracking different activities or events when your application is running is very important. In this article, you learned how to get started with logging by using Python’s standard logging library and best practices relating to logging in Python.
If you're looking for an even easier way to track your applications' activities, check out Airplane. Airplane is the developer platform for building custom internal tools. With Airplane, you can build Tasks, which are single or multi-step functions that anyone on your team can use, or Views, a React-based platform for building custom UIs quickly. Airplane also offers powerful activity and audit logs that make it easy to track your job executions and troubleshoot them when errors occur.
To get started, sign up for a free account or book a demo.
