Python dual-logging setup (console and log file) supporting different log levels and colorized output

python_dual_log_setup.py

#!/usr/bin/env python3
# -*- coding: utf-8 -*-

# -------------------------------------------------------------------------------
#                                                                               -
#  Python dual-logging setup (console and log file),                            -
#  supporting different log levels and colorized output                         -
#                                                                               -
#  Created by Fonic <https://github.com/fonic>                                  -
#  Date: 04/05/20 - 02/07/23                                                    -
#                                                                               -
#  Based on:                                                                    -
#  https://stackoverflow.com/a/13733863/1976617                                 -
#  https://uran198.github.io/en/python/2016/07/12/colorful-python-logging.html  -
#  https://en.wikipedia.org/wiki/ANSI_escape_code#Colors                        -
#                                                                               -
# -------------------------------------------------------------------------------

# Imports
import os
import sys
import logging

# Logging formatter supporting colorized output
class LogFormatter(logging.Formatter):

	COLOR_CODES = {
		logging.CRITICAL: "\033[1;35m", # bright/bold magenta
		logging.ERROR:    "\033[1;31m", # bright/bold red
		logging.WARNING:  "\033[1;33m", # bright/bold yellow
		logging.INFO:     "\033[0;37m", # white / light gray
		logging.DEBUG:    "\033[1;30m"  # bright/bold dark gray
	}

	RESET_CODE = "\033[0m"

	def __init__(self, color, *args, **kwargs):
		super(LogFormatter, self).__init__(*args, **kwargs)
		self.color = color

	def format(self, record, *args, **kwargs):
		if (self.color == True and record.levelno in self.COLOR_CODES):
			record.color_on  = self.COLOR_CODES[record.levelno]
			record.color_off = self.RESET_CODE
		else:
			record.color_on  = ""
			record.color_off = ""
		return super(LogFormatter, self).format(record, *args, **kwargs)

# Set up logging
def set_up_logging(console_log_output, console_log_level, console_log_color, logfile_file, logfile_log_level, logfile_log_color, log_line_template):

	# Create logger
	# For simplicity, we use the root logger, i.e. call 'logging.getLogger()'
	# without name argument. This way we can simply use module methods for
	# for logging throughout the script. An alternative would be exporting
	# the logger, i.e. 'global logger; logger = logging.getLogger("<name>")'
	logger = logging.getLogger()

	# Set global log level to 'debug' (required for handler levels to work)
	logger.setLevel(logging.DEBUG)

	# Create console handler
	console_log_output = console_log_output.lower()
	if (console_log_output == "stdout"):
		console_log_output = sys.stdout
	elif (console_log_output == "stderr"):
		console_log_output = sys.stderr
	else:
		print("Failed to set console output: invalid output: '%s'" % console_log_output)
		return False
	console_handler = logging.StreamHandler(console_log_output)

	# Set console log level
	try:
		console_handler.setLevel(console_log_level.upper()) # only accepts uppercase level names
	except:
		print("Failed to set console log level: invalid level: '%s'" % console_log_level)
		return False

	# Create and set formatter, add console handler to logger
	console_formatter = LogFormatter(fmt=log_line_template, color=console_log_color)
	console_handler.setFormatter(console_formatter)
	logger.addHandler(console_handler)

	# Create log file handler
	try:
		logfile_handler = logging.FileHandler(logfile_file)
	except Exception as exception:
		print("Failed to set up log file: %s" % str(exception))
		return False

	# Set log file log level
	try:
		logfile_handler.setLevel(logfile_log_level.upper()) # only accepts uppercase level names
	except:
		print("Failed to set log file log level: invalid level: '%s'" % logfile_log_level)
		return False

	# Create and set formatter, add log file handler to logger
	logfile_formatter = LogFormatter(fmt=log_line_template, color=logfile_log_color)
	logfile_handler.setFormatter(logfile_formatter)
	logger.addHandler(logfile_handler)

	# Success
	return True

# Main function
def main():

	# Set up logging
	script_name = os.path.splitext(os.path.basename(sys.argv[0]))[0]
	if (not set_up_logging(console_log_output="stdout", console_log_level="warning", console_log_color=True,
	                       logfile_file=script_name + ".log", logfile_log_level="debug", logfile_log_color=False,
	                       log_line_template="%(color_on)s[%(created)d] [%(threadName)s] [%(levelname)-8s] %(message)s%(color_off)s")):
		print("Failed to set up logging, aborting.")
		return 1

	# Log some messages
	logging.debug("Debug message")
	logging.info("Info message")
	logging.warning("Warning message")
	logging.error("Error message")
	logging.critical("Critical message")

# Call main function
if (__name__ == "__main__"):
	sys.exit(main())

usage.md

Usage:

There are three options:

1. Use the file above as your project's main.py and add your own code to it (i.e. replace the # Log some messages block in def main with your own code)
2. Copy class LogFormatter, def set_up_logging and the imports to your project's existing main.py (or some other source file), then call set_up_logging to set up logging (an example for this call is given in def main)
3. Save this file as is to a subdirectory of your project's sources (e.g. subdir modules, filename logging.py), add from <subdir>.<filename> import set_up_logging (e.g. from modules.logging import set_up_logging) to the imports section of your project's main.py (or some other source file) and then call set_up_logging to set up logging (an example for this call is given in def main)

New to python, so how do I actually use this? I assume I reference it in main.py somehow?

There are three options:

1. Use the file above as your project's main.py and add your own code to it (i.e. replace the # Log some messages block within def main with your own code). This is probably the best option if you are just getting started with Python.
2. Copy the imports, class LogFormatter and def set_up_logging to your project's existing main.py (or some other source file), then call set_up_logging to set up logging (an example for this call is given within def main). Also a good option to get started quickly.
3. Save this file as is to a subdirectory of your project's sources (e.g. subdir modules, filename logging.py), add from <subdir>.<filename> import set_up_logging (e.g. from modules.logging import set_up_logging) to the imports section of your project's main.py (or some other source file) and then call set_up_logging to set up logging (an example for this call is given in def main). This would be the real deal in terms of how Python projects are usually structured. Go this route if or when things are getting serious and you already have a basic understanding of how things work in Python.

There are three options:

1. Use the

Thanks so much. I really appreciate the guidance and explanation.

Thanks so much. I really appreciate the guidance and explanation.

You're welcome, glad to help.

Hello !

Thanks for the logger :) I'm using it in a project and I had encoding issues when dumping some fields in the logs.

I replaced logfile_handler = logging.FileHandler(logfile_file) by logfile_handler = logging.FileHandler(logfile_file, encoding='utf-8') to fix it.

Thanks for the logger :) I'm using it in a project and I had encoding issues when dumping some fields in the logs.
I replaced logfile_handler = logging.FileHandler(logfile_file) by logfile_handler = logging.FileHandler(logfile_file, encoding='utf-8') to fix it.

Interesting, good to know. What platform/OS are you using?

Thanks for the logger :) I'm using it in a project and I had encoding issues when dumping some fields in the logs.
I replaced logfile_handler = logging.FileHandler(logfile_file) by logfile_handler = logging.FileHandler(logfile_file, encoding='utf-8') to fix it.

Interesting, good to know. What platform/OS are you using?

I'm using windows 11 x64 with python 3.11 I haven't saved the warning pycharm gave me but if I remember right it was detecting some kind of windows char encoding in a utf-8 file.

I'm using windows 11 x64 with python 3.11 I haven't saved the warning pycharm gave me but if I remember right it was detecting some kind of windows char encoding in a utf-8 file.

Windows was my first guess, sounded like it. Well, if anyone else stumbles upon that issue, your comments will certainly be helpful.