g++ -std=c++17 -O2 -pthread main.cpp -o hybrid_logger

Hybrid SPSC Logger: Proof-of-Concept

📌 Overview

This project is a cross-platform, modern C++17 proof-of-concept demonstrating how to implement a high-performance, thread-friendly logging queue using hybrid SPSC (Single Producer Single Consumer) queues. It is designed to be a drop-in backend concept that could eventually integrate with loggers such as g3log, but this demo intentionally avoids any dependency on existing frameworks.

KjellKod / gist:79c4de803fd3149fec1277c00b9a90d9

Created December 11, 2023 22:26

c++ figure out weird template arguments

	#include <cxxabi.h>
	#include <typeinfo>

	void main(){
	int status;
	char * producerType = abi::__cxa_demangle(typeid(someType).name(),0,0,&status);
	char * consumerType = abi::__cxa_demangle(typeid(someType).name(),0,0,&status);
	std::cout << "Producer type: " << producerType << "\n";
	std::cout << "Consumer type: " << consumerType << "\n";
	free(producerType);

KjellKod / concurrentqueue.hpp

Last active April 19, 2018 06:44

G3log: Lock-free using the MPMC dynamic queue from "Moody Camel". 2 files in this gist. concurrentqueue.hpp and at the bottom shared_queue.hpp. See the shared_queue.hpp for how to use this with G3log.

	// Provides a C++11 implementation of a multi-producer, multi-consumer lock-free queue.
	// An overview, including benchmark results, is provided here:
	// http://moodycamel.com/blog/2014/a-fast-general-purpose-lock-free-queue-for-c++
	// The full design is also described in excruciating detail at:
	// http://moodycamel.com/blog/2014/detailed-design-of-a-lock-free-queue


	// Simplified BSD license:
	// Copyright (c) 2013-2015, Cameron Desrochers.
	// All rights reserved.

KjellKod / bench.cpp

Created June 28, 2015 21:12

spdlog performance test, focus on worst case latency. Each thread will write 1 million log entries. Number of threads is variable.

	//// to compile: c++ bench.cpp -o bench -Wall -Wshadow -Wextra -pedantic -std=c++11 -pthread -I../include -O3 -fPIC -Ofast -m64 -march=native

	// Alternative: c++ bench.cpp -o bench -Wall -Wshadow -Wextra -pedantic -std=c++11 -pthread -I../include -O3 -march=native


	// the test code itself is Public domain @ref: Unlicense.org
	// made by KjellKod, 2015, first published for testing of g3log at github.com/kjellkod/g3log
	// Feel free to share, modify etc with no obligations but also with no guarantees from my part either
	// enjoy - Kjell Hedstrom (aka KjellKod)
	//

KjellKod / main_worst_case_latency.cpp

Created June 28, 2015 21:09

G3log: Performance, focus on worst case latency. Each thread will write 1 million log entries. Number of threads is variable.

	// Public domain @ref: Unlicense.org
	// made by KjellKod, 2015. Feel free to share, modify etc with no obligations but also with no guarantees from my part either
	// enjoy - Kjell Hedstrom (aka KjellKod)
	//
	// c++ main_worst_case_latench.cpp -fPIC -Ofast -m64 -march=native -Wall -std=c++11 -stdlib=libc++ -Wunused -o g3log-bench -Ig3log/src/ g3log/build/libg3logger.a
	// OR: c++ main_worst_case_latench.cpp -O3 -march=native -Wall -std=c++11 -stdlib=libc++ -Wunused -o g3log-bench -Ig3log/src/ g3log/build/libg3logger.a

	#include <thread>
	#include <vector>
	#include <atomic>

KjellKod / main_g3log_worst_case_latency.cpp

Last active August 29, 2015 14:23

g3log performance test with atomic

	// c++ thisfile.cpp -fPIC -Ofast -m64 -march=native -Wall -std=c++11 -stdlib=libc++ -Wunused -o g3log-bench -Ig3log/src/ g3log/build/libg3logger.a



	// Public domain @ref: Unlicense.org
	// made by KjellKod, 2015. Feel free to share, modify etc with no obligations but also with no guarantees from my part either
	// enjoy - Kjell Hedstrom (aka KjellKod)

	#include <thread>
	#include <vector>

KjellKod / main_spdlog_worst_case_latency.cpp

Last active November 1, 2018 14:07

spdlog: performance test with atomics, when all threads logs in total less than the maximum queue size : 1048576; // 2 ^ 20

	//// to compile: c++ bench.cpp -o bench -Wall -Wshadow -Wextra -pedantic -std=c++11 -pthread -I../include -O3 -fPIC -Ofast -m64 -march=native

	// Alternative: c++ bench.cpp -o bench -Wall -Wshadow -Wextra -pedantic -std=c++11 -pthread -I../include -O3 -march=native


	// the test code itself is Public domain @ref: Unlicense.org
	// made by KjellKod, 2015, first published for testing of g3log at github.com/kjellkod/g3log
	// Feel free to share, modify etc with no obligations but also with no guarantees from my part either
	// enjoy - Kjell Hedstrom (aka KjellKod)
	//

KjellKod / shared_queue using ModyCamel's lock-free queue

Created June 25, 2015 23:34

	// 2015 by KjellKod.cc
	// A lock-free in-place version of the shared-queue used in g2log and g3log
	// instead of the normal shared-queue https://github.com/KjellKod/g3log/blob/e219a5e426d329a4a4c85aee92ec6eacc43a7125/src/shared_queue.hpp
	// it wraps Moody Camel's lock-free queue
	// http://moodycamel.com/blog/2014/detailed-design-of-a-lock-free-queue
	#pragma once

	#include "concurrentqueue.hpp"

Kjell Hedström KjellKod

g++ -std=c++17 -O2 -pthread main.cpp -o hybrid_logger