0 Automating Function Instrumentation in C++ with Clang and Python.md

Automating Function Instrumentation in C++ with Clang and Python

When dealing with large C++ projects, adding debugging or tracing code to every function manually can be time-consuming. By combining Python with Clang’s parsing library (libclang), this process can be automated.

Preprocessing and Parsing with Clang

The workflow begins with preprocessing the C++ source file using g++ -E. Preprocessing expands macros and resolves includes, producing a file that is easier to analyze with Clang.

Next, the Python script uses clang.cindex to create an AST (Abstract Syntax Tree). It walks through all nodes and collects functions such as:

• Regular functions
• Function templates
• Class methods, constructors, and destructors
• Lambdas

GPU-specific functions marked with __global__ or __device__ are skipped to avoid interference with CUDA code.

The script then records the start line and column of each function body (COMPOUND_STMT) to determine exactly where to insert new code.

Code Insertion

After detecting function bodies, the script can insert custom code automatically. For example, a simple logging statement:

printf("func: %s\n", __func__);

This line prints the function name whenever it is executed, helping with tracing and debugging.

To avoid shifting insertion points, the script sorts all target functions in reverse order by line and column before editing the source code.

Benefits

• Automation: No need to manually edit each function.
• Accuracy: Uses Clang’s AST, ensuring correct detection of functions even in complex C++ code.
• Flexibility: Developers can insert any debugging or instrumentation code, not just printf.

Example Run

Given a simple input file, the script will:

1. Preprocess it with g++ -E.
2. Extract preprocessor mapping information.
3. Parse the AST with Clang to locate all function bodies.
4. Insert the logging code at the start of each function.
5. Output a modified source file ready for compilation.

This approach allows developers to add consistent, automated instrumentation to C++ codebases—making debugging and runtime tracing much easier.

main.py

import clang.cindex
import subprocess
import re

# Clangのライブラリパスを指定（環境に応じて修正）
clang.cindex.Config.set_library_file("C:\\msys64\\ucrt64\\bin\\libclang.dll")

def preprocess_cpp(input_file, output_file):
    """g++ -Eでプリプロセスを実行"""
    try:
        subprocess.run(['g++', '-E', input_file, '-o', output_file], check=True)
        print(f"プリプロセス完了: {output_file}")
    except subprocess.CalledProcessError as e:
        print(f"プリプロセス失敗: {e}")

def find_function_bodies(file_path, preproc_info, orig_path):
    """関数本体(COMPOUND_STMT)の開始位置を特定"""
    index = clang.cindex.Index.create()
    tu = index.parse(file_path, args=['-std=c++17'], options=clang.cindex.TranslationUnit.PARSE_DETAILED_PROCESSING_RECORD)

    functions = []
    for cursor in tu.cursor.walk_preorder():
        
        if cursor.kind in (clang.cindex.CursorKind.FUNCTION_DECL, clang.cindex.CursorKind.FUNCTION_TEMPLATE,
                           clang.cindex.CursorKind.CXX_METHOD, clang.cindex.CursorKind.CONSTRUCTOR,
                           clang.cindex.CursorKind.DESTRUCTOR, clang.cindex.CursorKind.LAMBDA_EXPR):

            is_gpu_func = False
            for token in cursor.get_tokens():
                if token.spelling in ("__global__", "__device__"):
                    # print(f"gpu func")
                    is_gpu_func = True
                    break
            if is_gpu_func: continue

            if cursor.location.file and find_file_for_line(preproc_info, cursor.location.line) == orig_path:
                for child in cursor.get_children():
                    if child.kind == clang.cindex.CursorKind.COMPOUND_STMT:
                        functions.append({
                            'name': cursor.spelling,
                            'line': child.extent.start.line,
                            'column': child.extent.start.column
                        })

    return functions

def extract_preprocessed_info(file_path):
    """
    プリプロセスされたC++ソースコードからディレクティブ情報を抽出し、リストとして返す。
    """
    extracted_data = []
    pattern = r'#\s*(\d+)\s*"([^"]+)"(?:\s*(\d*)\s*(\d*)\s*)?'

    with open(file_path, 'r', encoding='utf-8') as file:
        for idx, line in enumerate(file, start=1):
            match = re.match(pattern, line)
            if match:
                line_number = int(match.group(1))
                file_path = match.group(2)
                flag1 = match.group(3) if match.group(3) else ''
                flag2 = match.group(4) if match.group(4) else ''
                extracted_data.append({
                    'Index': idx,          # プリプロセスされたファイルの行番号
                    'Line Number': line_number,  # 元ファイルの行番号
                    'File Path': file_path,
                    'Flag 1': flag1,
                    'Flag 2': flag2
                })

    return extracted_data

def find_file_for_line(extracted_data, target_line):
    """
    指定された行番号がどのファイル由来かを特定する。
    """
    current_file = None
    last_line_directive = 0

    for entry in extracted_data:
        if entry['Index'] <= target_line:
            current_file = entry['File Path']
            last_line_directive = entry['Index']
        else:
            break

    if current_file:
        # print(f"行番号 {target_line} はファイル '{current_file}' から来ています。")
        return current_file
    else:
        # print(f"行番号 {target_line} に対応するファイルが見つかりませんでした。")
        return None

def insert_code(file_path, functions, insert_text, output_path):
    """関数本体の最初にコードを挿入"""
    with open(file_path, 'r') as file:
        lines = file.readlines()

    # 関数を行番号でソート（逆順に挿入するため）
    functions_sorted = sorted(functions, key=lambda x: (x['line'], x["column"]), reverse=True)

    for func in functions_sorted:
        line_idx = func['line'] - 1  # 0始まりのインデックスに変換
        line = lines[line_idx]

        # 挿入位置をカラム指定で正確に特定
        insert_pos = func['column']

        # 波括弧の直後に挿入
        lines[line_idx] = (line[:insert_pos] + insert_text + line[insert_pos:])
        print(f"関数 '{func['name']}' にコードを挿入しました (行: {func['line']}, 列: {func['column']})")

    # 修正後のファイルを出力
    with open(output_path, 'w') as file:
        file.writelines(lines)
    print(f"修正後のファイルを {output_path} に出力しました。")

if __name__ == "__main__":
    input_cpp = 'sample.cpp'
    preprocessed_cpp = 'sample_E.cpp'
    preprocess_cpp(input_cpp, preprocessed_cpp)
    insert_text = 'printf("func: %s\\n", __func__);'

    preproc_info = extract_preprocessed_info(preprocessed_cpp)

    functions = find_function_bodies(preprocessed_cpp, preproc_info, input_cpp)
    print(f"検出された関数数: {len(functions)}")


    mod_cpp = "sample_E_mod.cpp"
    insert_code(preprocessed_cpp, functions, insert_text, mod_cpp)

sample.cpp

#include <cstdio>
#include <iostream>
using namespace std;

template <typename Func>
void check(char const* const filename, int const lineno, char const* const funcname, Func func)
{
    auto err = func();
    if (err != 0)
    {
        fprintf(stderr, "[debug] %s:%d call:%s error:%d\n", filename, lineno, funcname, err);
        exit(1);
    }
}

constexpr const char* get_filename(const char* filename_abs) {
    size_t const pos = std::string_view(filename_abs).rfind("/");
    return (pos != std::string_view::npos) ? &filename_abs[pos+1] : filename_abs;
}

#define CHECK(func, ...) check(get_filename(__FILE__), __LINE__, #func "(" #__VA_ARGS__ ")", [&](){return func(__VA_ARGS__);})

template <typename Func>
class Defer {
public:
    Defer(Func func) : func_(func) {}
    ~Defer() { this->func_(); }
private:
    Func func_;
};

#define CONCAT(a, b) CONCAT_INNER(a, b)
#define CONCAT_INNER(a, b) a ## b
#define UNIQUE_NAME(base) CONCAT(base, __LINE__)

#define DEFER_CHECK(func, ...) Defer UNIQUE_NAME(defer_)([&](){ CHECK(func, __VA_ARGS__);})

#define DEFER_CODE(code) Defer UNIQUE_NAME(defer_)([&]()code)

template<int i>
int hello() {
    cout << "Hello, World! (" << i << ")" << endl;
    return 0;
}

int main() {
    ([&](){ })();
    CHECK(hello<0>);
    DEFER_CHECK(hello<10>);
    return 0;
}