Gemini token-usage logger for interactive sessions.

gemwrap.sh

#!/usr/bin/env bash
# 
# !!!!!!!!!!!!!!
# SCRIPT IS BROKEN FOR NOW, NOT GOT NO TIME TO CHANGE, SORRY
# MIGHT GET UPDATED IN A MONTH OR SO... :(
# !!!!!!!!!!!!!!
# 
# gemwrap: run gemini, mirror its output, and log token stats + git root.
# Can also parse existing gemini output from stdin when used with --parse-only
#
# ╭─────────────────────────────────────────────────────────────────────────────╮
# │                                GEMWRAP                                      │
# │                                                                             │
# │ A wrapper for Google's Gemini CLI that logs token usage statistics to JSON │
# ╰─────────────────────────────────────────────────────────────────────────────╯
#
# DESCRIPTION:
#   gemwrap is a transparent wrapper around Google's Gemini CLI tool that:
#   - Runs gemini normally, showing all output as usual (coding, debugging, etc.)
#   - Extracts token usage statistics from gemini's cumulative stats output
#   - Logs structured data to ~/.gemini/tokens_used.log with session tracking
#   - Includes git repository context (branch, commit, remote provider/repo info)
#   - Prevents duplicate entries using deterministic session IDs
#   - Can also parse historical gemini output via stdin for retroactive logging
#
#   Google's Gemini CLI is an open-source AI agent that brings Gemini models
#   directly into your terminal. It's an agentic coding tool similar to Claude Code
#   or OpenAI's Codex CLI, designed for:
#   - Code understanding, writing, and debugging across large codebases
#   - File manipulation and command execution with natural language
#   - Multi-step workflows, testing, and git operations
#   - Integration with development environments via Model Context Protocol (MCP)
#   - Content generation, research, and task management beyond just coding
#
# INSTALLATION:
#   # Quick install to ~/bin (add ~/bin to PATH if needed)
#   curl -o ~/bin/gemwrap https://gist.github.com/kjanat/92fb86fa6f79ee9a1d40089062050da9/raw/gemwrap.sh
#   chmod +x ~/bin/gemwrap
#
#   # System-wide install (requires sudo)
#   curl -o /usr/local/bin/gemwrap https://gist.github.com/kjanat/92fb86fa6f79ee9a1d40089062050da9/raw/gemwrap.sh
#   chmod +x /usr/local/bin/gemwrap
#
#   # Or download to current directory
#   curl -O https://gist.github.com/kjanat/92fb86fa6f79ee9a1d40089062050da9/raw/gemwrap.sh
#   chmod +x gemwrap.sh
#
# DEPENDENCIES:
#   - gemini (Google's Gemini CLI - install from: https://github.com/google-gemini/gemini-cli)
#   - jq (for JSON processing)
#   - git (optional, for repository context)
#
# USAGE:
#   # Use exactly like gemini - all arguments are passed through
#   gemwrap "Help me debug this Python function"
#   gemwrap "Refactor this component for better performance"
#   gemwrap "Write tests for the user authentication module"
#   
#   # Parse existing gemini output from files or stdin
#   gemwrap --parse-only < old_session.txt
#   cat session1.txt session2.txt | gemwrap --parse-only
#   pbpaste | gemwrap --parse-only  # macOS clipboard
#
#   # Set up as alias to automatically log all gemini usage
#   alias gemini="gemwrap"
#
# EXAMPLES:
#   # Coding tasks (gemini's primary use case)
#   gemwrap "Fix the bug in auth.py and add error handling"
#   gemwrap "Create a React component for user profiles"
#   gemwrap "Optimize this database query and explain the changes"
#   
#   # Multi-step workflows
#   gemwrap "Run the tests, fix any failures, and commit the changes"
#   gemwrap "Analyze the git history and create a changelog"
#   
#   # Code analysis and explanations
#   gemwrap "Explain the architecture of this microservice"
#   gemwrap "Review this pull request for security issues"
#   
#   # Parse old gemini session output for token tracking
#   gemwrap --parse-only <<EOF
#   ╭─────────────────────────────────────╮
#   │  Agent powering down. Goodbye!      │
#   │  Cumulative Stats (5 Turns)         │
#   │  Input Tokens              12,345   │
#   │  Output Tokens              1,234   │
#   │  Thoughts Tokens              567   │
#   │  Total Tokens              14,146   │
#   │  Total duration (API)        2m 5s  │
#   │  Total duration (wall)      15m 30s │
#   ╰─────────────────────────────────────╯
#   EOF
#
#   # Debug mode for troubleshooting
#   GEMWRAP_DEBUG=1 gemwrap "help me with this error"
#
# OUTPUT:
#   Logs are written to: ~/.gemini/tokens_used.log
#   Each entry is a JSON object on its own line (JSONL format):
#   - session_id: Deterministic UUID based on token stats (prevents duplicates)
#   - timestamp: When the log entry was created
#   - session: {turns, command_args}
#   - tokens: {input, output, thoughts, total}
#   - duration: {api, wall}
#   - git: {project_root, branch, commit, provider, owner, repo}
#
# VIEW LOGS:
#   # Pretty print all logs (each line is a separate JSON object)
#   cat ~/.gemini/tokens_used.log | jq .
#   
#   # Show just token totals
#   cat ~/.gemini/tokens_used.log | jq '.tokens.total'
#   
#   # Group by repository
#   cat ~/.gemini/tokens_used.log | jq -s 'group_by(.git.git_repo)'
#   
#   # Convert to proper JSON array for analysis
#   cat ~/.gemini/tokens_used.log | jq -s .
#
# SOURCE:
#   Web:  https://gist.github.com/kjanat/92fb86fa6f79ee9a1d40089062050da9
#   Raw:  https://gist.github.com/kjanat/92fb86fa6f79ee9a1d40089062050da9/raw/gemwrap.sh
#
# ═══════════════════════════════════════════════════════════════════════════════

set -euo pipefail

# Constants
readonly SCRIPT_NAME="$(basename "$0")"
readonly SCRIPT_VERSION="2.0.0"
readonly SCRIPT_URL="https://gist.github.com/kjanat/92fb86fa6f79ee9a1d40089062050da9/raw/gemwrap.sh"
readonly LOG_DIR="${HOME}/.gemini"
readonly LOG_FILE="${LOG_DIR}/tokens_used.log"

# Functions
log_error() {
    echo "${SCRIPT_NAME}: error: $*" >&2
}

log_debug() {
    [[ "${GEMWRAP_DEBUG:-}" == "1" ]] && echo "${SCRIPT_NAME}: debug: $*" >&2
}

cleanup() {
    [[ -n "${tmp_file:-}" && -f "$tmp_file" ]] && rm -f "$tmp_file"
}

check_dependencies() {
    local missing_deps=()
    
    command -v gemini >/dev/null || missing_deps+=("gemini")
    command -v jq >/dev/null || missing_deps+=("jq")
    
    if [[ ${#missing_deps[@]} -gt 0 ]]; then
        log_error "missing required dependencies: ${missing_deps[*]}"
        exit 1
    fi
}

get_git_info() {
    local git_info=""
    
    # Get git root
    local project_root=""
    if project_root=$(git rev-parse --show-toplevel 2>/dev/null); then
        log_debug "found git root: $project_root"
        git_info+='"project_root":"'$project_root'",'
        
        # Get branch
        local branch=""
        if branch=$(git rev-parse --abbrev-ref HEAD 2>/dev/null); then
            git_info+='"git_branch":"'$branch'",'
        fi
        
        # Get commit
        local commit=""
        if commit=$(git rev-parse --short HEAD 2>/dev/null); then
            git_info+='"git_commit":"'$commit'",'
        fi
        
        # Get remote info
        local remote_url=""
        if remote_url=$(git remote get-url origin 2>/dev/null); then
            log_debug "found remote URL: $remote_url"
            
            # Extract provider and repo name
            if [[ $remote_url =~ github\.com[/:](.*)/(.*)\.git ]]; then
                local repo_owner="${BASH_REMATCH[1]}"
                local repo_name="${BASH_REMATCH[2]}"
                git_info+='"git_provider":"github",'
                git_info+='"git_owner":"'$repo_owner'",'
                git_info+='"git_repo":"'$repo_name'",'
            elif [[ $remote_url =~ gitlab\.com[/:](.*)/(.*)\.git ]]; then
                local repo_owner="${BASH_REMATCH[1]}"
                local repo_name="${BASH_REMATCH[2]}"
                git_info+='"git_provider":"gitlab",'
                git_info+='"git_owner":"'$repo_owner'",'
                git_info+='"git_repo":"'$repo_name'",'
            elif [[ $remote_url =~ bitbucket\.org[/:](.*)/(.*)\.git ]]; then
                local repo_owner="${BASH_REMATCH[1]}"
                local repo_name="${BASH_REMATCH[2]}"
                git_info+='"git_provider":"bitbucket",'
                git_info+='"git_owner":"'$repo_owner'",'
                git_info+='"git_repo":"'$repo_name'",'
            else
                git_info+='"git_remote_url":"'$remote_url'",'
            fi
        fi
    fi
    
    echo "$git_info"
}

extract_turns() {
    local content="$1"
    local turns=""
    
    # Extract turn count from "Cumulative Stats (X Turns)"
    if [[ $content =~ Cumulative\ Stats\ \(([0-9]+)\ Turn ]]; then
        turns="${BASH_REMATCH[1]}"
        log_debug "found $turns turns"
    fi
    
    echo "$turns"
}

parse_stats_block() {
    local content="$1"
    local stats_json=""
    
    # Find the stats block
    local stats_block=""
    if [[ $content =~ (Cumulative\ Stats.*╰) ]]; then
        stats_block="${BASH_REMATCH[1]}"
        log_debug "found stats block"
    else
        log_debug "no stats block found"
        return 1
    fi
    
    # Extract turn count
    local turns
    turns=$(extract_turns "$stats_block")
    
    # Parse individual stats
    local input_tokens output_tokens thoughts_tokens total_tokens
    local api_duration wall_duration
    
    [[ $stats_block =~ Input\ Tokens[[:space:]]+([0-9,]+) ]] && input_tokens="${BASH_REMATCH[1]}"
    [[ $stats_block =~ Output\ Tokens[[:space:]]+([0-9,]+) ]] && output_tokens="${BASH_REMATCH[1]}"
    [[ $stats_block =~ Thoughts\ Tokens[[:space:]]+([0-9,]+) ]] && thoughts_tokens="${BASH_REMATCH[1]}"
    [[ $stats_block =~ Total\ Tokens[[:space:]]+([0-9,]+) ]] && total_tokens="${BASH_REMATCH[1]}"
    [[ $stats_block =~ Total\ duration\ \(API\)[[:space:]]+([0-9hms ]+) ]] && api_duration="${BASH_REMATCH[1]}"
    [[ $stats_block =~ Total\ duration\ \(wall\)[[:space:]]+([0-9hms ]+) ]] && wall_duration="${BASH_REMATCH[1]}"
    
    # Generate deterministic session ID
    local session_id
    session_id=$(generate_session_id "$turns" "$input_tokens" "$output_tokens" "$thoughts_tokens" "$total_tokens" "$api_duration" "$wall_duration")
    
    # Build nested JSON structure
    local timestamp
    timestamp=$(date -Iseconds)
    
    local git_info
    git_info=$(get_git_info)
    
    # Remove trailing comma if present
    git_info="${git_info%,}"
    
    # Build the JSON with proper nesting
    stats_json=$(jq -n \
        --arg session_id "$session_id" \
        --arg timestamp "$timestamp" \
        --arg turns "$turns" \
        --arg input_tokens "$input_tokens" \
        --arg output_tokens "$output_tokens" \
        --arg thoughts_tokens "$thoughts_tokens" \
        --arg total_tokens "$total_tokens" \
        --arg api_duration "$api_duration" \
        --arg wall_duration "$wall_duration" \
        --arg command_args "${GEMWRAP_ARGS:-}" \
        '{
            session_id: $session_id,
            timestamp: $timestamp,
            session: {
                turns: ($turns | tonumber // null),
                command_args: $command_args
            },
            tokens: {
                input: ($input_tokens // null),
                output: ($output_tokens // null), 
                thoughts: ($thoughts_tokens // null),
                total: ($total_tokens // null)
            },
            duration: {
                api: ($api_duration // null),
                wall: ($wall_duration // null)
            }
        }')
    
    # Add git info if available
    if [[ -n "$git_info" ]]; then
        # Parse git info and add to JSON
        local git_json="{$git_info}"
        stats_json=$(echo "$stats_json" | jq ". + {git: $git_json}")
    fi
    
    log_debug "parsed stats: input=$input_tokens, output=$output_tokens, turns=$turns"
    echo "$stats_json"
}

generate_session_id() {
    local turns="$1" input="$2" output="$3" thoughts="$4" total="$5" api_dur="$6" wall_dur="$7"
    
    # Create deterministic data string
    local data_string="${turns}|${input}|${output}|${thoughts}|${total}|${api_dur}|${wall_dur}"
    
    # Generate deterministic UUID using SHA256 hash
    # Use first 32 chars of hash and format as UUID
    local hash
    hash=$(echo -n "$data_string" | sha256sum | cut -d' ' -f1)
    
    # Format as UUID: 8-4-4-4-12
    local uuid="${hash:0:8}-${hash:8:4}-${hash:12:4}-${hash:16:4}-${hash:20:12}"
    
    log_debug "generated session ID: $uuid for data: $data_string"
    echo "$uuid"
}

check_duplicate_entry() {
    local session_id="$1"
    
    # Return early if log file doesn't exist
    [[ ! -f "$LOG_FILE" ]] && return 0
    
    log_debug "checking for existing session ID: $session_id"
    
    # Check if session_id already exists in log file
    if grep -q "\"session_id\":\"$session_id\"" "$LOG_FILE" 2>/dev/null; then
        log_debug "duplicate session ID found, skipping"
        return 1
    fi
    
    return 0
}

write_log_entry() {
    local json_data="$1"
    
    # Ensure log directory exists
    mkdir -p "$LOG_DIR"
    
    # Extract session ID from JSON
    local session_id
    session_id=$(echo "$json_data" | jq -r '.session_id')
    
    # Check for duplicates
    if ! check_duplicate_entry "$session_id"; then
        echo "Entry already exists (ID: $session_id), skipping duplicate"
        return 0
    fi
    
    # Append to log file (JSONL format - one JSON object per line)
    echo "$json_data" >> "$LOG_FILE"
    log_debug "logged session $session_id to $LOG_FILE"
    echo "Token stats logged successfully"
}

parse_stdin_mode() {
    log_debug "parse-only mode: processing stdin"
    
    # Create temporary file for stdin content
    tmp_file=$(mktemp) || {
        log_error "failed to create temporary file"
        exit 1
    }
    trap cleanup EXIT
    
    # Read all stdin into temporary file
    cat > "$tmp_file"
    
    if [[ ! -s "$tmp_file" ]]; then
        log_error "no input provided via stdin"
        exit 1
    fi
    
    local content
    content=$(cat "$tmp_file")
    
    # Process the content
    local stats_json
    if stats_json=$(parse_stats_block "$content"); then
        write_log_entry "$stats_json"
        echo "Token stats logged successfully"
    else
        log_error "failed to parse stats from input"
        exit 1
    fi
}

show_version() {
    echo "$SCRIPT_NAME version $SCRIPT_VERSION"
    echo "Source: https://gist.github.com/kjanat/92fb86fa6f79ee9a1d40089062050da9"
}

update_script() {
    log_debug "attempting to update script"
    
    local script_path
    script_path=$(realpath "$0")
    local backup_path="${script_path}.backup.$(date +%s)"
    
    echo "Updating $SCRIPT_NAME from GitHub..."
    echo "Current version: $SCRIPT_VERSION"
    echo "Script location: $script_path"
    
    # Create backup
    if ! cp "$script_path" "$backup_path"; then
        log_error "failed to create backup at $backup_path"
        exit 1
    fi
    
    # Download new version with cache busting
    local temp_file
    temp_file=$(mktemp)
    local download_url="${SCRIPT_URL}?nocache=$(date +%s)"
    
    if ! curl -fsSL -H "Cache-Control: no-cache" -H "Pragma: no-cache" "$download_url" > "$temp_file"; then
        log_error "failed to download update from $SCRIPT_URL"
        rm -f "$temp_file"
        exit 1
    fi
    
    # Verify it's a valid script
    if ! head -1 "$temp_file" | grep -q "^#!/"; then
        log_error "downloaded file doesn't appear to be a valid script"
        rm -f "$temp_file"
        exit 1
    fi
    
    # Replace current script
    if ! mv "$temp_file" "$script_path"; then
        log_error "failed to replace script (check permissions)"
        rm -f "$temp_file"
        exit 1
    fi
    
    # Restore executable permissions
    chmod +x "$script_path"
    
    echo "✓ Successfully updated $SCRIPT_NAME"
    echo "✓ Backup saved to: $backup_path"
    echo "✓ Run '$SCRIPT_NAME --version' to verify"
}

main() {
    # Handle special flags first
    case "${1:-}" in
        --version|-v)
            show_version
            exit 0
            ;;
        --update)
            update_script
            exit 0
            ;;
        --parse-only)
            shift
            parse_stdin_mode "$@"
            return
            ;;
    esac
    
    # Check dependencies
    check_dependencies
    
    # Create temporary file with proper cleanup
    tmp_file=$(mktemp) || {
        log_error "failed to create temporary file"
        exit 1
    }
    trap cleanup EXIT
    
    log_debug "temporary file: $tmp_file"
    log_debug "gemini args: $*"
    
    # Run gemini, capturing both stdout and preserving exit code
    local gemini_exit_code=0
    gemini "$@" | tee "$tmp_file" || gemini_exit_code=$?
    
    # Only process stats if gemini ran successfully
    if [[ $gemini_exit_code -eq 0 ]]; then
        log_debug "processing token stats"
        local content
        content=$(cat "$tmp_file")
        
        local stats_json
        if stats_json=$(parse_stats_block "$content"); then
            write_log_entry "$stats_json"
        else
            log_debug "no stats found to log"
        fi
    else
        log_error "gemini exited with code $gemini_exit_code"
    fi
    
    # Preserve gemini's exit code
    exit $gemini_exit_code
}

# Set environment variable for accessing original args
export GEMWRAP_ARGS="$*"

# Run main function
main "$@"

This line [[ "${GEMWRAP_DEBUG:-}" == "1" ]] && echo "${SCRIPT_NAME}: debug: $*" >&2 will cause the script always failed if there is not GEMWRAP_DEBUG environment variable exist.

Try this:

log_debug() {
    [[ "${GEMWRAP_DEBUG:-}" == "1" ]] && echo "${SCRIPT_NAME}: debug: $*" >&2 || true
}

I can't find any stats block. I am using Gemini CLI v0.1.9.

gemwrap.sh: debug: no stats block found
gemwrap.sh: debug: no stats found to log