nvme-integrity-checker.sh

README.md

NVMe Integrity Checker

A bash script that creates and verifies SHA-256 sidecar files for data integrity checking. Perfect for protecting important files on NVMe drives, external storage, or any filesystem.

Features

• Safe operation - Never deletes or modifies original files
• Multiple modes - Verify-only, create+verify, create-only
• Flexible exclusions - Skip system directories or unwanted paths
• Recursive processing - Handle entire directory trees
• Clear reporting - Shows hash mismatches, missing files, and summary stats

Usage

./nvme-integrity-checker.sh [OPTIONS] <directory>

Options

• -c Create hashes + verify existing (default: verify only)
• -f Force overwrite existing hashes
• -r Recursive
• -e Exclude dirs (comma-separated)
• -n No verify (create only)
• -v Verbose
• -h Help

Examples

Verify-Only Mode (Default)

Check existing sidecar files for integrity violations:

# Verify files in current directory
./nvme-integrity-checker.sh .

# Verify recursively with verbose output
./nvme-integrity-checker.sh -rv /path/to/data

# Verify excluding specific directories
./nvme-integrity-checker.sh -rv -e "temp,cache" /Volumes/MyDrive

Create + Verify Mode

Create new sidecars and verify existing ones:

# Create sidecars for new files, verify existing ones
./nvme-integrity-checker.sh -cr /path/to/data

# With exclusions and verbose output
./nvme-integrity-checker.sh -crv -e "temp,logs,cache" /path/to/data

Create-Only Mode

Create sidecars for files that don't have them, skip verification:

# Fast mode - only create missing sidecars
./nvme-integrity-checker.sh -cnr /path/to/data

# Useful for initial setup or adding sidecars to new files
./nvme-integrity-checker.sh -cnrv -e "temp,logs" /Volumes/ExternalDrive

Output Examples

Successful Verification

Mode: Verify only | Dir: /data | Recursive: Yes

✓ Verified: /data/video.mp4
✓ Verified: /data/audio.wav

Summary: 2 files, 2 verified, 0 failed
✓ Completed successfully

Hash Mismatch Detection

Mode: Verify only | Dir: /data | Recursive: Yes

✓ Verified: /data/good_file.txt
HASH MISMATCH: corrupted_file.txt

Summary: 2 files, 1 verified, 1 failed
ERROR: Completed with errors

Create Mode

Mode: Create + Verify | Dir: /data | Recursive: Yes

Hash exists for: /data/old_file.txt
Verifying: /data/old_file.txt
✓ Verified: /data/old_file.txt
Hashing: /data/new_file.txt
Created: /data/new_file.txt.sha256
Verifying: /data/new_file.txt
✓ Verified: /data/new_file.txt

Summary: 2 files, 2 created, 0 failed
✓ Completed successfully

How It Works

1. Sidecar files: Creates .sha256 files alongside your data files
2. SHA-256 hashing: Uses cryptographically secure hashing
3. Non-destructive: Only creates new files, never modifies originals
4. Smart skipping: Automatically skips .sha256 files and dot files/directories

Use Cases

• NVMe drive integrity monitoring - Detect silent data corruption when the drives have been disconnected for prolonged periods of time
• Archive/Backup verification - Ensure backups haven't been corrupted
• Transfer verification - Confirm files copied correctly

Requirements

• Bash shell
• shasum command (standard on macOS/Linux)
• find command with -print0 support

nvme-integrity-checker.sh

#!/bin/bash

# NVMe integrity checker - creates/verifies SHA-256 sidecar files
set -euo pipefail

# Config
HASH_EXT=".sha256"
VERBOSE=0
FORCE=0
NO_VERIFY=0
EXCLUDE_DIRS=()

# Colors
RED='\033[0;31m'; GREEN='\033[0;32m'; YELLOW='\033[1;33m'; NC='\033[0m'

usage() {
    cat << 'EOF'
Usage: nvme-integrity-checker.sh [OPTIONS] <directory>

OPTIONS:
-c Create hashes + verify existing (default: verify only)
-f Force overwrite existing hashes
-r Recursive
-e Exclude dirs (comma-separated)
-n No verify (create only)
-v Verbose (show skipped files)
-h Help

EXAMPLES:
script.sh /data # Verify only
script.sh -c /data # Create + verify
script.sh -cn /data # Create only
script.sh -cr -e "temp,logs" /data # Create + verify, exclude dirs
EOF
    exit 1
}

error() { echo -e "${RED}ERROR: $@${NC}" >&2; }
success() { echo -e "${GREEN}✓ $@${NC}"; }

compute_hash() {
    local file="$1"
    shasum -a 256 "$file" | awk '{print $1}'
}

# Returns status code and message
verify_file_hash() {
    local file="$1" hash_file="${file}${HASH_EXT}"
    local stored_hash current_hash

    if [ ! -f "$hash_file" ]; then
        echo "no sidecar!"
        return 1
    fi

    stored_hash=$(cat "$hash_file" 2>/dev/null | tr -d '[:space:]')
    [ -z "$stored_hash" ] && {
        echo "EMPTY SIDECAR: $file"
        return 1
    }

    current_hash=$(compute_hash "$file")
    echo -n "sidecar found, computing hash.... "

    if [ "$stored_hash" = "$current_hash" ]; then
        echo "✓"
        return 0
    else
        echo "hash mismatch! (sidecar: $stored_hash, computed: $current_hash)"
        return 1
    fi
}

# Returns status code and message
create_file_hash() {
    local file="$1" hash_file="${file}${HASH_EXT}"
    local current_hash

    # If hash exists and not forced, only verify if not empty
    if [ -f "$hash_file" ] && [ $FORCE -eq 0 ]; then
        local stored_hash=$(cat "$hash_file" 2>/dev/null | tr -d '[:space:]')
        if [ -n "$stored_hash" ]; then
            # Verify existing hash unless NO_VERIFY is set
            if [ $NO_VERIFY -eq 0 ]; then
                echo -n "sidecar found, computing hash.... "
                current_hash=$(compute_hash "$file")
                if [ "$stored_hash" = "$current_hash" ]; then
                    echo "✓"
                    return 0
                else
                    echo "hash mismatch! (sidecar: $stored_hash, computed: $current_hash)"
                    return 1
                fi
            else
                echo "sidecar found, skipping verification"
                return 0
            fi
        fi
    fi

    # Create new hash
    echo -n "no sidecar, creating.... "
    current_hash=$(compute_hash "$file")
    echo "$current_hash" > "$hash_file"
    echo "sidecar created!"

    # Verify newly created hash unless NO_VERIFY is set
    if [ $NO_VERIFY -eq 0 ]; then
        local stored_hash=$(cat "$hash_file" 2>/dev/null | tr -d '[:space:]')
        local current_hash=$(compute_hash "$file")
        echo -n "Verification: "
        if [ "$stored_hash" = "$current_hash" ]; then
            echo "✓"
            return 0
        else
            echo "hash mismatch! (sidecar: $stored_hash, computed: $current_hash)"
            return 1
        fi
    fi

    return 0
}

process_files() {
    local mode="$1" total=0 processed=0 failed=0 skipped=0
    local find_expr=()

    # Build find expression
    find_expr=(
      -type f
      ! -name "*$HASH_EXT"
      ! -path "*/.DocumentRevisions-V100/*"
      ! -path "*/.DS_Store/*"
      ! -path "*/.fseventsd/*"
      ! -path "*/.Spotlight-V100/*"
      ! -path "*/.TemporaryItems/*"
      ! -path "*/.Trashes/*"
      ! -path "*/.*"
  )

  # Exclude user-specified directories
  if [ ${#EXCLUDE_DIRS[@]:-} -gt 0 ]; then
      for exclude_dir in "${EXCLUDE_DIRS[@]}"; do
        find_expr+=( ! -path "*/$exclude_dir/*" )
    done
fi

find_expr+=(-print0)

# If recursive is disabled, add maxdepth
[ $RECURSIVE -eq 0 ] && find_expr=(-maxdepth 1 "${find_expr[@]}")

# Count total files first (for verbose output of skipped files)
if [ $VERBOSE -eq 1 ]; then
    local all_files=0
    while IFS= read -r -d '' file; do
        ((all_files++))
    done < <(find "$TARGET_DIR" -maxdepth 1 -type f -print0 2>/dev/null)
fi

# Process files
while IFS= read -r -d '' file; do
    ((total++))

    # Skip directories (extra safety)
    if [ -d "$file" ]; then
      [ $VERBOSE -eq 1 ] && echo "Skipping directory: $file"
      continue
  fi

  echo -n "Verifying $file ... "

  if [ "$mode" = "create" ]; then
    if create_file_hash "$file"; then
        ((processed++))
    else
        ((failed++))
    fi
else
    if verify_file_hash "$file"; then
        ((processed++))
    else
        ((failed++))
    fi
fi
done < <(cd "$TARGET_DIR" && find . "${find_expr[@]}" 2>/dev/null)

# Calculate skipped files if verbose
if [ $VERBOSE -eq 1 ]; then
    skipped=$((all_files - total))
    [ $skipped -gt 0 ] && echo "Skipped: $skipped files"
fi

echo -e "\nSummary: $total files, $processed $([ "$mode" = "create" ] && echo "created" || echo "verified"), $failed failed"
[ $failed -gt 0 ] && return 1 || return 0
}

# Parse arguments
MODE="" RECURSIVE=0 TARGET_DIR=""
while [[ $# -gt 0 ]]; do
    case $1 in
        --create) MODE="create"; shift ;;
        --force) FORCE=1; shift ;;
        --recursive) RECURSIVE=1; shift ;;
        --no-verify) NO_VERIFY=1; shift ;;
        --exclude|-e) IFS=',' read -ra EXCLUDE_DIRS <<< "$2"; shift 2 ;;
        --verbose) VERBOSE=1; shift ;;
        -h|--help) usage ;;
    -*)
        for (( i=1; i<${#1}; i++ )); do
            case ${1:$i:1} in
                c) MODE="create" ;;
                f) FORCE=1 ;;
                r) RECURSIVE=1 ;;
                n) NO_VERIFY=1 ;;
                v) VERBOSE=1 ;;
                *) error "Unknown: -${1:$i:1}"; usage ;;
            esac
        done
        shift ;;
        *) TARGET_DIR="$1"; shift ;;
    esac
done

# Validate and run
[ -z "$MODE" ] && MODE="verify"
[ -z "$TARGET_DIR" ] && { error "No directory specified"; usage; }
[ ! -d "$TARGET_DIR" ] && { error "Directory not found: $TARGET_DIR"; exit 1; }

if [ "$MODE" = "create" ]; then
    if [ $NO_VERIFY -eq 1 ]; then
        echo "Mode: Create only | Dir: $TARGET_DIR | Recursive: $([ $RECURSIVE -eq 1 ] && echo "Yes" || echo "No")"
    else
        echo "Mode: Create + Verify | Dir: $TARGET_DIR | Recursive: $([ $RECURSIVE -eq 1 ] && echo "Yes" || echo "No")"
    fi
else
    echo "Mode: Verify only | Dir: $TARGET_DIR | Recursive: $([ $RECURSIVE -eq 1 ] && echo "Yes" || echo "No")"
fi
[ ${#EXCLUDE_DIRS[@]:-} -gt 0 ] && echo "Excluding: ${EXCLUDE_DIRS[*]}"
    echo

    if process_files "$MODE"; then
        success "Completed successfully"
    else
        error "Completed with errors"
        exit 1
    fi