Cleans up old files regularly.

README.md

Clean Old Files Utility

This utility provides a configurable way to clean up old files in specified directories on Debian systems. The clean-old-files script checks directories for files older than a configurable number of hours and removes them, excluding specified files and patterns. All actions, warnings, and errors are logged to the local syslog and outputted to STDERR.

---

Table of Contents

• Features
• Requirements
• Installation
• Configuration
• Usage
• Viewing Logs
• Customising the Schedule
• Uninstallation
• Author
• License
• Disclaimer

---

Features

• Configurable Directories and Patterns: Specify which directories to clean and which patterns to exclude.
• Age Threshold: Set the age threshold in hours for files to be processed.
• Dry Run Mode: List files without deleting them by default.
• Syslog Logging: Logs all actions to the local syslog.
• Systemd Integration: Includes service and timer unit files for periodic execution.

---

Requirements

• Debian-based operating system.
• Zsh shell installed (zsh).
• Systemd (for service and timer units).

---

Convenience Script

Usage Instructions

To install the clean-old-files utility using the install.sh script, you can run the following command:

sudo bash -c "$(curl -fsSL https://gist.githubusercontent.com/troykelly/353f99a8206d49e6a6aae5f6bb4e35b5/raw/install.sh)"

Important Notes:

• Running as Root: The script installs files to system directories and requires root privileges. It is invoked with sudo to ensure it has the necessary permissions.

• Security Consideration: Running scripts directly from the internet with elevated privileges can be risky. Ensure you trust the source before executing such commands.

• Alternative: If you prefer to inspect the script before running it, you can download it first:

  curl -fsSL https://gist.githubusercontent.com/troykelly/353f99a8206d49e6a6aae5f6bb4e35b5/raw/install.sh -o install.sh
  sudo bash install.sh

What the Script Does:

1. Checks for Required Commands: Ensures that curl and systemctl are available.

2. Downloads Files:

   • Downloads clean-old-files script to /usr/local/sbin/ and sets it as executable.
   • Downloads the environment file to /etc/.
   • Downloads the systemd service and timer files to /etc/systemd/system/.

3. Sets Permissions and Ownership:

   • Sets appropriate permissions and ownership for each file.

4. Reloads systemd Daemon:

   • Reloads the systemd manager configuration to recognize the new unit files.

5. Enables and Starts the Timer:

   • Enables the clean-old-files.timer to start at boot.
   • Starts the timer immediately.

6. Logs Progress:

   • Outputs informative messages during the installation process.

After Installation:

• Configuration:

  • Edit /etc/clean-old-files.env to adjust configuration variables to your requirements.

Manual Installation

Step 1: Copy the Script

Save the clean-old-files script to /usr/local/sbin/ and make it executable:

sudo cp clean-old-files /usr/local/sbin/
sudo chmod +x /usr/local/sbin/clean-old-files

Step 2: Create the Environment File

Create the /etc/clean-old-files.env file with the default configuration:

sudo touch /etc/clean-old-files.env
sudo chmod 644 /etc/clean-old-files.env
sudo chown root:root /etc/clean-old-files.env

Step 3: Install the Systemd Unit Files

Copy the service and timer files to /etc/systemd/system/:

sudo cp clean-old-files.service /etc/systemd/system/
sudo cp clean-old-files.timer /etc/systemd/system/

Step 4: Reload Systemd and Enable the Timer

Reload the systemd daemon to recognise the new unit files, enable the timer to start at boot, and start the timer:

sudo systemctl daemon-reload
sudo systemctl enable clean-old-files.timer
sudo systemctl start clean-old-files.timer

---

Configuration

Adjust the Environment File

Modify the configuration variables in /etc/clean-old-files.env to suit your requirements:

# Age threshold in hours. Files older than this will be processed.
FILE_CLEANUP_HOURS_THRESHOLD=24

# Space-separated list of directories to clean.
FILE_CLEANUP_DIRECTORIES="/path/to/directory1 /path/to/directory2"

# Space-separated list of patterns to exclude from deletion.
FILE_CLEANUP_EXCLUDE_PATTERNS="*.log *.tmp"

# Set to a truthy value to enable deletion. Otherwise, the script will only list files.
FILE_CLEANUP_DOIT=0

Truthy Values for FILE_CLEANUP_DOIT: 1, yes, true, on (case-insensitive).

Examples

• Set the Age Threshold to 48 Hours:

  FILE_CLEANUP_HOURS_THRESHOLD=48

• Specify Directories:

  FILE_CLEANUP_DIRECTORIES="/home/user/downloads /var/tmp"

• Exclude Specific Patterns:

  FILE_CLEANUP_EXCLUDE_PATTERNS="*.conf important_file.txt"

• Enable Deletion:

  FILE_CLEANUP_DOIT=1

---

Usage

Testing the Script

Before enabling deletion, it's recommended to run the script manually to see which files would be affected.

Run the Script Manually:

sudo /usr/local/sbin/clean-old-files

Check the Logs:

sudo journalctl -t clean-old-files

Enabling Deletion

Once satisfied with the files that would be processed, set FILE_CLEANUP_DOIT to a truthy value in /etc/clean-old-files.env to enable deletion.

Enable Deletion in Environment File:

FILE_CLEANUP_DOIT=yes

Run the Script Manually or Wait for Timer:

sudo /usr/local/sbin/clean-old-files

---

Viewing Logs

All log messages are sent to the local syslog with the identifier clean-old-files. You can view the logs using:

sudo journalctl -t clean-old-files

---

Customising the Schedule

To adjust how often the script runs, edit the OnCalendar value in clean-old-files.timer.

Examples:

• Every Day at Midnight:

  OnCalendar=*-*-* 00:00:00

• Every 15 Minutes:

  OnCalendar=*:0/15

After making changes, reload the daemon and restart the timer:

sudo systemctl daemon-reload
sudo systemctl restart clean-old-files.timer

For more scheduling options, refer to the systemd.time manual.

---

Uninstallation

To remove the service, timer, and configuration:

sudo systemctl stop clean-old-files.timer
sudo systemctl disable clean-old-files.timer
sudo rm /etc/systemd/system/clean-old-files.service
sudo rm /etc/systemd/system/clean-old-files.timer
sudo rm /usr/local/sbin/clean-old-files
sudo rm /etc/clean-old-files.env
sudo systemctl daemon-reload

---

Author

• Name: Troy Kelly
• Username: @troykelly
• Contact: troy@team.production.city

---

Disclaimer

Ensure you thoroughly test the script in a safe environment before deploying it on production systems. Adjust file paths and configurations according to your specific setup. The author is not responsible for any loss or damage caused by the use of this script.

---

clean-old-files

#!/usr/bin/env zsh
# =============================================================================
# File: clean-old-files
# Description:
#   This script checks specified directories for files older than a configurable
#   number of hours and removes them, excluding specified files and patterns.
#   All actions, warnings, and errors are logged to the local syslog and output
#   to STDERR.
#   If the FILE_CLEANUP_DOIT variable is not truthy, the script will only list
#   the files and their details (including human-readable size) instead of deleting.
# Author:
#   Troy Kelly (@troykelly) <troy@team.production.city>
# Date Created:
#   Sunday, 22 September 2024
# Version:
#   1.7
#   - Corrected variable handling to avoid collisions.
#   - Restored original variable names as per user's request.
#   - Followed best practices.
# =============================================================================

set -euo pipefail  # Exit on error, treat unset variables as errors, pipelines fail on first error.

# -----------------------------------------------------------------------------
# Global Configuration Variables
# -----------------------------------------------------------------------------

# The age threshold in hours. Files older than this will be processed.
typeset -i FILE_CLEANUP_HOURS_THRESHOLD
FILE_CLEANUP_HOURS_THRESHOLD=${FILE_CLEANUP_HOURS_THRESHOLD:-24}

# Whether to actually delete files. If FILE_CLEANUP_DOIT is truthy, files will be deleted.
FILE_CLEANUP_DOIT="${FILE_CLEANUP_DOIT:-0}"

# Load FILE_CLEANUP_DIRECTORIES from environment variable before declaring it as an array
FILE_CLEANUP_DIRECTORIES_STRING="${FILE_CLEANUP_DIRECTORIES:-}"

typeset -a FILE_CLEANUP_DIRECTORIES
if [[ -n "${FILE_CLEANUP_DIRECTORIES_STRING}" ]]; then
    FILE_CLEANUP_DIRECTORIES=("${(ps: :)FILE_CLEANUP_DIRECTORIES_STRING}")
else
    FILE_CLEANUP_DIRECTORIES=("/var/log" "/tmp")
fi

# Load FILE_CLEANUP_EXCLUDE_PATTERNS from environment variable before declaring it as an array
FILE_CLEANUP_EXCLUDE_PATTERNS_STRING="${FILE_CLEANUP_EXCLUDE_PATTERNS:-}"

typeset -a FILE_CLEANUP_EXCLUDE_PATTERNS
if [[ -n "${FILE_CLEANUP_EXCLUDE_PATTERNS_STRING}" ]]; then
    FILE_CLEANUP_EXCLUDE_PATTERNS=("${(ps: :)FILE_CLEANUP_EXCLUDE_PATTERNS_STRING}")
else
    FILE_CLEANUP_EXCLUDE_PATTERNS=("*.log" "*.tmp")
fi

# -----------------------------------------------------------------------------
# Function Definitions
# -----------------------------------------------------------------------------

# Logs a message to syslog and outputs to STDERR with a given priority.
# Args:
#   $1: The syslog priority (e.g., "info", "warning", "err", "debug").
#   $2: The message to log.
log_message() {
  local priority="$1"
  local message="$2"
  # Output to STDERR
  echo "${message}" >&2
  # Log to syslog
  logger -p "user.${priority}" -t "clean-old-files" "${message}" 2> /dev/null
}

# Handles script termination gracefully.
cleanup() {
  log_message "info" "Script terminated."
  exit 0
}

# Checks if a variable is truthy (1, yes, true, on).
# Args:
#   $1: The variable value to check.
# Returns:
#   0 if truthy, 1 otherwise.
is_truthy() {
  local value="${(L)1}"  # Convert to lower case
  case "${value}" in
    1|yes|true|on) return 0 ;;
    *) return 1 ;;
  esac
}

# Processes each directory, finds and deletes or lists old files.
process_directories() {
  log_message "info" "Script started."

  # Debugging: Print the configurations (these can be commented out after testing)
  log_message "debug" "Directories to clean: ${FILE_CLEANUP_DIRECTORIES[@]}"
  log_message "debug" "Exclude patterns: ${FILE_CLEANUP_EXCLUDE_PATTERNS[@]}"
  log_message "debug" "Hours threshold: ${FILE_CLEANUP_HOURS_THRESHOLD}"
  log_message "debug" "DOIT: ${FILE_CLEANUP_DOIT}"

  local dir
  for dir in "${FILE_CLEANUP_DIRECTORIES[@]}"; do
    if [[ -d "${dir}" ]]; then
      log_message "info" "Processing directory: ${dir}"

      # Build the find command.
      local find_command=( "find" "${dir}" "-type" "f" "-mmin" "+$((FILE_CLEANUP_HOURS_THRESHOLD * 60))" )

      # Add exclusion patterns to the find command.
      local pattern
      for pattern in "${FILE_CLEANUP_EXCLUDE_PATTERNS[@]}"; do
        find_command+=( "!" "-name" "${pattern}" )
      done

      # Execute the find command and process files.
      local files_to_process
      if files_to_process=("${(@f)$( "${find_command[@]}" 2> /dev/null )}"); then
        if [[ ${#files_to_process[@]} -eq 0 ]]; then
          log_message "info" "No files to process in directory: ${dir}"
        else
          local file file_size
          for file in "${files_to_process[@]}"; do
            # Check if file still exists (it might have been deleted).
            if [[ -e "${file}" ]]; then
              # Get the human-readable file size.
              if file_size=$(du -h -- "${file}" 2> /dev/null | cut -f1); then
                if is_truthy "${FILE_CLEANUP_DOIT}"; then
                  if rm -f -- "${file}"; then
                    log_message "info" "Deleted file: ${file} (Size: ${file_size})"
                  else
                    log_message "err" "Failed to delete file: ${file}"
                  fi
                else
                  log_message "info" "Found file: ${file} (Size: ${file_size})"
                fi
              else
                log_message "warning" "Could not determine size of file: ${file}"
              fi
            else
              log_message "warning" "File not found or inaccessible: ${file}"
            fi
          done
        fi
      else
        log_message "warning" "Failed to execute find command in directory: ${dir}"
      fi
    else
      log_message "err" "Directory not found or inaccessible: ${dir}"
    fi
  done

  log_message "info" "Script completed."
}

# -----------------------------------------------------------------------------
# Signal Handlers
# -----------------------------------------------------------------------------

# Catch signals and clean up before exiting.
trap 'cleanup' INT TERM HUP

# -----------------------------------------------------------------------------
# Main Script Execution
# -----------------------------------------------------------------------------

process_directories

exit 0

clean-old-files.env

# =============================================================================
# File: /etc/clean-old-files.env
# Description:
#   Environment variables for configuring the clean-old-files script.
# Author:
#   Troy Kelly (@troykelly) <troy@team.production.city>
# Date Created:
#   Sunday, 22 September 2024
# Version:
#   1.0
# =============================================================================

# Age threshold in hours. Files older than this will be processed.
FILE_CLEANUP_HOURS_THRESHOLD=24

# Space-separated list of directories to clean.
FILE_CLEANUP_DIRECTORIES="/var/log /tmp"

# Space-separated list of patterns to exclude from deletion.
FILE_CLEANUP_EXCLUDE_PATTERNS="*.log *.tmp"

# Set to a truthy value to enable deletion. Otherwise, the script will only list files.
FILE_CLEANUP_DOIT=0

clean-old-files.service

# =============================================================================
# File: clean-old-files.service
# Description:
#   Systemd service unit file for the clean-old-files script.
#   Configurable via environment variables defined in EnvironmentFile.
# Author:
#   Troy Kelly (@troykelly) <troy@team.production.city>
# Date Created:
#   Sunday, 22 September 2024
# Version:
#   1.0
# =============================================================================

[Unit]
Description=Clean Old Files Service
After=network.target

[Service]
Type=oneshot
ExecStart=/usr/local/sbin/clean-old-files
User=root
Group=root
StandardOutput=syslog
StandardError=syslog
SyslogIdentifier=clean-old-files
EnvironmentFile=/etc/clean-old-files.env

[Install]
WantedBy=multi-user.target

clean-old-files.timer

# =============================================================================
# File: clean-old-files.timer
# Description:
#   Systemd timer unit file to schedule the clean-old-files.service.
# Author:
#   Troy Kelly (@troykelly) <troy@team.production.city>
# Date Created:
#   Sunday, 22 September 2024
# Version:
#   1.0
# =============================================================================

[Unit]
Description=Runs Clean Old Files Service Periodically

[Timer]
OnCalendar=hourly
Persistent=true

[Install]
WantedBy=timers.target

install.sh

#!/usr/bin/env bash
# =============================================================================
# File: install.sh
# Description:
#   Installation script for the clean-old-files utility.
#   Downloads required files from the provided Gist URLs,
#   installs them to the appropriate locations, sets permissions,
#   and enables the systemd service and timer.
# Author:
#   Troy Kelly (@troykelly) <troy@team.production.city>
# Date Created:
#   Sunday, 22 September 2024
# Version:
#   1.0
# =============================================================================

set -euo pipefail  # Exit on error, unset variables are errors, pipelines fail on first error.

# -----------------------------------------------------------------------------
# Configuration Variables
# -----------------------------------------------------------------------------

# Gist Raw URLs
SCRIPT_URL="https://gist.githubusercontent.com/troykelly/353f99a8206d49e6a6aae5f6bb4e35b5/raw/clean-old-files"
ENV_FILE_URL="https://gist.githubusercontent.com/troykelly/353f99a8206d49e6a6aae5f6bb4e35b5/raw/clean-old-files.env"
SERVICE_FILE_URL="https://gist.githubusercontent.com/troykelly/353f99a8206d49e6a6aae5f6bb4e35b5/raw/clean-old-files.service"
TIMER_FILE_URL="https://gist.githubusercontent.com/troykelly/353f99a8206d49e6a6aae5f6bb4e35b5/raw/clean-old-files.timer"

# Destination Paths
SCRIPT_DEST="/usr/local/sbin/clean-old-files"
ENV_FILE_DEST="/etc/clean-old-files.env"
SERVICE_FILE_DEST="/etc/systemd/system/clean-old-files.service"
TIMER_FILE_DEST="/etc/systemd/system/clean-old-files.timer"

# -----------------------------------------------------------------------------
# Function Definitions
# -----------------------------------------------------------------------------

# Logs a message with a given level.
# Args:
#   $1: The log level (INFO, ERROR).
#   $2: The message to log.
log_message() {
  local level="$1"
  local message="$2"
  echo "[$level] $message"
}

# Downloads a file from a URL to a destination path.
# Args:
#   $1: The URL to download from.
#   $2: The destination path.
download_file() {
  local url="$1"
  local dest="$2"
  if curl -fsSL "$url" -o "$dest"; then
    log_message "INFO" "Downloaded $url to $dest"
  else
    log_message "ERROR" "Failed to download $url"
    exit 1
  fi
}

# Checks if a command exists.
# Args:
#   $1: The command to check.
# Returns:
#   0 if the command exists, 1 otherwise.
command_exists() {
  command -v "$1" >/dev/null 2>&1
}

# -----------------------------------------------------------------------------
# Main Installation Steps
# -----------------------------------------------------------------------------

log_message "INFO" "Starting installation of clean-old-files utility."

# Check for required commands
if ! command_exists curl; then
  log_message "ERROR" "curl is required but not installed. Please install curl and retry."
  exit 1
fi

if ! command_exists systemctl; then
  log_message "ERROR" "systemctl is required but not available. Ensure you are on a system with systemd."
  exit 1
fi

# Step 1: Download the script to /usr/local/sbin and make it executable
download_file "$SCRIPT_URL" "$SCRIPT_DEST"
chmod +x "$SCRIPT_DEST"
log_message "INFO" "Installed script to $SCRIPT_DEST and set executable permission."

# Step 2: Download the environment file to /etc
download_file "$ENV_FILE_URL" "$ENV_FILE_DEST"
chmod 644 "$ENV_FILE_DEST"
chown root:root "$ENV_FILE_DEST"
log_message "INFO" "Installed environment file to $ENV_FILE_DEST with appropriate permissions."

# Step 3: Download the service file to /etc/systemd/system
download_file "$SERVICE_FILE_URL" "$SERVICE_FILE_DEST"
chmod 644 "$SERVICE_FILE_DEST"
chown root:root "$SERVICE_FILE_DEST"
log_message "INFO" "Installed service file to $SERVICE_FILE_DEST."

# Step 4: Download the timer file to /etc/systemd/system
download_file "$TIMER_FILE_URL" "$TIMER_FILE_DEST"
chmod 644 "$TIMER_FILE_DEST"
chown root:root "$TIMER_FILE_DEST"
log_message "INFO" "Installed timer file to $TIMER_FILE_DEST."

# Step 5: Reload systemd daemon
systemctl daemon-reload
log_message "INFO" "Reloaded systemd daemon."

# Step 6: Enable and start the timer
systemctl enable clean-old-files.timer
systemctl start clean-old-files.timer
log_message "INFO" "Enabled and started clean-old-files.timer."

log_message "INFO" "Installation completed successfully."

# -----------------------------------------------------------------------------
# End of Script
# -----------------------------------------------------------------------------