CCrsync - a primitive clone of rsync for ComputerCraft

00-readme.md

CCrsync

It's a primitive """clone""" of rsync for ComputerCraft. I needed a program that would incrementally download files off of a static website served by nginx, onto my in-game computer.

Setup

To set up your website, simply enable JSON autoindexing in nginx. For instance:

location /ccst/ {
   autoindex on;
   autoindex_format json;
   try_files $uri $uri.html $uri/ =404;
}

Note that the format used must be JSON, as this program does not support HTML or XML indexes.

Usage

Download the program and rxi's Lua JSON library onto your computer with wget:

wget https://raw.githubusercontent.com/rxi/json.lua/master/json.lua json.lua
wget https://gist.githubusercontent.com/liquidev/40e7532fdc6424f31960207a64e9699b/raw/7572f4accd348de048be4e53aedd0f49759e1497/10-rsync.lua rsync.lua

Usage is as follows:

rsync <url> <directory>

For instance, to rsync https://example.com/my_project/ into the directory my_project:

rsync https://example.com/my_project/ my_project

Details

CCrsync works by recursively walking JSON indexes generated by nginx to determine which files need to be downloaded. The files' modification dates (mtime fields) are saved in an index.json file inside the provided directory, which is read upon the next time CCrsync is executed to skip downloading files that have not changed since the last sync.

As of now CCrsync is super primitive; it does not even attempt to remove files that don't exist anymore. Consider this more of a proof of concept than a usable development tool.

10-rsync.lua

-- CCrsync - tool for downloading files from a remote computer
 
local json = require "json"
 
local args = { ... }
local url = args[1] or error("no project URL provided")
local dest = args[2] or error("no destination directory provided")
 
print("Creating destination directory")
fs.makeDir(shell.dir().."/"..dest)
local index = {}
do
  local file = io.open(dest.."/index.json")
  if file ~= nil then
    print("Loading index")
    local index_json = file:read('a')
    index = json.decode(index_json)
    file:close()
  end
end
 
local function http_get(url)
  print("GET "..url)
  local request, err = http.get(url)
  if not request then
    error("GET failed: "..err)
  end
  return request.readAll()
end
 
local function http_get_json(url)
  return json.decode(http_get(url))
end
 
local function walk_dir(cwd, url)
  local listing = http_get_json(url)
  for _, entry in ipairs(listing) do
    io.write(entry.type == "directory" and "d" or "*")
    io.write(" ")
    local save_path = cwd.."/"..entry.name
    io.write(save_path)
    if entry.mtime ~= index[save_path] then
      io.write('\n')
      if entry.type == "directory" then
        fs.makeDir(save_path)
        walk_dir(save_path, url..entry.name.."/")
      else
        local contents = http_get(url..entry.name)
        local file = io.open(save_path, "w")
        file:write(contents)
        file:close()
      end
    else
      io.write(" - up to date\n")
    end
    index[save_path] = entry.mtime
  end
end
 
walk_dir(shell.dir().."/"..dest, url)
 
print("Saving index")
local file = io.open(dest.."/index.json", "w")
file:write(json.encode(index))
file:close()