Cfengine bundle to download a file.

dl.cf

body common control {
    bundlesequence => { "dl" };
}

bundle agent dl
{

vars:

    "file" string => "cfengine-3.6.5.tar.gz";
    "url" string => "http://s3.amazonaws.com/cfengine.package-repos/tarballs/$(file)";
    "path" string => "/tmp";

methods:

    "get" usebundle => download_file("$(url)","$(path)/$(file)");

}

bundle agent download_file(url,file)
# @depends paths
# @brief Download a file from a network location
#
# Use `curl` to download a file from a network location. See the curl(1) man page for
# a list of protocols supported by curl.
#
# @param url The full URL to the requested resource
# @param file The full local path to store the downloaded resource data
#
# Example:
#
# ```cf3
# vars:
#     "url" string => "http://s3.amazonaws.com/cfengine.package-repos/tarballs/cfengine-3.6.5.tar.gz";
#     "file" string => "/tmp/cfengine-3.6.5.tar.gz";
# methods:
#     "download" usebundle => download_file("$(url)","$(file)");
# ```
{

vars:
    "file_date"
        string => "-z $(file)",
        ifvarclass => fileexists("$(file)");
    "curlflags" string => "--silent --location --continue-at -";

defaults:
    "file_date" string => " ";

commands:
    "$(path[curl]) $(file_date) $(curlflags) --output $(file) $(url)"
}

Probably want contain => silent to cut down noise.

Fileexists will act as a blocker even if the disk file is corrupted.

If the website provided a hash you could get it first, or supply your own, to determine if the larger download is required and to confirm the download when finished.

I want it to work with standard web servers, not needing to have anything beyond HTTP specs. But you're right, there's currently no way to account for an updated remote file.

I've updated it to check the local file timestamp with the remote file timestamp, and instead of --progress-bar, it now uses --silent to avoid unnecessary output. Any error messages will still be surfaced.