Skip to content

Instantly share code, notes, and snippets.

@automactic

automactic/proposal.md

Last active April 28, 2017 04:55
Show Gist options
  • Save automactic/826092b5d15224c85ac494ad644c1c94 to your computer and use it in GitHub Desktop.
Save automactic/826092b5d15224c85ac494ad644c1c94 to your computer and use it in GitHub Desktop.
Download ZIP
Zimfarm task proposal
Raw
proposal.md
  • Forget about the concept of templates
  • We create tasks directly from a json uploaded by admin, which is an array of task objects. The task object looks like below.
  • Every time admin upload a new json, all executing tasks continues, all already scheduled tasks get cancelled, system is populated with tasks from new json. (We can further discuss the strategy used here)
  • Database only keeps task execution history, including what is the command being executed, when task started / finished, which worker executed it, and if success or not
{
    "id": "12345-67890-qwerty",
    "command": "./build_zim.sh -args",
    "scheduling": {
        "repeat": true,
        "initial_exec_time": "2017-05-01T00:00:00.000Z",
        "time_interval": 3600
    }
}
@automactic
Copy link
Author

will sort all these out in the morning
first version: upload json, execute all tasks as soon as possible
assume shell script always exists, else produce error
task: name, command, [memory storage, bandwidth]
file upload: rsync? or something else

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment