lotas

Queue Forecasting — Spec

Overview

A standalone tool that listens to Mozilla's Taskcluster Pulse event stream, collects task lifecycle data, and predicts how long tasks will take to run and wait in queue. Lives in tools/queue-forecasting/ within the Taskcluster monorepo.

Environment

lotas

use std::net::TcpStream;
use std::io::{self, Read};
use std::time::{SystemTime, UNIX_EPOCH};

fn get_current_unix_timestamp() -> u64 {
    let start = SystemTime::now();
    
    let since_the_epoch = start.duration_since(UNIX_EPOCH).expect("Ehmmm..");

    // Adjust for the difference between 1900 and 1970

lotas

-- assist easier future migration, add readable columns to azure_queue_messages

alter table azure_queue_messages
  add column task_queue_id text null, -- expose cryptic names
  add column priority int null, -- don't forget about the priority
  add column payload jsonb null; -- instead of base64 encoded string use jsonb


-- patch functions to use new columns
CREATE OR REPLACE FUNCTION public.azure_queue_put_extra(queue_name text, message_text text, visible timestamp without time zone, expires timestamp without time zone, task_queue_id text, priority int, payload jsonb)

lotas

Little helper to call Taskcluster api calls in command line

Main problem that auth uses HAWK which requires libraries to calculate hmac of the authorization header

Usage in shell:

tc_api POST queue/v1/task '{payload here'}

tc_api GET auth/v1/ping

lotas

Worker Versions

Generic Worker

Total: 23

Count by version:

lotas

Taskcluster H2 Planning 2022

Developer Experience

• Write blog posts to describe how to use TC
• For developers and community:
  • ! easy to install (dumb defaults that work right of the box)
  • ! easy to deploy (include external services/ ingress/certbot) (FCP-50, FCP-51) (terraform)
  • ! easy to bootstrap new repo integration: quickstart /sample configs
• ! simple steps to run it locally: docker-compose / kind / include documentation

lotas

Keybase proof

I hereby claim:

• I am lotas on github.
• I am lotask (https://keybase.io/lotask) on keybase.
• I have a public key ASDBuLZ35wSiEoudq33lE9fcwwv_yNxYimN5YyhMmp4ySQo

To claim this, I am signing this object:

lotas

/* global phantom, console */
'use strict';

var system = require('system');
if (system.args.length < 3) {
    console.error('I need more arguments: render-image [template] [outputFilename]');
    phantom.exit(1);
}

var page = require('webpage').create();

lotas

#include <stdio.h>
#include <stdlib.h>

int main(int argc, char **argv) {

    if (argc < 2) {
        printf("No input\n");
        return 1;
    }

lotas

{
  "VCALENDAR": [
    {
      "VERSION": "2.0",
      "PRODID": "-//f1timing v1.0//EN",
      "NAME": "Formula One Official Calendar",
      "X-WR-CALNAME": "Formula One Official Calendar",
      "VEVENT": [
        {
          "UID": "f1cal48@formula1.com",