Skip to content

Instantly share code, notes, and snippets.

View kcmannem's full-sized avatar
◾️

Krishna Mannem kcmannem

◾️
View GitHub Profile
@kcmannem
kcmannem / stream_in_out_linux_zstd.go
Created May 4, 2019 21:41
baggage claim with zstd streaming
package main
import (
//"bufio"
"bytes"
"fmt"
"github.com/datadog/zstd"
"io"
"io/ioutil"
"os"
@kcmannem
kcmannem / y.md
Last active July 3, 2019 16:47
why zstd breaks

Doing some manual tests of baggageclaim on both linux and darwin.

#DARWIN

In Darwin when we intialize a volume with contents and call stream out with zstd encoding. The output is malformed silently, theres no error thrown by the library for us to detect this. idk why its happening.

/t/test $ curl -X PUT -H "Accept-Encoding: zstd" localhost:7788/volumes/hello/stream-out > thing.zst
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
# Cost of Concourse
If we're able to disect and expose the costs for running Concourse. We can better answer customer questions as to why they're spending so much for this tool. Costs that are caused by running a customers workload may get grouped as a cost for running Concourse itself. By having a framework which decomposes the fixed, variable, and marginal costs, we can better nagivate and control this conversation.
# Types of Cost
By design Concourse has a set of cluster manangment components that drives costs up when compared to similar sized worker pool on other build systems. We can classify this as the __Fixed Cost__ for running Concourse. Regardless of the deployment size, at minimum this cost has to be payed. The supported deployment method of Concourse for our customers is through BOSH, so we mustn't forget it's costs as well.
This is what I see as the minium well running deployement scheme: