Skip to content

Instantly share code, notes, and snippets.

View tobz's full-sized avatar
💁‍♂️
it me

Toby Lawrence tobz

💁‍♂️
it me
View GitHub Profile
use std::sync::{Arc, atomic::{AtomicU64, Ordering}};
use dyn_clone::DynClone;
use crate::IntoF64;
/// A counter handler.
pub trait CounterFn: DynClone {
/// Increments the counter by the given amount.
fn increment(&self, value: u64);
use std::sync::{
Arc,
atomic::{AtomicU64, Ordering},
};
struct Key;
// This example just focuses on counters to reduce the boilerplate for sketching this out...
// but essentially, we would go from "register a key and then call the operation with that key"
// to something more like "register a metric, and call the operation on the metric handle."

Keybase proof

I hereby claim:

  • I am tobz on github.
  • I am tobz (https://keybase.io/tobz) on keybase.
  • I have a public key ASDe76wiWlZWIa0DoButTyswesb5WmgTFoN79ggGgIURBwo

To claim this, I am signing this object:

@tobz
tobz / bpf_prog.c
Last active February 20, 2021 16:44
#include <uapi/linux/ptrace.h>
struct in_flight_op_data_t
{
u64 start;
int conn_id;
int type;
};
struct dist_key_t
@tobz
tobz / main.rs
Last active January 30, 2021 13:27
use futures::future::try_join_all;
use http::{Method, Request};
use hyper::{Body, Client as HyClient, Uri};
use log::info;
use tokio::time::{sleep, Duration, Instant};
use tokio::sync::oneshot;
use tower::{service_fn, Service, ServiceBuilder, ServiceExt};
use std::sync::Arc;
#[tokio::main]
@tobz
tobz / before.txt
Created October 25, 2020 21:08
metrics const fast path testing
macros/uninitialized/no_labels
time: [1.5263 ns 1.5276 ns 1.5300 ns]
Found 10 outliers among 100 measurements (10.00%)
1 (1.00%) low mild
5 (5.00%) high mild
4 (4.00%) high severe
macros/uninitialized/with_static_labels
time: [1.5107 ns 1.5120 ns 1.5136 ns]
Found 16 outliers among 100 measurements (16.00%)
3 (3.00%) low mild
type CowString = Cow<str, 'static>;
pub enum Label {
// Key and value are both static and do not change ever.
Static(CowString, CowString),
// Key is always static, but value can be changed after construction.
Dynamic(CowString, Option<CowString>),
}
error[E0502]: cannot borrow `self.buf` as mutable because it is also borrowed as immutable
--> src/lib.rs:137:13
|
137 | self.buf.reserve(1024);
| ^^^^^^^^^^^^^^^^^^^^^^ mutable borrow occurs here
...
150 | match parse_ascii_response(&self.buf) {
| --------- immutable borrow occurs here
...
154 | return Ok(response);
@tobz
tobz / client.rs
Last active August 10, 2020 13:42
pub struct Client {
buf: BytesMut,
last_read_n: Option<usize>,
conn: Connection,
}
impl Client {
pub async fn get_response(&mut self) -> Result<Response<'_>, Status<'_>> {
// If we serviced a previous request, advance our buffer forward.
if let Some(n) = self.last_read_n {
Using DDSketch with the default values (2048 bins, 0.01 alpha, 1.0e-19 minimum), I find that storing tightly grouped/high sigfig values (i.e. 0.750064549 through 0.755188464) results in capturing all values in a single bin, which leads to all quantiles reporting the minimum value.
Playing with the alpha lets me get reasonable quantiles. In this case, I switched alpha to be 0.000001. My understanding is that alpha is directly related, or closely related, to the relative error guarantees, which makes sense for explaining how the values can get blackholed into a single bin without a small enough alpha.
What isn't clear to me is how to empirically derive a correct alpha value based on expected values, and how these values would effect measurements when the values in the sketch are not so tightly grouped i.e. am I hurting some measurements and not hurting others?
Example of my use case:
>>> from ddsketch.ddsketch import DDSketch
>>> sketch = DDSketch()