Skip to content

Instantly share code, notes, and snippets.

@wheresrhys
Last active February 3, 2016 21:01
Show Gist options
  • Select an option

  • Save wheresrhys/569c0e8c6757550a4976 to your computer and use it in GitHub Desktop.

Select an option

Save wheresrhys/569c0e8c6757550a4976 to your computer and use it in GitHub Desktop.
Keen Query architecture (aka what the hell was I thinking)

The keen-query object

This has a number of methods which construct query parameters for keen e.g. .group(), .filter(). These are all shorthands for the various bits of keen urls In addition there are a handful of special methods, the most significant of which are

  • clone, doing exactly as you'd expect
  • getInstance, returns either a clone (if the kq is already populated with data) or this if still constructing an unexecuted query. keen-query objects are immutable once they've fetched data from keen
  • getTable, gets the data retrieved by the keen-query instance
  • print(method) - converts the keen-query to a variety of things e.g a url to query keen directly. Most significantly, for some of the methods (e.g. ascii, json) it will fetch data from keen and render it appropriately

The table object

Once data is retrieved from keen (or in, theory, via transform utilities for redshift data) it is put into a table object, with a few properties, including

  • axes - an array of the axes of the table, with te property name and values they take
  • data - a flat structure representing a multiple dimensional matrix e.g. data[4,25]=12.
  • humanize() - a method which converts the flat structure to a headings row, and a number of data rows (similar to html table)
    The advantage of the flat structure is that it's easy to make immutable so once the data is fetched once it can be reused over and over. It's also safer and easier to work with than nested arrays

Keen query aggregators

These are objects which contain

  • one or more keen-query objects
  • a rule for combining them (which generally acts on the table held in each keen-query)
  • a data table of their own, the result of the above operation
  • much of the same interface as keen-query objects, delegating the methods to each of teh keen-query's it contains The intention is that a consumer of something returned by KeenQuery.build() (the method that converts a query string into an object) can treat th eresult the same whether it's a simple query or an aggregated one

How keen-query strings relate to all the above

Each string of the form thing->method1()->method2()... is converted into a keen-query object using KeenQuery.build(). Strings of the form e.g. @ratio(thing->..., thing2->...) are converted into an aggregation of the multiple queries it contains. These can be nested indefinitely (though 1 or 2 levels deep is all anyone would ever need probably)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment