using uglify to turn javascript functions into DynamoDB query language

api.js

// just a quick brainstorm for my dynamo API: https://github.com/jed/dynamo

// is it worth it to use uglify-js to parse functions into ASTs, to be transformed into dynamo's non-standard query language?

// the good:
// - no need to learn new API, just use javascript to query
// - API ends up resembling well-known .map and .filter style
// - would be entirely optional, compiling into specifiable query objects

// the bad:
// - kinda hacky
// - only a subset of JS could be supported
// - would be difficult to query by variable

// would love to hear some feedback at @jedschmidt

items.filter("age", "GT", 18) // would become
items.filter(function(x){ return x.age > 18 })

items.filter("age", "LT", 18) // would become
items.filter(function(x){ return x.age < 18 })

items.filter("age", "GE", 18) // would become
items.filter(function(x){ return x.age >= 18 })

items.filter("age", "LE", 18) // would become
items.filter(function(x){ return x.age <= 18 })

items.filter("age", "EQ", 18) // would become
items.filter(function(x){ return x.age == 18 })

items.filter("age", "NE", 18) // would become
items.filter(function(x){ return x.age != 18 })

items.filter("age", "NULL") // would become
items.filter(function(x){ return !("age" in x) })

items.filter("age", "NOT_NULL") // would become
items.filter(function(x){ return "age" in x })

items.filter("tags", "CONTAINS", "fun") // would become
items.filter(function(x){ return x.tags.indexOf("fun") >= 0 })

items.filter("tags", "NOT_CONTAINS", "fun") // would become
items.filter(function(x){ return x.tags.indexOf("fun") < 0 })

items.filter("name", "BEGINS_WITH", "Mc") // would become
items.filter(function(x){ return x.name.indexOf("Mc") == 0 })

items.filter("name", "IN", ["peter", "paul", "mary"]) // would become
items.filter(function(x){ return ["peter", "paul", "mary"].indexOf(x.name) >= 0 })

items.filter("age", "BETWEEN", [18, 30]) // would become
items.filter(function(x){ return x.age >= 18 && x.age <= 30 })

@evilhackerdude:

the (pretty lousy) docs don't seem to indicate that filters can be combined, but if they could i'd imagine it'd be limited to &&.

i realize the queries are limited, but that's the price you pay with dynamo. i guess the idea here is that you can provide a simple M/R-like interface. any queries too complex would just throw. so you could have an immediately grokkable hello world like this:

dynamo = require("dynamo")
db = dynamo.createClient()

db.tables
  .get("recipes")
  .items
    .filter(function(x){ x.user == "jim" && x.ingredients.indexOf("bacon") >= 0 })
    .map(function(recipe){ return {name: recipe.name, time: recipe.time } })
    .fetch(console.log)

the filter call sets the query parameters, and the map call could be used to tell dynamo which attributes to return, to save bandwidth.

@jed

.filter(function(x){ x.user == "jim" && x.ingredients.indexOf("bacon") >= 0 })

What about operators where there are many obvious different ways to express the same thing?
For example, BEGINS_WITH could be done with match, indexOf, slice + comparison among others.
You would have to document a list of accepted expressions even though at first glance one would expect every possible way to work.

I really liked your idea of translating the concepts without the AST stuff.
Doesn't go into the realm of leaky ORM abstractions and is much better than the raw dynamo API.

@jed - I agree with the general sentiment of not hiding too much after having worked with too many ORMs and query extensions (ie, LINQ) and spending far too long debugging the twisted ASTs they generate. DynamoDB is one of those beasts where it pays to have an understanding of exactly what's happening under the hood given its limited interface and potential for costly hidden queries.

So far my favourite is the "translate the concepts but scrap the ASTs" idea. BETWEEN could potentially be >=<= to distinguish/clarify it, but the presence of the array may be enough to make it clear anyway.

Personally, I don't mind the approach taken by @mranney w/ Redis where each command maps to a function. Perhaps that's because it's very easy to jump to and from the Redis CLI, where you get familiarity with the commands.

So with DynamoDB, something like:

db.get('recipes').where('name').beginsWith('mc').fetch('name', 'time'); // or .select('name', 'time')

or this would be shorter, but doesn't scan quite as well:

db.get('recipes').beginsWith('name', 'mc').fetch('name', 'time');

I realise this is veering from your filter/map idea - but I'm just not sure how far you can stretch the native feel of enumerations to a rigid database API.

thanks for the feedback, all.

@evilhackerdude, you're right. there's no way to enumerate the myriad ways that someone can check a substring, even if i do limit folks to indexOf.

@mhart, i think i'll start with generic query, scan, and filter methods that take the comparison operator names like above, and eventually specialize it into methods if it seems natural.