gianm

Joins

About this document

This is a design document describing the mechanics of joins in Druid. It is related to the initial join support proposal (apache/druid#8728) but covers different things: the proposal is more about motivation, design considerations, scope, and direction. This document is more about implementation.

Virtual join segments

Mechanics

gianm

1) from:([email protected]) to:([email protected])
Do this: Skip Inbox, Apply label "apache-gitbox"

2) to:([email protected]) "This is an automated email from the ASF dual-hosted git repository." "https://gitbox.apache.org/repos"
Do this: Skip Inbox, Apply label "apache-gitbox"

3) ((to:([email protected]) OR to:([email protected])) -from:[email protected] -"This is an automated email from the ASF dual-hosted git repository") -"https://gitbox.apache.org/repos"
Do this: Skip Inbox, Apply label "apache-druid-dev"	

gianm

#!/bin/bash -eux

#
# Backports patches from master to a release branch.
#
# Usage:
#  export GIT_TOKEN=...
#  druid-backport [branch] [pr-number], like: "druid-backport 0.12.1 5706"
#
# Bugs:

gianm

  84 import com.google.common.annotations.VisibleForTesting;
   1 import com.google.common.base.CharMatcher;
  53 import com.google.common.base.Charsets;
 260 import com.google.common.base.Function;
  14 import com.google.common.base.Functions;
  52 import com.google.common.base.Joiner;
   9 import com.google.common.base.Objects;
  74 import com.google.common.base.Optional;
 446 import com.google.common.base.Preconditions;
 134 import com.google.common.base.Predicate;

gianm

<code_scheme name="Calcite style" version="173">
  <option name="OTHER_INDENT_OPTIONS">
    <value>
      <option name="INDENT_SIZE" value="2" />
      <option name="TAB_SIZE" value="2" />
    </value>
  </option>
  <option name="LINE_SEPARATOR" value="&#xA;" />
  <option name="GENERATE_FINAL_LOCALS" value="true" />
  <option name="GENERATE_FINAL_PARAMETERS" value="true" />

gianm

#!/usr/bin/env python

# Copyright 2017 Imply Data, Inc.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
#     http://www.apache.org/licenses/LICENSE-2.0
#

gianm

See http://druid.io/docs/0.11.0-rc2/operations/tls-support.html for reference docs.

Quick start:

1. Add "simple-client-sslcontext" to druid.extensions.loadList in common.runtime.properties.
2. Add the properties below to common.runtime.properties.
3. For local testing, create a new self-signed certificate using the following command (but for production you should use a real certificate signed by a CA). Note that with these certificates the important part is the CN, which must match the hostname of your server. The command for testing purposes is: openssl req -newkey rsa:2048 -nodes -keyout key.pem -x509 -days 3650 -out cert.pem
4. Create pkcs12 bundle with: openssl pkcs12 -inkey key.pem -in cert.pem -export -out key.p12 -name druid
5. Create keystore: keytool -importkeystore -destkeystore imply-keystore.jks -srckeystore key.p12 -srcstoretype PKCS12
6. Create truststore: keytool -import -alias druid -file cert.pem -keystore imply-truststore.jks

gianm

#!/bin/bash -eux

# Note: assumes "base" and "milestone" are the same (for the backport PR)

base="$1"
pr="$2"
myrepo="gianm"
backport="backport-$pr-to-$base"

if [ "$(curl -u "$myrepo:$GIT_TOKEN" -s https://api.github.com/repos/druid-io/druid/pulls/"$pr" |jq -r '.merged')" != "true" ]

gianm

  @Test
  public void testGroupByWithLookupDimensionSpec()
  {
    Map<String, String> map = new HashMap<>();
    map.put("automotive", "health105");
    map.put("business", "health20");
    map.put("entertainment", "travel47");
    map.put("health", "health55");
    map.put("mezzanine", "health09");
    map.put("news", "health0000");

gianm

dataSources:
  - spec:
      dataSchema:
        dataSource: druid-metrics
        parser:
          type: string
          parseSpec:
            format: json
            timestampSpec:
              column: timestamp